Web Demo audio output has breaks

[siddharth1712]

I ran the web demo on my system which has a a single A100 GPU. But the output audio is breaking after every word or so. Is this expected? How can I improve or optimize this?

[lxchtan]

Inference is currently slow (RTF > 1 on a single A100). Suggest using two GPUs or waiting for our upcoming vLLM integration for better performance.

[siddharth1712]

Okay thank you for the information, can you also tell the timeline for the vLLM integration?

[Mousaic]

Inference is currently slow (RTF > 1 on a single A100). Suggest using two GPUs or waiting for our upcoming vLLM integration for better performance.

I found even using two gpus,Inference still slow. dit not change much

[keunlee]

smaller inference model (doesn't seem to be available at the moment) and/or possibly adjusting the audio output buffer? I've still yet to try that, and see what happens.

[yangyyt]

Inference is currently slow (RTF > 1 on a single A100). Suggest using two GPUs or waiting for our upcoming vLLM integration for better performance.

Hello, may I ask what the current progress is?