Skip to content

Add text-to-speech and text-to-audio task helpers for WaveSpeed provider - #2379

Open
chengzeyi wants to merge 1 commit into
huggingface:mainfrom
chengzeyi:wavespeed-audio-tasks
Open

Add text-to-speech and text-to-audio task helpers for WaveSpeed provider#2379
chengzeyi wants to merge 1 commit into
huggingface:mainfrom
chengzeyi:wavespeed-audio-tasks

Conversation

@chengzeyi

@chengzeyi chengzeyi commented Aug 16, 2026

Copy link
Copy Markdown

What

Adds text-to-speech and text-to-audio task support for the wavespeed inference provider:

  • WavespeedAITextToSpeechTask — sends the input text as text (the field WaveSpeed TTS endpoints such as wavespeed-ai/vibevoice and chatterbox/text-to-speech expect), mirroring how the fal.ai TTS helper handles the same translation.
  • WavespeedAITextToAudioTask — reuses the base payload (prompt = inputs), matching WaveSpeed music/audio endpoints such as wavespeed-ai/ace-step/prompt-to-audio.

Both reuse the existing WaveSpeed submit-and-poll response handling already used by the image/video helpers.

Fix in textToSpeech()

textToSpeech() called providerHelper.getResponse(res, undefined, undefined, ...) without url/headers. That works for providers that return audio synchronously (fal, replicate), but WaveSpeed's helper polls an async task result and requires them. This PR passes url/headers the same way textToAudio() already does; providers that ignore these arguments are unaffected.

Why

WaveSpeed serves several audio models whose HF Hub counterparts currently have no inference provider at all (e.g. microsoft/VibeVoice-1.5B, Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice, k2-fsa/OmniVoice, ACE-Step/Ace-Step1.5, MiniMaxAI/MiniMax-Music3). We'd like to register those model mappings, which requires client support for the tasks first.

Testing

  • pnpm run check, pnpm run lint, pnpm run build pass in packages/inference.
  • Added textToSpeech / textToAudio cases to the WaveSpeed block in InferenceClient.spec.ts.
  • Verified live against the WaveSpeed production API through the new helpers: textToSpeech (VibeVoice) returned an audio/mpeg blob, textToAudio (ACE-Step prompt-to-audio) returned an audio/wav blob.

I'm the maintainer on the WaveSpeed side (we operate the wavespeed provider). cc @Wauplin @SBrandeis @hanouticelina

🤖 Generated with Claude Code

https://claude.ai/code/session_0139Zu5TfHzGVyPeatY9HJAR


Note

Low Risk
Scoped provider additions plus aligning textToSpeech with an existing textToAudio pattern; no auth or data-model changes.

Overview
Adds WaveSpeed support for text-to-speech and text-to-audio, wiring new task helpers into the provider registry and reusing the existing submit-and-poll Blob flow used for other WaveSpeed media tasks.

WavespeedAITextToSpeechTask maps client inputs to the API text field (for endpoints like VibeVoice); WavespeedAITextToAudioTask keeps the default prompt-style payload for music/prompt-to-audio routes. Integration tests cover VibeVoice TTS and ACE-Step text-to-audio.

textToSpeech() now passes request URL and headers into getResponse, matching textToAudio(), so async providers (including WaveSpeed) can poll task status; synchronous providers that ignore those args stay unchanged.

Reviewed by Cursor Bugbot for commit 7e94f0c. Bugbot is set up for automated code reviews on this repo. Configure here.

WaveSpeed serves TTS models (e.g. microsoft/VibeVoice-1.5B) and music/audio
generation models (e.g. ACE-Step) behind the same async task API already used
by its image/video helpers. This adds WavespeedAITextToSpeechTask (sends the
input text as `text`) and WavespeedAITextToAudioTask (sends it as `prompt`,
matching the existing base payload), both reusing the shared polling logic.

textToSpeech() previously called getResponse() without url/headers, which only
works for providers that return audio synchronously; WaveSpeed's helper needs
them to poll the task result. Pass them the same way textToAudio() already does
(providers that ignore them are unaffected).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139Zu5TfHzGVyPeatY9HJAR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant