Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/io/speech/SpeechProviderResolver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,7 @@ export class SpeechProviderResolver extends EventEmitter {
{ id: 'openai-tts', kind: 'tts' as const, envVars: ['OPENAI_API_KEY'] },
{ id: 'elevenlabs', kind: 'tts' as const, envVars: ['ELEVENLABS_API_KEY'] },
{ id: 'deepgram-aura', kind: 'tts' as const, envVars: ['DEEPGRAM_API_KEY'] },
{ id: 'minimax-tts', kind: 'tts' as const, envVars: ['MINIMAX_API_KEY'] },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Preserve the live provider during resolver refresh.

After SpeechRuntime registers the real MiniMax provider, refresh() registers the same ID with provider: null. The existing registration is overwritten. resolveTTS() then returns null, and synthesis fails.

Preserve non-null registrations or add a MiniMax provider factory before adding this core entry. Add a refresh-after-construction regression test.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/io/speech/SpeechProviderResolver.ts` at line 450, Update the resolver
refresh logic around the MiniMax core entry in SpeechProviderResolver so
registering { id: 'minimax-tts', kind: 'tts' } cannot overwrite an existing
non-null provider with null; preserve the live registration or supply the
MiniMax provider factory before this entry is added. Add a regression test that
constructs SpeechRuntime, calls refresh(), and verifies resolveTTS() still
returns the registered MiniMax provider.

{
id: 'azure-speech-tts',
kind: 'tts' as const,
Expand Down
13 changes: 13 additions & 0 deletions src/io/speech/SpeechRuntime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import { SpeechSession } from './SpeechSession.js';
import { BuiltInAdaptiveVadProvider } from '../hearing/providers/BuiltInAdaptiveVadProvider.js';
import { ElevenLabsTextToSpeechProvider } from './providers/ElevenLabsTextToSpeechProvider.js';
import { OpenAITextToSpeechProvider } from './providers/OpenAITextToSpeechProvider.js';
import { MiniMaxTextToSpeechProvider } from './providers/MiniMaxTextToSpeechProvider.js';
import { OpenAIWhisperSpeechToTextProvider } from '../hearing/providers/OpenAIWhisperSpeechToTextProvider.js';
import type {
ProviderRequirements,
Expand Down Expand Up @@ -90,6 +91,18 @@ export class SpeechRuntime {
this.registry.registerTtsProvider(tts);
this.registerProviderInResolver(tts, 'tts');
}

const miniMaxApiKey = env['MINIMAX_API_KEY'];
if (miniMaxApiKey) {
const tts = new MiniMaxTextToSpeechProvider({
apiKey: miniMaxApiKey,
region: env['MINIMAX_REGION'] === 'china' ? 'china' : 'global',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

sed -n '1,140p' src/io/speech/SpeechRuntime.ts
printf '\n-- region references --\n'
rg -n --hidden --glob '!node_modules' --glob '!dist' --glob '!build' \
  "MINIMAX_REGION|MiniMaxTextToSpeechProvider|region:.*china|region.*global" .
printf '\n-- documentation references --\n'
rg -n --hidden --glob '!node_modules' --glob '!dist' --glob '!build' \
  "MINIMAX_REGION|MiniMax|MiniMax.*region|minimax.*region" README.md docs .github 2>/dev/null || true

Repository: framerslab/agentos

Length of output: 7653


🏁 Script executed:

printf '%s\n' '-- provider implementation --'
sed -n '1,180p' src/io/speech/providers/MiniMaxTextToSpeechProvider.ts
printf '%s\n' '-- provider tests around region behavior --'
sed -n '1,120p' src/io/speech/__tests__/MiniMaxTextToSpeechProvider.test.ts
printf '%s\n' '-- all environment/config documentation files --'
git ls-files | rg -i '(^|/)(readme|docs|env|config)|minimax|speechruntime'

Repository: framerslab/agentos

Length of output: 14341


🌐 Web query:

MiniMax API documentation domestic region value cn global endpoint MiniMax TTS

💡 Result:

MiniMax API service regions are distinguished by the platform where your account and subscription were purchased, rather than a simple configuration parameter [1][2]. You must match your API Key to the correct regional base URL [1][2]. For MiniMax Text-to-Speech (TTS) and other API services, use the following endpoints: Global/International Region: Base URL: https://api.minimax.io [1][2] Platform: https://platform.minimax.io [1] Mainland China Region (cn): Base URL: https://api.minimaxi.com [1][2] Platform: https://platform.minimaxi.com [1] Key implementation details: 1. Authentication: A Global API Key will fail if used with the Mainland China host (and vice versa) [1][2]. Ensure your credentials correspond to the platform where they were generated [1]. 2. TTS Endpoints: Once you have selected the correct base URL for your region, TTS endpoints are accessed via standard paths such as /v1/t2a_v2 (for synchronous HTTP) or /ws/v1/t2a_v2 (for WebSocket) [3][4][5]. 3. Tooling: If using tools like the MiniMax CLI (mmx-cli), the region is often auto-detected from the API Key, but it can be manually configured if necessary using: mmx config set --key region --value global # For international services mmx config set --key region --value cn # For mainland China services [6][7]

Citations:


🏁 Script executed:

printf '%s\n' '-- SpeechRuntime tests --'
rg -n -A18 -B8 "MINIMAX|MiniMax|autoRegisterFromEnv|register.*Tts" \
  src/io/speech/__tests__/SpeechRuntime.test.ts

printf '%s\n' '-- deterministic mapping probe --'
python3 - <<'PY'
from pathlib import Path
import re

source = Path("src/io/speech/SpeechRuntime.ts").read_text()
match = re.search(
    r"region:\s*env\['MINIMAX_REGION'\]\s*===\s*'china'\s*\?\s*'china'\s*:\s*'global'",
    source,
)
if not match:
    raise SystemExit("mapping expression not found")

def current(value):
    return "china" if value == "china" else "global"

for value in ["china", "cn", "CN", " cn ", "global", "unsupported"]:
    print(f"{value!r} -> {current(value)!r}")
PY

Repository: framerslab/agentos

Length of output: 3882


Map MINIMAX_REGION=cn to the China endpoint.

MINIMAX_REGION='cn' currently selects global, so China credentials use https://api.minimax.io instead of https://api.minimaxi.com. Normalize the value and map both cn and china to china. Reject unsupported values instead of defaulting to global.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/io/speech/SpeechRuntime.ts` at line 99, Update the region mapping in
SpeechRuntime to normalize MINIMAX_REGION and map both “cn” and “china” to the
China endpoint, while mapping only the supported global value to “global.”
Reject unsupported or missing values instead of silently defaulting to “global.”

Source: MCP tools

model: env['MINIMAX_TTS_MODEL'] ?? 'speech-2.8-hd',
voice: env['MINIMAX_TTS_VOICE'],
});
this.registry.registerTtsProvider(tts);
this.registerProviderInResolver(tts, 'tts');
}
}
}

Expand Down
219 changes: 219 additions & 0 deletions src/io/speech/__tests__/MiniMaxTextToSpeechProvider.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
import { EventEmitter } from "node:events";
import WebSocket from "ws";
import { describe, expect, it, vi } from "vitest";
import { MiniMaxTextToSpeechProvider } from "../providers/MiniMaxTextToSpeechProvider.js";

function response(body: unknown): Response {
return {
ok: true,
status: 200,
statusText: "OK",
json: vi.fn(async () => body),
} as unknown as Response;
}

describe("MiniMaxTextToSpeechProvider", () => {
it("synthesizes hex audio through the global HTTP endpoint", async () => {
const fetchImpl = vi.fn(
async (_input: string | URL | Request, _init?: RequestInit) =>
response({
data: { audio: "000102ff", status: 2 },
extra_info: { audio_length: 1250, usage_characters: 5 },
base_resp: { status_code: 0, status_msg: "success" },
}),
);
const provider = new MiniMaxTextToSpeechProvider({
apiKey: "test-key",
fetchImpl,
});

const result = await provider.synthesize("hello", {
voice: "English_expressive_narrator",
providerSpecificOptions: {
languageBoost: "English",
pronunciationDict: { tone: ["hello/hello"] },
audioSetting: { sample_rate: 32000 },
voiceModify: { pitch: 1 },
subtitleEnable: true,
},
});

expect(result.audioBuffer).toEqual(Buffer.from([0, 1, 2, 255]));
expect(result.durationSeconds).toBe(1.25);
const [url, init] = fetchImpl.mock.calls[0]!;
expect(url).toBe("https://api.minimax.io/v1/t2a_v2");
expect((init?.headers as Record<string, string>).Authorization).toBe(
"Bearer test-key",
);
expect(JSON.parse(init?.body as string)).toMatchObject({
model: "speech-2.8-hd",
text: "hello",
stream: false,
output_format: "hex",
language_boost: "English",
voice_setting: { voice_id: "English_expressive_narrator" },
audio_setting: { sample_rate: 32000, format: "mp3" },
pronunciation_dict: { tone: ["hello/hello"] },
voice_modify: { pitch: 1 },
subtitle_enable: true,
});
});

it("uses the China endpoint and downloads URL output", async () => {
const fetchImpl = vi
.fn(
(
_input: string | URL | Request,
_init?: RequestInit,
): Promise<Response> =>
Promise.resolve(undefined as unknown as Response),
)
.mockResolvedValueOnce(
response({
data: { audio: "https://cdn.example.com/audio.wav", status: 2 },
base_resp: { status_code: 0 },
}),
)
.mockResolvedValueOnce({
arrayBuffer: vi.fn(async () => Buffer.from("audio")),
} as unknown as Response);
const provider = new MiniMaxTextToSpeechProvider({
apiKey: "test-key",
region: "china",
fetchImpl,
});

const result = await provider.synthesize("hello", {
outputFormat: "wav",
providerSpecificOptions: { outputFormat: "url" },
});

expect(fetchImpl.mock.calls[0][0]).toBe(
"https://api.minimaxi.com/v1/t2a_v2",
);
expect(result.audioBuffer.toString()).toBe("audio");
expect(result.mimeType).toBe("audio/wav");
});

it("creates and queries asynchronous speech tasks", async () => {
const fetchImpl = vi
.fn(
(
_input: string | URL | Request,
_init?: RequestInit,
): Promise<Response> =>
Promise.resolve(undefined as unknown as Response),
)
.mockResolvedValueOnce(
response({
task_id: "task-1",
file_id: 42,
base_resp: { status_code: 0 },
}),
)
.mockResolvedValueOnce(
response({
task_id: "task-1",
status: "success",
file_id: 42,
base_resp: { status_code: 0 },
}),
);
const provider = new MiniMaxTextToSpeechProvider({
apiKey: "test-key",
fetchImpl,
});

await expect(provider.createAsync("long text")).resolves.toMatchObject({
task_id: "task-1",
});
await expect(provider.queryAsync("task-1")).resolves.toMatchObject({
status: "success",
});
expect(fetchImpl.mock.calls.map((call) => call[0])).toEqual([
"https://api.minimax.io/v1/t2a_async_v2",
"https://api.minimax.io/v1/query/t2a_async_query_v2",
]);
expect(JSON.parse(fetchImpl.mock.calls[1]![1]?.body as string)).toEqual({
task_id: "task-1",
});
});

it("runs the WebSocket start, continue, and finish protocol", async () => {
class Socket extends EventEmitter {
sent: string[] = [];
send(value: string) {
this.sent.push(value);
}
close() {}
}
const socket = new Socket();
const provider = new MiniMaxTextToSpeechProvider({
apiKey: "test-key",
webSocketFactory: (url, headers) => {
expect(url).toBe("wss://api.minimax.io/ws/v1/t2a_v2");
expect(headers.Authorization).toBe("Bearer test-key");
return socket as unknown as WebSocket;
},
});

const resultPromise = provider.synthesizeWebSocket("hello");
socket.emit(
"message",
JSON.stringify({
event: "connected_success",
base_resp: { status_code: 0 },
}),
);
socket.emit(
"message",
JSON.stringify({
event: "task_started",
base_resp: { status_code: 0 },
}),
);
socket.emit(
"message",
JSON.stringify({
event: "task_continued",
data: { audio: "0001ff" },
base_resp: { status_code: 0 },
}),
);
socket.emit(
"message",
JSON.stringify({
event: "task_finished",
base_resp: { status_code: 0 },
}),
);

await expect(resultPromise).resolves.toMatchObject({
audioBuffer: Buffer.from([0, 1, 255]),
mimeType: "audio/mpeg",
});
expect(socket.sent.map((value) => JSON.parse(value).event)).toEqual([
"task_start",
"task_continue",
"task_finish",
]);
});

it("rejects invalid hex audio", async () => {
const fetchImpl = vi.fn(
async (_input: string | URL | Request, _init?: RequestInit) =>
response({
data: { audio: "not-hex", status: 2 },
base_resp: { status_code: 0 },
}),
);
const provider = new MiniMaxTextToSpeechProvider({
apiKey: "test-key",
fetchImpl,
});

await expect(provider.synthesize("hello")).rejects.toThrow(
"invalid hex audio",
);
});
});
2 changes: 2 additions & 0 deletions src/io/speech/__tests__/SpeechRuntime.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ describe('SpeechRuntime', () => {
env: {
OPENAI_API_KEY: 'sk-openai',
ELEVENLABS_API_KEY: 'sk-elevenlabs',
MINIMAX_API_KEY: 'sk-minimax',
},
});

Expand All @@ -25,6 +26,7 @@ describe('SpeechRuntime', () => {
expect(runtime.getProvider('openai-whisper')).toBeDefined();
expect(runtime.getProvider('openai-tts')).toBeDefined();
expect(runtime.getProvider('elevenlabs')).toBeDefined();
expect(runtime.getProvider('minimax-tts')).toBeDefined();
});

it('should hydrate speech providers from the extension manager', async () => {
Expand Down
11 changes: 11 additions & 0 deletions src/io/speech/__tests__/providerCatalog.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,17 @@ describe('providerCatalog', () => {
expect(entry!.kind).toBe('stt');
});

it('should expose MiniMax HTTP, async, and WebSocket TTS capabilities', () => {
const entry = findSpeechProviderCatalogEntry('minimax-tts');
expect(entry).toMatchObject({
kind: 'tts',
streaming: true,
defaultModel: 'speech-2.8-hd',
envVars: ['MINIMAX_API_KEY'],
});
expect(entry?.features).toEqual(expect.arrayContaining(['async', 'websocket']));
});

it('should mark nvidia-nemo as unavailable (planned but not implemented)', () => {
const entry = findSpeechProviderCatalogEntry('nvidia-nemo');
expect(entry).toBeDefined();
Expand Down
1 change: 1 addition & 0 deletions src/io/speech/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ export * from './providers/OpenAITextToSpeechProvider.js';
export * from './providers/ElevenLabsTextToSpeechProvider.js';
export * from './providers/DeepgramTextToSpeechProvider.js';
export * from './providers/AzureSpeechTTSProvider.js';
export * from './providers/MiniMaxTextToSpeechProvider.js';
// STT/VAD providers have moved to the hearing/ module
export * from '../hearing/providers/OpenAIWhisperSpeechToTextProvider.js';
export * from '../hearing/providers/BuiltInAdaptiveVadProvider.js';
Expand Down
11 changes: 11 additions & 0 deletions src/io/speech/providerCatalog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,17 @@ export const SPEECH_PROVIDER_CATALOG: readonly SpeechProviderCatalogEntry[] = [
defaultVoice: 'aura-2-thalia-en',
features: ['cloud', 'tts', 'streaming', 'low-latency'],
},
{
id: 'minimax-tts',
kind: 'tts',
label: 'MiniMax TTS',
envVars: ['MINIMAX_API_KEY'],
local: false,
streaming: true,
description: 'Speech synthesis via MiniMax HTTP, async, and WebSocket APIs.',
defaultModel: 'speech-2.8-hd',
features: ['cloud', 'tts', 'streaming', 'websocket', 'async'],
},
{
id: 'google-cloud-tts',
kind: 'tts',
Expand Down
Loading