three.ws Audio

@three-ws/audio-mcp

Give a 3D AI agent a voice and a face — text-to-speech, speech-to-text, audio-to-face lipsync, and the motion-capture clip library, from any MCP client.

A Model Context Protocol server that exposes the three.ws voice and animation pipeline over stdio. Make an avatar speak, let users talk back, turn that speech into a per-frame lipsync track, and browse the recorded motion-capture library — everything you need to drive a 3D agent's mouth and movement.

The speech, recognition, and lipsync lanes run on free NVIDIA NIM models (Magpie TTS, Riva ASR, Audio2Face-3D), with OpenAI as the paid TTS backstop — the same provider policy as the three.ws platform. No key required to start; point THREE_WS_BASE at a deployment and go.

Install

npm install @three-ws/audio-mcp

Or run with npx (no install):

npx @three-ws/audio-mcp

Quick start

Claude Code, one line:

claude mcp add audio -- npx -y @three-ws/audio-mcp

Claude Desktop / Cursor (claude_desktop_config.json or mcp.json):

{

"mcpServers": { "audio": { "command": "npx", "args": ["-y", "@three-ws/audio-mcp"] } } }

Inspect the surface with the MCP Inspector:

npx -y @modelcontextprotocol/inspector npx @three-ws/audio-mcp

Tools

All five tools are read-only: they transform input (synthesize/recognize/animate) or read live library data, never mutating platform state. None are idempotent — the audio bytes, transcript, and catalog move between calls.

Input parameters

`text_to_speech` — text (required, 1–4096 chars), voice (default nova), format (mp3|opus|aac|flac|wav|pcm, default mp3), model (OpenAI backstop only), language (BCP-47, default en-US), speed (0.5–2.0, OpenAI lane only).

`speech_to_text` — audio (required, base64 or data: URL), format (wav|pcm|flac|ogg, default wav), language (BCP-47, default en-US), sampleRate (Hz, PCM only), words (bool), model (advanced).

`audio_to_face` — text or audio (one required), format (wav|pcm, audio path), sampleRate (PCM only), voice / language (text path).

`motion_capture_clips` — kind (face|pose|hand|vmc), include_public (bool, authenticated only), limit (1–100, default 50), cursor.

`motion_capture_clip` — id (required, clip UUID).

Examples

// text_to_speech

{ "text": "Hello, I'm your three.ws agent.", "voice": "nova" }

{ "ok": true, "voice": "Magpie-Multilingual.EN-US.Aria", "model": "magpie-tts-multilingual", "format": "wav", "mime": "audio/wav", "sizeBytes": 274476, "audio": "data:audio/wav;base64,UklGR…" }

// audio_to_face (text path — returns audio + the lipsync track together)

{ "text": "Watch my lips move." }

{ "ok": true, "audio": { "base64": "UklGR…", "contentType": "audio/wav", "format": "wav" }, "animation": { "fps": 30, "blendShapeNames": ["jawOpen", "mouthClose", "…"], "durationSec": 1.9, "frames": [ { "t": 0.0, "w": [0.0, 0.1, /* … */] } ] } }

A face frame is { t, w }: t = seconds from clip start, w = weights in the order of blendShapeNames (ARKit-52 naming). Play the audio and sample the track by the audio element's currentTime.

Notes on payloads

Audio crosses the wire as base64 (TTS returns it, ASR/A2F accept it). Both the synthesis input (text) and the recognition input (audio, ~8 MB cap) are bounded by the platform — keep clips to a spoken utterance rather than long recordings. For lipsync on audio you already have, pass wav/pcm directly; browser-recorded WebM/Opus must be decoded to WAV or PCM first (Riva and Audio2Face do not accept the WebM container).

Requirements

• Node.js >= 20.
• Network access to https://three.ws (or your own THREE_WS_BASE).

Environment variables

Links

• Homepage: https://three.ws
• Changelog: https://three.ws/changelog
• Issues: https://github.com/nirholas/three.ws/issues
• License: Apache-2.0, see LICENSE

Part of the three.ws SDK suite — 3D AI agents, on-chain identity, and agent payments.

Website · Changelog · GitHub