feat(gnani): add X-Source/X-API-Request-ID headers and widen audio co… - #6743
feat(gnani): add X-Source/X-API-Request-ID headers and widen audio co…#6743Gnani-AI-Mintlify wants to merge 1 commit into
Conversation
…nfig options Sets X-Source: livekit and a generated X-API-Request-ID on every outbound STT/TTS request (REST, SSE, and WebSocket), so usage originating from this plugin can be attributed correctly downstream, matching the identity-header convention used by the other transports in Gnani's Vachana platform. Also widens the TTS encoding/container/bitrate/sample-rate Literals in models.py to match what the API actually supports (pcm_mulaw/pcm_alaw, mulaw/alaw, 32k/64k bitrates, 24000/48000 Hz). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| GnaniTTSEncodings = Literal["linear_pcm", "oggopus", "pcm_mulaw", "pcm_alaw"] | ||
| GnaniTTSContainers = Literal["raw", "mp3", "wav", "ogg", "mulaw", "alaw"] |
There was a problem hiding this comment.
🔴 Newly allowed mu-law/A-law audio formats produce broken or silent speech output
Mu-law and A-law audio formats are now offered as valid choices (GnaniTTSEncodings/GnaniTTSContainers at livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/models.py:54-55) even though the playback path cannot interpret them, so anyone selecting them hears noise or nothing at all.
Impact: Users who pick the newly advertised telephony audio formats get garbled or missing agent speech.
How the container string flows into the audio emitter and decoder
_mime_type() in livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/tts.py:249-252 maps any container other than raw to audio/<container>, producing audio/mulaw / audio/alaw. AudioEmitter.initialize (livekit-agents/livekit/agents/tts/tts.py:883-888) only treats audio/pcm and audio/raw as raw PCM, so these are sent to AudioStreamDecoder, whose MIME→libav table (livekit-agents/livekit/agents/utils/codecs/decoder.py:46-62) has no entry for mu-law/A-law; PyAV then has to auto-detect a headerless G.711 stream, which fails.
The other combination is equally broken: encoding="pcm_mulaw" with container="raw" yields audio/pcm, so 8-bit companded samples are pushed through as 16-bit linear PCM (sample_width stays 2 in GnaniTTSOptions), producing loud noise at half the expected duration.
Supporting these formats requires decoding G.711 to linear PCM in the plugin (or restricting the literals) rather than just widening the type unions.
Prompt for agents
The Gnani TTS plugin now advertises mu-law/A-law encodings and containers, but nothing in the plugin or the shared audio pipeline can handle them. In livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/tts.py, _mime_type() maps container -> "audio/<container>", so container="mulaw"/"alaw" yields audio/mulaw / audio/alaw. AudioEmitter.initialize only treats audio/pcm and audio/raw as raw PCM, and AudioStreamDecoder's MIME table (livekit-agents/livekit/agents/utils/codecs/decoder.py) has no mapping for G.711, so PyAV must auto-detect a headerless stream and will fail. Conversely encoding="pcm_mulaw" with container="raw" maps to audio/pcm and 8-bit companded bytes get played back as 16-bit linear PCM. Either add explicit G.711 decoding in the plugin (convert mu-law/A-law bytes to linear PCM before pushing to the emitter, and keep the emitted mime type as audio/pcm with the right sample width), or don't expose these values in GnaniTTSEncodings/GnaniTTSContainers until the pipeline supports them.
Was this helpful? React with 👍 or 👎 to provide feedback.
3 participants
No description provided.