Input streaming

Why input streaming

The classic LLM-to-voice pipeline buffers the model’s output into full sentences before synthesizing; the user hears nothing until the first sentence completes. Svara doesn’t need the full sentence. The model was trained on interleaved text/audio chunks where each chunk is generated with the complete history of everything already said plus a ~3-word lookahead peek of what comes next. A live pipeline therefore only holds back about three words:

• Speech starts mid-sentence, roughly ten words into the stream: measured 0.86 s to first audio versus 1.92 s for sentence buffering on the same simulated LLM stream
• Prosody stays continuous across chunk seams: every chunk hears its past, so there are no restarts or intonation resets at chunk boundaries
• With a real LLM (streaming at typical API cadence), audio began ~450 ms after the LLM’s first token

Two WebSocket endpoints expose this: a native one (recommended) and an ElevenLabs-compatible one for existing EL integrations. Both authenticate with your usual header on the upgrade request; see Authentication.

Native WebSocket

Connect with query parameters, send text fragments as JSON messages the moment they’re produced, and read binary frames back: raw PCM, 16-bit LE mono at 24 kHz. The server chunks at safe boundaries and keeps rolling text-and-audio context, so chunk N+1 continues chunk N’s prosody. Don’t pre-chunk or sentence-split yourself.

Messages you send

import asyncio, json, os, websockets
from openai import AsyncOpenAI

llm = AsyncOpenAI()
URL = "wss://platform.kenpathlabs.com/v1/audio/speech/stream-input?voice=3b5275e4f2&mode=eager"  # voice: Tara

async def speak(prompt: str):
    headers = {"Authorization": f"Bearer {os.environ['SVARA_API_KEY']}"}
    async with websockets.connect(URL, additional_headers=headers) as ws:

        async def feed():
            stream = await llm.chat.completions.create(
                model="gpt-4o-mini", stream=True,
                messages=[{"role": "user", "content": prompt}])
            async for event in stream:
                delta = event.choices[0].delta.content or ""
                if delta:
                    await ws.send(json.dumps({"text": delta}))
            await ws.send(json.dumps({"text": ""}))   # end of stream

        feeder = asyncio.create_task(feed())
        async for frame in ws:                        # binary = PCM16 @ 24 kHz
            if isinstance(frame, bytes):
                player.feed(frame)                    # your audio sink
            elif json.loads(frame).get("type") == "done":
                break
        await feeder

asyncio.run(speak("Explain how rainbows form, in three sentences."))

Eager mode

mode=eager is the lowest-latency configuration and maps directly onto how the model was trained. The server buffers incoming deltas at word granularity (mid-word token deltas are handled), and as soon as chunk_words + peek_words complete words exist it synthesizes the first chunk_words with the next few words attached as the lookahead peek:

With defaults, speech starts roughly 13 words into the LLM’s output; typically well before the first sentence ends.

ElevenLabs-compatible WebSocket

A drop-in implementation of the ElevenLabs realtime protocol, for existing EL integrations: the official EL SDKs’ WebSocket clients work unmodified. The full protocol is supported:

• BOS: {"text": " ", "generation_config": {"chunk_length_schedule": [120,160,250,290]}} (schedule values clamped 50-500; auto_mode supported)
• Text: {"text": "fragment ", "try_trigger_generation": true}, plus {"flush": true}; a bare space is a keepalive
• EOS: {"text": ""}
• Audio frames: {"audio": "<base64>", "alignment": …, "normalizedAlignment": …} in the negotiated output_format (query param, e.g. pcm_24000, mp3_44100_128), then {"isFinal": true} and a clean close (code 1000)
• inactivity_timeoutquery param: 20 s default, 180 s max

Character timings in alignment are chunk-relative and approximate: chunk boundaries are sample-exact, characters uniform within a chunk.

Tips

• Forward raw deltas. Don’t sentence-split, batch, or “clean up” the LLM stream client-side; the server’s chunker is the one that knows the model’s trained format.
• Buffer ~150 ms on playback. A small jitter buffer on your audio sink absorbs network variance without hurting perceived latency.
• Keep the socket warm for turns, not sessions. Open the connection when a reply starts, close on {"type": "done"}. Each connection is one utterance with one prosodic arc.
• Concurrent conversations need concurrent streams: each open socket counts against your plan’s concurrency limit (see Rate limits).