Your Dictation Feature Doesn’t Need a WebSocket

Open any tutorial on adding voice input to an app, and you’ll end up in the same place: a WebSocket, a session lifecycle, a partial-transcript handler, and some end-of-turn logic you now have to tune.

Then look at what you’re actually transcribing. A two-second clip. The user held a button, said “remind me to email Priya about the Q3 numbers,” and let go. The audio finished recording before your first packet went out.

There’s nothing to stream. You have a complete file, and you want a complete transcript. That’s a request and a response — the oldest shape in web development — and reaching for a streaming API here means paying for connection management you never wanted.

So let’s build the request/response version instead, and then look at where the milliseconds go, because that part turns out to be more interesting than the code.

Do you need a WebSocket for short audio clips?

No. If the audio is already finished recording, a single HTTP request is the right shape, and streaming buys you nothing.

Here’s the honest version of the three-way choice:

Streaming is for audio that doesn’t exist yet. Live captions, a voice agent listening for a caller to finish a thought, a meeting notetaker running for forty minutes. You need partial results as words arrive, so you hold a connection open, and the API pushes text back at you.

That’s a real problem worth a real WebSocket.

Async batch is for audio that’s long and finished. Upload a podcast, get a job ID, poll or wait for a webhook, retrieve the transcript. The latency is measured in seconds to minutes and nobody minds, because nobody is standing there waiting.

Sync is the one people forget exists. The audio is finished, it’s short, and somebody is standing there waiting. Push-to-talk dictation. A voicemail. An IVR menu response. A voice agent turn where you’re doing your own turn detection and just want the utterance transcribed.

Push-to-talk lands squarely in the third bucket, and most implementations use the first one. The tell is when you find yourself writing code to detect the end of a turn — for audio where the user already told you the turn was over by lifting their finger.

The Sync Speech-to-Text API exists for that third case. You POST a clip, you get a transcript back in the same response, and the median round trip is about 134 milliseconds.

How do you transcribe a short clip in a single HTTP request?

Send the audio to POST https://sync.assemblyai.com/transcribe and read the transcript out of the JSON response. There’s no job ID, no polling loop, and no session to open or close.

The SDKs wrap it in one method:

import os

from assemblyai.sync.v1 import SyncTranscriber

transcriber = SyncTranscriber(api_key=os.environ["ASSEMBLYAI_API_KEY"])
result = transcriber.transcribe("./utterance.wav")
print(result.text)

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

const result = await client.sync.transcribe("./utterance.wav");
console.log(result.text);

If you’d rather not add a dependency to a mobile backend or an edge function, it’s a multipart form post:

curl -X POST https://sync.assemblyai.com/transcribe 
  -H 'Authorization: <YOUR_API_KEY>' 
  -H 'X-AAI-Model: universal-3-5-pro' 
  -F 'audio=@utterance.wav;type=audio/wav'

The X-AAI-Model header is required on every request. The API key goes in Authorization with no Bearer prefix.

What comes back:

{
  "text": "Remind me to email Priya about the Q3 numbers.",
  "words": [
    { "text": "Remind", "confidence": 0.98 },
    { "text": "me", "confidence": 0.99 }
  ],
  "confidence": 0.97,
  "audio_duration_ms": 2140,
  "session_id": "eb92c4ff-4bbb-429f-9b99-7279d7fe738f",
  "request_time_ms": 131.4
}

Two fields worth knowing about. request_time_ms is the server-side processing time, which is useful when you’re trying to work out whether a slow request was the model or the network — and you’ll want that distinction in a minute. And session_id is the first thing support asks for when something goes wrong, so log it on every request rather than only on failures.

Per-word timestamps are opt-in. Set timestamps: true and you get start and end in milliseconds on each word, at a small latency cost. Timings are exact or absent, never estimated: a word the model can’t align comes back without them. Leave the flag off and words carry text and confidence only. For a dictation box that’s usually what you want — you’re inserting text at a cursor, not building a karaoke player.

A few constraints to design around. Audio runs from 80 milliseconds to 120 seconds, files cap at 40 MB, and the API takes WAV or raw PCM S16LE — 16-bit only. WAV carries its sample rate and channel count in the header; raw PCM doesn’t, so sending PCM means passing sample_rate and channels in the config, and both are required. There’s no URL ingestion, so you’re sending bytes, not a link. And there’s a 30-second deadline on the request itself, which is worth knowing if you’re pushing clips near the two-minute ceiling. The full list is in the audio requirements.

Error handling deserves more than a try block that swallows everything. The API returns machine-readable codes — bad_audio, audio_too_short, audio_too_large, unsupported_media_type, capacity_exceeded, inference_timeout and a few more — and the 429 and 503 responses carry a Retry-After value you should actually respect:

import os

from assemblyai.sync.v1 import SyncTranscriber, SyncTranscriptError

with SyncTranscriber(api_key=os.environ["ASSEMBLYAI_API_KEY"]) as transcriber:
    try:
        result = transcriber.transcribe("./utterance.wav")
    except SyncTranscriptError as error:
        # 429/503 are transient — back off and retry, honoring retry_after.
        # 500/504 are safe to retry once.
        # 400/413/415 mean the audio or config is wrong. Fix it, don't retry.
        raise RuntimeError(
            f"{error.status_code}/{error.error_code}: {error}"
        ) from error

    print(result.session_id, result.text)

That’s the whole integration. Under fifty meaningful lines, including error handling, which is roughly the point.

Where do the milliseconds actually go?

Here’s the part nobody writes about, and it’s the thing that will actually determine whether your dictation feature feels instant.

On a streaming API, you open the connection once at the start of a session and amortize the setup across everything that follows. On a single-request API, there is no session to amortize against. Every cold request pays for connection setup before a single byte of audio moves.

Three things have to happen first:

1. DNS resolution — often cached, but a cold lookup is a round trip of its own.
2. TCP handshake — SYN, SYN-ACK, ACK. One round trip.
3. TLS handshake — ClientHello, certificate, key exchange, both sides derive session keys. One more round trip on TLS 1.3. Two on TLS 1.2.

None of that work depends on your audio. It’s pure plumbing, and it sits in front of every cold request. For a client near the serving region it’s a few tens of milliseconds. For a client on another continent, it can be well over 100 — which, against a ~134 ms transcription, means you could spend more time shaking hands than transcribing.

The fix is a /warm endpoint, and the trick is when you call it.

import os

import assemblyai as aai
from assemblyai.sync.v1 import SyncTranscriber

# Keep the idle connection alive long enough to cover the recording.
aai.settings.keepalive_expiry = 120

with SyncTranscriber(api_key=os.environ["ASSEMBLYAI_API_KEY"]) as transcriber:
    transcriber.warm()          # user presses the button
    # ... user speaks, you record ...
    result = transcriber.transcribe("./utterance.wav")   # no handshake here
    print(result.text)

GET /warm is an unauthenticated no-op that returns {"warm": "toasty"}. The response body is irrelevant. The entire value of the call is on the wire: making any request forces your HTTP client to resolve DNS, open the socket, and complete the TLS negotiation, and the resulting connection lands in your client’s pool. The /transcribe request that follows reuses it and starts uploading audio immediately.

The ideal moment to call it is when you know audio is coming but don’t have it yet — which in a push-to-talk UI is the exact instant the user presses the button. The handshake then runs concurrently with the recording. By the time they let go, the connection is open and warm, and you’ve moved the entire setup cost off the critical path and into time the user was going to spend talking anyway.

Two gotchas will quietly undo this. Pooled connections expire — httpx, for example, drops idle connections after five seconds by default — and the server closes its side after a few minutes of idle. Warm too early and you pay the handshake anyway, silently, with nothing in your logs to say so. And the warm and the transcribe have to share a connection pool: the same client object, the same base URL. The Sync API serves a global endpoint plus regional US and EU endpoints for data residency, and a connection warmed against one of them does nothing for a request sent to another.

This is also why request_time_ms earns its place in the response. If your end-to-end latency is 400 ms and request_time_ms says 130, you don’t have a model problem. You have a handshake problem, and now you know which one to go fix. There’s more on the mechanics in the connection pre-warming docs.

How do you improve accuracy on names, IDs, and jargon?

Short clips are the hardest thing to transcribe well, which is the uncomfortable irony of dictation. A long recording gives the model context to work with. Two seconds of “send it to Siobhan at Anthropic” gives it almost nothing.

The Sync API runs on Universal-3.5 Pro, and it takes three kinds of context — all optional, all in the same config part, none of them costing extra.

Keyterms (keyterms_prompt) bias the model toward specific tokens. Your user’s contact list, your product catalog, the acronyms your industry uses and nobody else does. The list caps at 2048 characters across all terms:

const result = await client.sync.transcribe("./utterance.wav", {
  keyterms_prompt: ["Siobhan", "Anthropic", "Q3 OKRs"],
});

Contextual prompting (prompt) describes the audio rather than instructing the model, in up to 4096 characters:

const result = await client.sync.transcribe("./utterance.wav", {
  prompt: "Voice memo dictated by a sales rep about a customer call.",
});

Keep it to one short block — a description, not a keyword dump, and not commands about punctuation or formatting, which are already handled. Note that a custom prompt replaces the managed default entirely, including its language steering, so if your audio isn’t English, say so in the description.

Conversation context (conversation_context) hands over the dialogue that came before this clip. It’s built for multi-turn exchanges where the current utterance only makes sense given the last one — an agent asks “what’s your order number?” and the caller answers with a string of digits and letters that could go several ways:

const result = await client.sync.transcribe("./utterance.wav", {
  conversation_context: [
    "Hi, thank you for calling. How can I help?",
    "I'd like to check on the status of my order.",
    "Got it. Do you know your order ID number?",
  ],
});

Turns go in chronological order, oldest first, with no speaker labels. It holds 100 turns or 4096 characters, and context over either cap is trimmed rather than rejected — the oldest turns drop off the front, so a long-running call degrades gracefully instead of erroring.

The advice that matters most here is counterintuitive: start with none of it. All three are robust to irrelevant input — the model stays grounded in the audio and won’t insert words that weren’t spoken — but a long keyterm list stuffed with common words invites overcorrection. Ship without them, find the terms your model actually gets wrong, then add only those. Details are in prompting and keyterms and conversation context.

Language selection is a separate knob. In the SDKs it’s language_codes, which always takes a list — one code for monolingual audio, several for code-switching. (Over raw HTTP the config field is language_code, and it accepts either a single code or an array.) Nineteen languages are supported, from English and Spanish through Japanese, Mandarin, Urdu, and Hebrew.

const result = await client.sync.transcribe("./utterance.wav", {
  language_codes: ["en", "es"],
});

How does this compare to other synchronous transcription endpoints?

Most major providers ship something in this shape. They differ mostly in how much audio they’ll accept and what they’ll take it as.

Google enforces both of its limits together — 60 seconds, 10 MB, or both, whichever you hit first.

The 60-second ceilings on Google and Azure are the practical constraint to watch. They’re fine for push-to-talk, where two to ten seconds is typical, and they fall over the moment someone dictates a long voice memo or you want to transcribe a full voicemail in one shot. Azure’s short-audio REST endpoint is also strict about formats — 16 kHz mono WAV or OGG Opus, nothing else — which means a resampling step in your pipeline if your recorder doesn’t already produce that.

The more useful question isn’t which ceiling is highest. It’s whether the provider treats short-clip transcription as a first-class path with its own latency work, or as a convenience wrapper on the batch pipeline. The presence of something like a pre-warming endpoint is a decent signal either way.

When should you not use a sync API?

Reach for something else in four cases, and it’s worth being blunt about them.

• The audio runs over two minutes. Use async transcription. Submit the file, get a webhook, move on.
• The audio doesn’t exist yet. Live captions, an agent that needs to barge in mid-sentence, anything where you need words before the speaker stops. That’s streaming, and no amount of clever request batching substitutes for it.
• You need speaker diarization. The Sync API doesn’t do it. Two-second dictation clips have one speaker, so this rarely bites, but if you’re transcribing a snippet of a conversation and need to know who said what, use async or streaming.
• You need PII redaction. Also not available on Sync. If you’re handling audio that requires redaction at the transcription layer, that’s an async workflow.

The pattern in that list: the sync path is deliberately narrow. It does one thing — short finished audio, fast, in one round trip — and drops the features that don’t fit that shape. That’s the tradeoff you’re accepting, and it’s the right one for a dictation box.

The broader point

Most “real-time” voice features aren’t streaming problems. They’re request/response problems wearing a streaming costume.

The user pressed a button. They said a thing. They let go. Somewhere in the middle of that, we collectively decided the answer was a persistent bidirectional connection with a turn-detection model attached — and then spent our optimization budget tuning end-of-turn thresholds instead of the thing that was actually costing us 100 milliseconds, which was a TLS handshake we could have run while the user was still talking.

Look at your voice feature and ask when the audio finishes. If the answer is “before I send it,” you were never streaming anything.

Frequently asked questions

Do you need a WebSocket to transcribe short audio clips?

No. A WebSocket is only necessary when audio is still arriving and you need partial transcripts as words are spoken. For a clip that has already finished recording — a push-to-talk utterance, a voicemail, an IVR response — a single HTTP request returns a complete transcript with no session management, no polling, and less code. AssemblyAI’s Sync Speech-to-Text API returns a finished transcript in about 134 milliseconds at p50.

How do I call a speech-to-text API from a mobile app?

Record the audio locally, then POST the bytes to a synchronous transcription endpoint and read the transcript from the response. With the Sync API that’s POST https://sync.assemblyai.com/transcribe as a multipart form with an audio part, an Authorization header, and an X-AAI-Model header — no SDK required, which matters on mobile where dependency weight counts. Keep your API key on a backend rather than shipping it in the client, and have the app talk to your server.

How do I get a transcript without polling for job status?

Use a synchronous endpoint instead of an async one. Async transcription APIs return a job ID that you poll or attach a webhook to, because they’re built for long files where processing takes seconds to minutes. A synchronous API does the transcription inside the same HTTP exchange and returns the text in the response body, which removes job IDs, polling loops, and webhook infrastructure from your application entirely.

What is the maximum audio length for a synchronous speech-to-text request?

It varies by provider: AssemblyAI’s Sync API accepts 80 milliseconds to 120 seconds per request with a 40 MB cap, Google Cloud Speech-to-Text’s Recognize method caps at 60 seconds or 10 MB, and Azure’s speech-to-text REST API for short audio caps at 60 seconds. OpenAI’s transcription endpoint has no stated duration limit but caps files at 25 MB. For anything longer, every provider expects you to move to an asynchronous or batch endpoint.

Why is my speech-to-text API call slower than the advertised latency?

Usually because connection setup is counted in your latency budget but not in the provider’s benchmark. A cold HTTPS request pays for DNS resolution, a TCP handshake, and a TLS handshake before your audio starts uploading — tens of milliseconds nearby, over 100 milliseconds intercontinentally. Compare the provider’s server-side timing field against your own end-to-end measurement to tell the two apart; the Sync API returns request_time_ms for exactly this. Pre-warming the connection when recording starts moves that cost off the critical path.

How do you improve transcription accuracy for names and technical terms?

Give the model context about vocabulary it can’t otherwise know. The Sync API accepts a keyterms_prompt list for explicit vocabulary such as contact names and product names (up to 2048 characters), a prompt describing what the audio is about (up to 4096 characters), and conversation_context holding prior turns for multi-turn exchanges — all included at no additional cost. Start with none of them and add only the terms the model demonstrably gets wrong, since large keyterm lists padded with common words can cause overcorrection.