Code โบ tail-villain
Why Live Voice Needed a Relay
Managing Gemini Live credentials and long-lived sessions through a backend WebSocket relay
Live voice had reached the point where audio worked, but the boundary around it was still too weak for a product. A direct browser connection to Gemini Live looks tempting at first. It removes one layer from the implementation, but it also exposes credentials at the client boundary and leaves the lifetime of a long WebSocket session in the wrong place.
In Tail Villain, a voice interview is not just playback. The candidate answers, and Kovill follows up immediately on the weak part of that answer. Once that becomes real-time audio, credentials, session lifetime, transcript persistence, and turn control all become part of the same path.
That was the point where Live voice needed a backend relay.
The first failures all looked like one problem. I would click Start, and the interview seemed to end almost immediately. It could have been the new interruption work. It could have been the UI accidentally completing the session. It could also have been the upstream Google Live socket closing.
The first concrete bug was not a voice bug at all. A Start and End guard introduced a React hook-order violation because a callback hook was declared below a loading return. The loaded render had one more hook than the loading render. I replaced that callback with a plain async function and kept moving.
Then the Start button itself became suspicious. In a real-time UI, swapping Start into End or Finish in the same physical position is risky. A fast second click, or an ended state reached too early, can look like the system completed the session by itself. I added a short guard after Start and, more importantly, blocked empty Live sessions from calling complete or generating a report. A session with no transcript turns should be restartable, not completed.
But the remaining failure still happened without a complete call. Once the relay logs were narrow enough, the order was visible: upstream opened, upstream closed, then the client socket closed. The frontend was not ending the interview. The upstream Live connection was not staying open.
The region was the difference. The Seoul region could pass a basic handshake, but it did not sustain the real browser relay flow. Moving the Live relay back to us-central1 produced a direct probe that stayed open long enough to receive setupComplete. On the screen, both failures looked like Live ended immediately. In the logs, they were different classes of failure.
That debugging pass made the relay architecture feel less optional.
The browser sends microphone PCM frames to the backend. The backend forwards them into a Vertex AI Live session, then sends model audio and transcription packets back to the browser. The user experiences a voice interview, but the product keeps credentials server-side and can still persist transcript rows into the existing interview message model.
This was not a file-upload flow. The browser was not recording MP3 files and handing them off later. Live sessions move in small audio frames, and assistant audio comes back as a stream. The relay had to preserve that streaming behavior while sitting at the one place where the product could still decide what should be stored, what should be shown, and when a turn is actually valid.
Runtime also mattered. Long-lived WebSocket relay sessions are a poor fit for a serverless backend shaped around short requests. Cloud Run became the more natural production target because the relay needed a runtime that could own the socket, not just handle a request and disappear.
The harder temptation was state tracking.
Text interviews already had a structured idea of progress: reasoning, next action, evaluation, and session state. In a text-generation path, JSON and tool calling can make that shape feel controllable. It was natural to try native Live function calling so voice and state could advance together.
It did not behave like a clean text-mode JSON loop.
When the relay sent a tool response, the upstream Live session could close or the expected spoken answer could be suppressed depending on timing. When the relay omitted the tool response because the state update was only telemetry, the model could repeatedly call the same tool and fail to return to spoken interviewer output. A downstream UI guard could hide duplicate turns, but it could not stop upstream generation from becoming unstable.
So I gated the Live state tool behind an opt-in environment flag and made the default path voice-first. The voice interview should speak, listen, and preserve a usable transcript before it tries to keep perfect in-band state parity with text mode.
That felt less elegant than sharing one state model everywhere. But Native Audio Live did not behave like text generation with a schema attached. A function response was not a harmless acknowledgment. It could trigger generation, suppress generation, or destabilize the socket.
The final shape was less about adding voice and more about creating a safe corridor for it.
The browser owns the microphone and playback. The backend owns credentials, relay lifetime, and transcript persistence. The model owns real-time audio generation. When those responsibilities blur, the first demo may come together faster, but every failure looks the same afterward.
The useful lesson from that week was that region failures, socket lifecycle bugs, accidental completion, and Live tool-calling instability can all present as the same user-visible symptom: the voice interview just ends. The relay boundary gave me enough evidence to separate them.
The default rule became simple. Stabilize speech and turn-taking first. State tracking and evaluation parity can move to a separate async pass over the transcript if needed. A broken voice session cannot be repaired after the user has already lost the turn.
Live voice needed a relay because the product needed a boundary, not just a pipe.