Lately, everyone has been talking about voice assistants with a visual presence: users much prefer talking to a lifelike avatar that blinks naturally, tilts its head, and moves its lips in sync with speech rather than staring at a faceless audio waveform. Yet the moment you sit down to build one for a Russian-speaking service, you run straight into a wall.
Off-the-shelf all-in-one Western platforms (like HeyGen or Tavus) are tuned for English—try making them speak Russian and you get an unnatural, synthetic accent. Meanwhile, we have Yandex Realtime Speech, which streams crisp, natural Russian speech with minimal latency.
While building the voice receptionist "Aika" (Айка), developer Evgeny Ignatiev (eignatiev) decided to bridge both worlds: keep voice synthesis on Yandex's reliable cloud, and offload face rendering to real-time avatar services like Anam (the Cara-4 model) and Simli.
I examined the resulting architecture, and there is a lot to learn from it—several tricky engineering trade-offs are hiding under the hood.
The Sample-Rate Mismatch: Sliding Audio Resampling
The very first hurdle in this pairing is a sample-rate mismatch. Through a FastAPI backend, the Yandex Realtime server emits raw speech via pcm_chunk events in PCM16 format at 24 kHz. Avatar rendering services, however, strictly expect 16 kHz input.
If you do a naive per-frame resample on each chunk, you get a tiny phase discontinuity at every packet boundary. In the user's headphones, that turns into ugly clicks and crackles, while the avatar's neural net starts twitching its lips erratically.
To keep the audio crystal clear, Ignatiev wrote a sliding linear interpolation resampler that preserves the fractional position offset (pos) and the last sample of the previous chunk (prev), stitching packet boundaries seamlessly:
const AVATAR_RATE = 16000;
export const pcmResampler = {
pos: 0,
prev: 0,
reset() {
this.pos = 0;
this.prev = 0;
},
run(int16: Int16Array, inRate: number): Int16Array {
const n = int16.length;
if (!n) return new Int16Array(0);
const step = inRate / AVATAR_RATE;
const out = new Int16Array(Math.ceil((n - this.pos) / step) + 1);
let p = this.pos;
let k = 0;
while (p < n - 1) {
const i = Math.floor(p);
const f = p - i;
const a = i < 0 ? this.prev : int16[i];
// Linear interpolation between adjacent samples
out[k++] = Math.round(a + (int16[i + 1] - a) * f);
p += step;
}
this.pos = p - n;
this.prev = int16[n - 1];
return out.subarray(0, k);
},
};
Starting the WebRTC Session and Fighting iOS Safari
The second challenge is delivering the video stream to the client. Anam renders portrait video at 768×1152 and streams it over WebRTC.
Here is how the client SDK connects and streams audio:
import { createAnamClient } from "@anam-ai/js-sdk";
async function startAvatarSession(videoElement: HTMLVideoElement) {
// 1. Fetch a one-time session token from our own backend
const response = await fetch("/voice/avatar-session", { method: "POST" });
const { sessionToken } = await response.json();
// 2. Initialize the Anam client and bind it to the HTML5 video element
const anamClient = createAnamClient(sessionToken, {
videoElement,
sessionOptions: { videoQuality: "high" },
});
await anamClient.startStreaming();
// 3. Push resampled audio chunks to drive lip-sync animation
function feedAudioChunk(rawChunk24k: Int16Array) {
const chunk16k = pcmResampler.run(rawChunk24k, 24000);
anamClient.streamAudio(chunk16k);
}
return { anamClient, feedAudioChunk };
}
On mobile Safari for iOS, though, a classic autoplay trap awaits: the browser flatly refuses to run video.play() if the media stream arrives asynchronously after a network token request. To bypass this, right at the moment the user physically taps the call button, the app feeds a blank canvas.captureStream() into the video element. Safari treats playback as user-initiated, so when the WebRTC stream arrives moments later, it takes over without being blocked.
Earlier, we looked closely at the race between voice AI agents as Claude Voice and ChatGPT Voice went real-time, and being able to assemble a comparable interactive stack on top of domestic speech synthesis is a solid step forward.
Avatar services bill by the minute and enforce concurrent session limits. If the WebRTC handshake fails to connect within 15 seconds—or the session pool is full—the system doesn't drop the call; it smoothly hides the video pane and continues the conversation in voice-only mode.
In the end, you get a responsive, lifelike assistant face with spot-on Russian pronunciation and resilience against mobile network quirks.

![Node.JS [ru]](/api/digests/it_development/daily/20261019/assets/sources/we-use-js.jpg?v=bf10c5582422a79e168ea74e644a1cc5b66afc9ff54835527b184a48f1dd7bbf)
