Interactive Avatar: Concepts

📘
Scope of this page: this describes the LiveKit plugin path. If you're calling the REST endpoints directly, the primitives are the same but you mint the room token yourself — see Start an interactive avatar session.

The mental model

An Interactive Avatar is not a video you generate and download. It's a live participant that Synthesia renders in real time and places into a LiveKit room you already control.

The integration is a plugin you attach to your LiveKit Agent—it does not replace your agent. Your agent produces speech the way it always does; the component routes that audio to Synthesia's hosted avatar worker, which generates the matching video and broadcasts lip-synced video and audio back to every participant in the room.

The key inversion to internalize: you own the conversation; Synthesia owns the rendered avatar. No GPU runs in your agent process—the render happens on the hosted worker. (For the full client/agent/LiveKit/Synthesia responsibility split, see the Overview.)

Core primitives

Your LiveKit Agent—the conversational program you run (Python 3.10+, livekit-agents >= 1.5.17). It orchestrates your stack and produces the audio the avatar speaks. The plugin attaches to an existing agent; Node.js agents are not supported.

The plugin (synthesia)—the Synthesia component. Install it directly:

pip install livekit-plugins-synthesia

Import it as from livekit.plugins import synthesia. It captures your agent's synthesized audio and forwards it to the avatar worker.

AvatarSession—the single object you attach to your agent. It authenticates, dispatches the worker, and wires audio. You configure it with an AvatarConfig(avatar_ids=[...]). An API key is required — either passed as api_key or read from the SYNTHESIA_API_KEY environment variable — and you can optionally override api_url or join_timeout.

AvatarConfig—the configuration you pass to AvatarSession. It carries avatar_ids: one to five gallery ids of avatars available to your workspace, each prefixed av_…. The first id is rendered; the rest are precomputed so swap_avatar() can switch to them mid-session. avatar_ids must be a list, even for one avatar — passing a bare string raises SynthesiaError.

Finding your avatar ID

Synthetic and personal avatars only — stock actor-based avatars (for example Ryan or Ada) can't be used as interactive avatars.

For an eligible avatar your plan gives you access to:

Copy its ID from the Avatars page: open the ••• menu and select Copy ID.
Convert it with POST /api/interactive-avatars/avatars. Poll until status is completed, then use the returned id in avatar_ids.

A repeat request for the same source returns the existing interactive avatar rather than starting a new conversion — safe to call again if you're not sure whether it's been converted already. Trying to convert a stock avatar, or one outside your plan, returns an error rather than an interactive avatar.

💡
Testing this in the Try It panel? Select Request Example from the dropdown above the cURL preview before hitting Try It. The default view only pre-fills regenerate; it omits the required sourceId field, which can look like a missing parameter. Request Example shows the full request body, sourceId included.

The hosted avatar worker—Synthesia's GPU render service. It turns your agent's audio into an expressive, lip-synced avatar and joins your room as a participant, using a default identity you can configure — see the Plugin reference for the default and the constraint that it must be unique per concurrent avatar in a room.

The LiveKit room—the real-time session your user and agent are already connected to. The worker joins this room; it is the shared space where audio and video are exchanged.

Streams (audio in, media out)—your agent's speech reaches the worker over a LiveKit data stream; the rendered avatar is published back into the room as normal video and audio tracks your client subscribes to.

Token—the avatar worker authenticates to your room with a LiveKit room token. On the plugin path, the plugin mints this token inside your process, exactly like any other participant, and your LiveKit secret never leaves your process. On the REST path you mint this token yourself and pass it as livekitToken. Either way it is a LiveKit token — there is no separate Synthesia session token.

Session—one live conversation instance, corresponding to one AvatarSession in one room. Concurrency is measured in concurrent sessions.

Events—the plugin surfaces two lifecycle events on the AvatarSession: session_ended (the room ended cleanly) and error (the avatar track dropped unexpectedly mid-session; carries a SynthesiaConnectionError).

📌
Turn-taking today. Conversational turn-taking—listening, speaking, and interruption—is handled by your LiveKit AgentSession and your model; the plugin surfaces only the two lifecycle events above. Build against these two events.

The session lifecycle

Conceptually, an AvatarSession moves through three phases:

Start—you attach the avatar (await avatar.start(...)). It authenticates to Synthesia, dispatches the worker, the worker joins the room as a participant, and audio routing is wired. This phase completes only once the avatar has joined and published its video track.
Active—your agent's speech is routed to the worker; the avatar renders and speaks in the room as a live participant.
End—the room disconnects cleanly (session_ended) or the avatar drops unexpectedly (error). Shutdown restores normal audio routing.

Two rules govern the wiring, and breaking either one fails silently:

Attach the avatar before session.start(). Attaching afterwards does not work and usually raises no error — the avatar simply never appears.
Never reassign session.output.audio after attaching. The plugin owns that routing; overriding it kills lip-sync while leaving the avatar visible.

See the Overview and the Quickstart guides for full integration detail.

Testing your integration

Run your agent with dev (or connect --room <name>) against a real LiveKit room.

Two different things are called "Console," and only one of them works:

	What it is	Use it?
`python agent.py console`	A terminal run mode that uses a mock room	No. The avatar silently never appears, and no error is raised.
LiveKit Cloud Agent Console	A real hosted room in the LiveKit dashboard (Agents → Console)	Yes. The fastest way to test an avatar agent.

📘
Stuck?
The Claude Code / Cursor skill can scan your codebase and suggest fixes.

Glossary

Agent—your Python LiveKit program (STT + LLM + TTS orchestration, or a realtime model).
Plugin (synthesia)—the Synthesia component, installed via pip install livekit-plugins-synthesia.
AvatarSession—the object that authenticates, dispatches the worker, and wires audio.
AvatarConfig—configuration for AvatarSession; carries avatar_ids (one to five gallery ids).
Avatar worker—Synthesia's hosted GPU render service; joins your room as a participant.
Room—the LiveKit real-time session shared by your user, your agent, and the avatar.
Participant—any member of a room; the rendered avatar joins as one.
Session—one live conversation instance; the unit of concurrency.
Token—the LiveKit room token used to authenticate the avatar worker to your room.
BYO—Bring Your Own STT / LLM / TTS.