--- updatedAt: 2026-10-15T21:34:40.000Z agentTools: projectIndex: https://docs.synthesia.io/llms.txt --- # Interactive Avatar: LiveKit Plugin reference Complete reference for the Synthesia plugin for LiveKit Agents, available for Python (livekit-plugins-synthesia) and Node.js (@livekit/agents-plugin-synthesia). Use this page when you're already building and need exact signatures, defaults, and failure behavior. For the mental model behind these objects, see [Concepts](https://docs.synthesia.io/reference/ia-concepts); for runnable code, see the [Quickstarts](https://docs.synthesia.io/reference/interactive-avatar-quickstarts). In Python, all symbols live in `livekit.plugins.synthesia`. In Node.js, import the plugin as `import * as synthesia from '@livekit/agents-plugin-synthesia`'. ## Requirements Python * Python 3.10 or later. Tested on 3.10, 3.11 and 3.12. * `livekit-agents` 1.8.2 or later, the plugin's only dependency, installed automatically. Install the plugin: ```python pip install "livekit-plugins-synthesia~=1.8" ``` ```node npm install @livekit/agents-plugin-synthesia ``` If you're following LiveKit's docs, `livekit-agents[synthesia]~=1.8` installs the same package. ## Environment variables Read when the corresponding argument isn't passed explicitly. | Variable | Used for | | -------------------- | ----------------------------------------------------------- | | `SYNTHESIA_API_KEY` | Your Synthesia workspace API key | | `SYNTHESIA_API_URL` | API base URL. Defaults to `https://developers.synthesia.io` | | `LIVEKIT_URL` | Your LiveKit project URL | | `LIVEKIT_API_KEY` | LiveKit API key | | `LIVEKIT_API_SECRET` | LiveKit API secret | All five are secrets and belong in your agent's environment or a secret manager. `SYNTHESIA_API_KEY` is workspace-bound and must never reach frontend code. `LIVEKIT_URL` accepts `https://` as well as `wss://`. The scheme is normalised before it reaches the avatar worker, so the `https://` value LiveKit Cloud injects into deployed agents works unchanged. ## Minimal usage ```python from livekit.plugins import synthesia avatar = synthesia.AvatarSession( synthesia.AvatarConfig(avatar_ids=[""]), ) await avatar.start(session, room=ctx.room) # before session.start() ``` ```node import { voice } from '@livekit/agents'; import * as synthesia from '@livekit/agents-plugin-synthesia'; const session = new voice.AgentSession(/* ... */); const avatar = new synthesia.AvatarSession( new synthesia.AvatarConfig({ avatarIds: [''], }), ); await avatar.start(session, ctx.room); // before session.start() ``` The plugin replaces `session.output.audio`, so the avatar lip-syncs whatever speech your agent produces. You can swap STT, LLM or TTS providers freely and these two lines never change. ## `synthesia.AvatarSession` ```python AvatarSession( avatar_config, *, api_key=None, api_url=None, join_timeout=30.0, avatar_participant_identity=None, avatar_participant_name=None, ) ``` | Parameter | Type | Default | Description | | ----------------------------- | -------------- | -------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `avatar_config` | `AvatarConfig` | **required** | The avatars to render. See below. | | `api_key` | `str \| None` | `None` | Synthesia workspace API key. Falls back to `SYNTHESIA_API_KEY`. | | `api_url` | `str \| None` | `None` | API base URL. Falls back to `SYNTHESIA_API_URL`, then `https://developers.synthesia.io`. | | `join_timeout` | `float` | `30.0` | Seconds to wait for the avatar to join and publish before raising `SynthesiaError` with `type=ErrorType.TIMEOUT`. Raise it if the worker cold-starts slowly. | | `avatar_participant_identity` | `str \| None` | `"synthesia-avatar-agent"` | The LiveKit identity the avatar joins under. Must be unique per concurrent avatar in a room, because LiveKit evicts an existing participant when a second joins with the same identity. | | `avatar_participant_name` | `str \| None` | `"Synthesia avatar"` | The LiveKit display name the avatar joins under. | Passing an empty or whitespace-only string for either participant field raises `SynthesiaError`. Omit them to take the defaults. ### Properties | Property | Description | | ----------------- | -------------------------------------------- | | `avatar_identity` | The LiveKit identity the avatar joins under. | | `provider` | Always `"synthesia"`. | ## `synthesia.AvatarConfig` ```python AvatarConfig(avatar_ids) ``` | Field | Type | Description | | ------------ | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `avatar_ids` | `Sequence[str]` | One to five ids of avatars available to your workspace, each prefixed `av_…`. The first is the active avatar; the rest are precomputed so `swap_avatar()` can switch to them mid-session. | `avatar_ids` is validated when you construct the config, not when the session starts: * A bare string instead of a list raises `ValueError`. * Fewer than one or more than five ids raises `ValueError`. An id your workspace can't access raises `SynthesiaError` with `type=ErrorType.UNKNOWN_AVATAR` later, at `start()`. Pass every avatar you might swap to up front. Adding one afterwards requires a new session. ## Methods ### `await avatar.start(agent_session, room, *, livekit_url=None, livekit_api_key=None, livekit_api_secret=None)` Mounts the avatar into `room`, launches the worker, and wires `session.output.audio`. Returns once the avatar has joined and published its video track. Call this before `AgentSession.start()`. LiveKit credentials fall back to `LIVEKIT_URL`, `LIVEKIT_API_KEY` and `LIVEKIT_API_SECRET`. The room must already be connected, because the plugin mints the avatar's token on behalf of your agent's identity and needs one to exist. Connect the room, then attach the avatar. Calling `start()` on an already-started session returns immediately and does nothing. Calling it while another `start()` is still running raises `SynthesiaError`. ### `await avatar.swap_avatar(avatar_id, *, timeout=15.0)` Switches the rendered avatar mid-session to another id from `avatar_ids`. Pass `"default"` to return to the first id. Returns the now-active avatar id once the swap has taken effect. ```python await avatar.swap_avatar("") await avatar.swap_avatar("default") ``` ```node await avatar.swapAvatar(''); await avatar.swapAvatar('default'); ``` Raises `SynthesiaError` if the session hasn't started or is shutting down, `type=ErrorType.UNKNOWN_AVATAR` if the target wasn't in `avatar_ids`, and `type=ErrorType.CONNECTION` if the swap request to the worker fails. A swap the worker rejects outright raises `SynthesiaError` with no `type` set.
### `await avatar.aclose()` Cooperative shutdown; restores normal audio routing. Called automatically when the room disconnects, or when the avatar drops unexpectedly. ## Events | Event | Fires when | | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | `session_ended` | The room disconnected. A clean end. | | `error` | The avatar stopped publishing video while the room was still connected, meaning the worker failed. Carries a `SynthesiaConnectionError`. | ```python avatar.on("session_ended", lambda: ...) avatar.on("error", lambda exc: ...) ``` `session_ended` means you or your user ended the session; `error` means the avatar went away on its own. Either one triggers shutdown automatically, so you don't need to call `aclose()` in the handler. ## Exceptions Every failure raises `SynthesiaError`, which subclasses LiveKit's `livekit.agents.APIError`, so you can catch Synthesia failures alongside every other plugin's. There is no exception hierarchy below it. To tell failures apart, read the `type` attribute. | Attribute | Description | | ------------- | ----------------------------------------------------------------------------------------------------------------- | | `type` | An `ErrorType` member identifying the failure, or `None` when the failure happened before a request was made. | | `retryable` | Whether retrying the same call could plausibly succeed. | | `retry_after` | Seconds to wait, when the backend supplied a value. Populated for rate-limit and concurrency-limit failures only. | | `status` | The HTTP status the API answered, or `None` if no answer arrived. | | `request_id` | The API's `requestId`, when the response carried one. Quote it when contacting support. | | `body` | The raw response body, when there was one. | ### ErrorType | Member | Meaning | | --------------------- | ---------------------------------------------------------------------------------------------------------------------- | | `AUTH` | The API key is invalid, expired, or lacks the scope this endpoint requires. | | `FEATURE_NOT_IN_PLAN` | Your workspace's plan doesn't include interactive avatars. | | `UNKNOWN_AVATAR` | An avatar isn't accessible to your workspace. Also raised by `swap_avatar()` for a target that wasn't in `avatar_ids`. | | `QUOTA_EXCEEDED` | Your workspace's session quota is exhausted. | | `RATE_LIMITED` | Throttled. Carries `retry_after` when the backend supplied one. | | `CONCURRENCY_LIMIT` | Every concurrent-session slot for your plan is in use. | | `TIMEOUT` | The avatar didn't join within `join_timeout`. | | `CONNECTION` | No usable response, or the avatar dropped mid-session. | Some failures carry no `type` at all, including a missing API key and a swap the worker rejects. Handle `type is None` rather than assuming every `SynthesiaError` is classified. ```python try: await avatar.start(session, room=ctx.room) except synthesia.SynthesiaError as e: if e.type in ( synthesia.ErrorType.QUOTA_EXCEEDED, synthesia.ErrorType.FEATURE_NOT_IN_PLAN, ): ... # fall back to an audio-only session raise ``` ```node try { await avatar.start(session, ctx.room); } catch (e) { if ( e instanceof synthesia.SynthesiaError && (e.type === synthesia.ErrorType.QUOTA_EXCEEDED || e.type === synthesia.ErrorType.FEATURE_NOT_IN_PLAN) ) { // fall back to an audio-only session } throw e; } ```
### Retry guidance Branch on `retryable`, not on `type`; the plugin sets it per failure for exactly this decision. Honor `retry_after` when it's set. For a `TIMEOUT` failure, raise `join_timeout` before retrying, since cold starts are the usual cause. Starting a session does not retry internally. A transient server error surfaces immediately as `SynthesiaError` with `type=ErrorType.CONNECTION`, so retrying is yours to do. Never wrap auth, plan, quota, unknown-avatar, or validation failures in a retry loop. For the HTTP-level error codes behind these error types, see [Errors](https://docs.synthesia.io/reference/errors). ## Troubleshooting ### Prefer automated help? The [Claude Code / Cursor skill](https://github.com/synthesia-ai/skills) can scan your codebase and suggest fixes directly. | Symptom | Likely cause | Fix | | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SynthesiaError: a Synthesia API key is required` | No `SYNTHESIA_API_KEY` and no `api_key=`. | Set the env var or pass `api_key=`. | | `SynthesiaError: LiveKit url, API key, and API secret are required` | One of the three is missing or blank at `start()`. | Set the env vars or pass them to `start()`. A blank secret is caught here deliberately: it still mints a token Synthesia accepts, and LiveKit would only reject it much later. | | `ValueError: avatar_ids must be a list of ids, not a single string` | `AvatarConfig(avatar_ids="")`. | Wrap it in a list: `avatar_ids=[""]`. | | `ValueError: avatar_ids must contain between 1 and 5 ids` | An empty list, or more than five. | Pass one to five ids. | | `SynthesiaError: livekit_url … is not a ws:// or wss:// URL` | A malformed `LIVEKIT_URL`. | Use your project's `wss://` URL. `https://` is normalized automatically. | | `SynthesiaError: the room's local participant has no identity` | `start()` ran before the room finished connecting. | Connect the room first, then attach the avatar. | | `SynthesiaError: start() is already in progress` | Two concurrent `start()` calls. | Await the first one. | | `type=ErrorType.AUTH` | Key invalid or expired, or missing the scope this endpoint requires. | Check the key and its scopes. | | `type=ErrorType.FEATURE_NOT_IN_PLAN` | Your plan doesn't include interactive avatars. | Retrying won't help. Contact Synthesia. | | `type=ErrorType.UNKNOWN_AVATAR` at `start()` | An avatar id isn't accessible to your workspace. | Use an id your workspace has access to. | | `type=ErrorType.UNKNOWN_AVATAR` on `swap_avatar()` | The target id wasn't passed to `AvatarConfig`. | Include every swappable id, up to five, in `avatar_ids` up front. | | `SynthesiaError: swap_avatar() requires a started avatar session` | Called before `start()`, or during shutdown. | Only swap while the session is live. | | `type=ErrorType.CONCURRENCY_LIMIT` | All concurrent session slots are in use. | End an active session, or wait for one to end. | | `type=ErrorType.TIMEOUT` | The avatar didn't join in time. Usually a cold start or network problem. | Raise `join_timeout` and retry. | | Avatar never appears, no error | Terminal `console` run mode. | Run with `dev` or `connect` against a real room, and attach the avatar first. See [Testing your integration](https://docs.synthesia.io/reference/ia-concepts). | | Avatar joins but doesn't lip-sync | The agent isn't producing audio, or `session.output.audio` was reassigned after `start()`. | Confirm the agent speaks without the avatar attached; never reassign `output.audio` after attaching. | | Avatar video looks low-res or blurry | Subscriber-side adaptive streaming downscaled the track. | Create the room with `adaptiveStream: false`, or render the avatar large and call `setVideoQuality(VideoQuality.HIGH)`. Resolution is set by the hosted worker; there's no plugin-side control. | | Works locally, fails when deployed | Secrets missing from the deploy environment. | Confirm all five environment variables exist in the runtime. | | `type=ErrorType.TIMEOUT` on most sessions, not just cold starts | Typical join time has been running \~45 seconds against a 30-second default `join_timeout`. | Raise `join_timeout` well above the default until this improves. | | Session drops and the agent doesn't recover | The plugin has no built-in reconnect. | Handle the `error` event yourself and decide whether to retry. | | A room refuses a new session, or the avatar won't (re)join a room it was just in | A known issue can leave a closed session active in LiveKit, blocking a new one in the same room. | Check the LiveKit dashboard; close the stale room manually, then retry. | ## Choosing an avatar Only synthetic and personal avatars work. Stock actor-based avatars (for example Ryan or Ada) can't be used as interactive avatars, whatever your plan. For an eligible avatar your plan gives you access to: 1. Copy its ID from the [Avatars page](https://docs.synthesia.io/docs/synthesia-avatars): open the `•••` menu and select **Copy ID**. 2. Convert it with [`POST /api/interactive-avatars/avatars`](https://docs.synthesia.io/reference/createinteractiveavatar). Poll until `status` is `completed`, then pass the returned `id` in `avatar_ids`. A repeat request for the same source returns the existing interactive avatar rather than starting a new conversion. Converting a stock avatar, or one outside your plan, returns an error rather than an interactive avatar. ## Additional resources * [LiveKit's Synthesia integration guide](https://docs.livekit.io/agents/models/avatar/plugins/synthesia/): LiveKit's own setup walkthrough for this plugin. * [`livekit-plugins-synthesia` on PyPI](https://pypi.org/project/livekit-plugins-synthesia/) * [`@livekit/agents-plugin-synthesia` on npm](https://www.npmjs.com/package/@livekit/agents-plugin-synthesia?activeTab=readme) # Sibling pages * [Interactive Avatar: Concepts](https://docs.synthesia.io/reference/ia-concepts.md) * [Interactive Avatar: Integration Overview](https://docs.synthesia.io/reference/ia-overview.md) * [Interactive Avatar: Quickstarts](https://docs.synthesia.io/reference/interactive-avatar-quickstarts.md) * [Create an interactive avatar](https://docs.synthesia.io/reference/createinteractiveavatar.md) * [Start an interactive avatar session](https://docs.synthesia.io/reference/createinteractiveavatarsession.md) * [Interactive Avatars: Operational trust](https://docs.synthesia.io/reference/ia-operational-trust.md)