Interactive Avatar: Minimal Quickstart

The smallest useful Synthesia Interactive Avatar app: a Python LiveKit agent you can talk to, rendered as a photoreal, lip-synced avatar in your browser. Drop in four API keys and go.

How it works: the agent listens with Cartesia Ink-2 STT (via LiveKit Inference), thinks with OpenAI Realtime used as a text-only streaming brain, and speaks with Cartesia Sonic TTS. The Synthesia plugin reroutes that speech to a hosted avatar worker, which joins the LiveKit room as a regular participant publishing lip-synced video. The frontend is a plain LiveKit client; it needs zero Synthesia-specific code.

Latency: realtime_preflight.py hides ~0.3–0.5 s per turn by starting the Realtime reply on the eager end-of-turn transcript. Look for Adopting realtime preflight speculation in the logs; disable with REALTIME_PREFLIGHT_ENABLED=false.

Prerequisites

• Python 3.10+
• LiveKit Cloud project (free tier works): LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET from cloud.livekit.io
• Synthesia API key with Interactive Avatar access
• Cartesia API key from play.cartesia.ai
• OpenAI API key from platform.openai.com

Run it

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

cp .env.example .env   # then fill in the four keys

python agent.py dev    # terminal 1: the agent worker
python server.py       # terminal 2: frontend at http://localhost:8080

Open http://localhost:8080, click Start, and allow the microphone. Expected sequence: you join the room → the agent connects → the avatar joins and publishes video (cold starts can take longer) → it greets you. Say hello.

Don't use python agent.py console — it runs a mock room and the avatar will silently never appear. Always test through a real room, like this frontend.

The integration, in three lines

Everything avatar-specific in agent.py is:

from livekit.plugins import synthesia

avatar = synthesia.AvatarSession(synthesia.AvatarConfig(avatar_ids=[AVATAR_ID]))
await avatar.start(session, room=ctx.room)   # before session.start() — order matters

The agent produces speech exactly as it always does; the plugin intercepts session.output.audio and the avatar lip-syncs it. That means you can swap any STT/LLM/TTS providers in AgentSession and the avatar lines never change.

Make it yours

All in agent.py:

• Personality — edit INSTRUCTIONS.
• Voice — set CARTESIA_VOICE_ID to any Cartesia library voice.
• Avatar — set AVATAR_ID to any avatar your Synthesia workspace has access to.
• Models — swap the stt= / llm= / tts= lines for any LiveKit-supported provider. Preflight is a no-op for non-Realtime models.

server.py mints room tokens with sync_streams=True — keep that when you build your own token endpoint, it's what keeps the avatar's audio and video in sync in the browser. And keep SYNTHESIA_API_KEY server-side: it's a workspace-bound secret that must never reach frontend code.

Troubleshooting