> ## Documentation Index
> Fetch the complete documentation index at: https://anam.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LiveKit Configuration

> Configuration options, advanced examples, and API reference for the Anam LiveKit plugin

## Installation

```bash theme={"system"}
pip install livekit-plugins-anam
```

## Environment variables

| Service | Where to get it |
| - | - |
| **Anam** | [lab.anam.ai](https://lab.anam.ai) |
| **LiveKit** | [LiveKit Cloud](https://livekit.io) or self-hosted |
| **LLM providers** | DeepGram, ElevenLabs, OpenAI, Google AI Studio, etc. |

```bash .env theme={"system"}
ANAM_API_KEY=your_anam_api_key
ANAM_AVATAR_ID=your_avatar_id

LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

OPENAI_API_KEY=your_openai_api_key
# or
GEMINI_API_KEY=your_gemini_api_key
```

> [!NOTE]
>
> **LiveKit endpoint requirements:** `LIVEKIT_URL` must be reachable from the public internet because Anam's engine joins your room as a participant. Loopback hosts (`localhost`, `127.0.0.1`, `::1`), private, link-local, reserved and documentation addresses, cluster-internal names such as `*.svc`, `*.local`, and `*.internal`, and single-label hosts are rejected when the session token is created (HTTP 400 at `details.environment.livekitUrl`).
>
> Hostnames must resolve only to public addresses. DNS resolution is checked when the session starts; an unresolvable hostname or one with any non-public address returns HTTP 400 with `invalid_livekit_url`. This also applies to previously issued tokens.
>
> Public `ws://` and `wss://` endpoints and the SDK's `http://` and `https://` aliases are accepted.

## PersonaConfig

Configure the avatar identity and model:

```python theme={"system"}
persona_config = anam.PersonaConfig(
    name="Maya",             # Display name for the avatar
    avatarId="uuid-here",    # Avatar appearance ID
    avatarModel="cara-4",    # Avatar model version
)
```

**name** `string` *required*

Display name for the avatar. Used in logs and debugging.

**avatarId** `string` *required*

UUID of the avatar to use. Get this from the [Avatar Gallery](https://anam.ai/docs/resources/avatar-gallery.md) or [Anam Lab](https://lab.anam.ai/avatars).

**avatarModel** `string`

Avatar model version to use when rendering the avatar. Use `cara-4` for new production avatars.

## AvatarSession

```python theme={"system"}
avatar = anam.AvatarSession(
    persona_config=anam.PersonaConfig(...),
    api_key="your_api_key",
    api_url="https://api.anam.ai",  # Optional
)
```

**persona\_config** `PersonaConfig` *required*

Configuration for the avatar's identity and appearance.

**api\_key** `string` *required*

Your Anam API key.

**api\_url** `string` (default: [https://api.anam.ai](https://api.anam.ai))

Anam API endpoint. Override for staging or self-hosted deployments.

### start()

Starts the avatar session and connects it to the LiveKit room.

```python theme={"system"}
await avatar.start(session, room=ctx.room)
```

**session** `AgentSession` *required*

The LiveKit agent session to connect the avatar to.

**room** `rtc.Room` *required*

The LiveKit room instance from the job context.

## Director Notes cues

Inline **cues** let the avatar shift emotion or delivery mid-turn (see [Director Notes](https://anam.ai/docs/personas/director-notes.md) for the cue list and preset styles). Your LLM writes cues as square-bracketed tags in the spoken text, for example `[warm] Hello there. [curious] What brings you here?`.

Director Notes require a Cara 4 avatar. They work best with a voice that matches the intended performance and a neutral-expression avatar source image; an overly smiley, sad, or angry source image can limit how far cues can move the performance.

In a LiveKit agent, Anam reads those cues from the agent's **TTS-aligned transcript**, which you publish to the room. Wiring this up takes two things:

1. **`use_tts_aligned_transcript=True`** on the `AgentSession`, so the agent produces a transcript with per-word timings.
2. A **JSON transcription sink** scoped to the agent's own identity, so that timed transcript (cue tags included) is published to the room on the `lk.transcription` stream, where the Anam engine reads it to drive per-utterance expression.

> [!WARNING]
>
> This path leaves the `[cue]` tags in the text sent to your TTS, so it only works with a TTS that does **not** read square-bracketed cues aloud — for example Cartesia `sonic-3.6` or ElevenLabs `eleven_v3_conversational`. Cue support varies by model; do not use a model that speaks the tags aloud.

```python theme={"system"}
from livekit.agents import AgentSession
from livekit.agents.voice.room_io import _ParticipantTranscriptionOutput
from livekit.plugins import anam, cartesia

session = AgentSession(
    # ... stt / llm / vad / turn_detection ...
    tts=cartesia.TTS(model="sonic-3.6"),  # does not vocalise "[cue]" tags
    use_tts_aligned_transcript=True,
)

avatar = anam.AvatarSession(
    persona_config=anam.PersonaConfig(
        name="Cara",
        avatarId=os.getenv("ANAM_AVATAR_ID"),
        avatarModel="cara-4",
    ),
    api_key=os.getenv("ANAM_API_KEY"),
)

# Order matters: avatar.start sets the audio sink, then attach the transcription
# sink, then session.start — both sinks stay in place.
await avatar.start(session, room=ctx.room)

# Publish the timed, cue-tagged transcript so the engine can resolve cues against it.
# `participant` MUST be the agent's own identity (it is the transcript's sender);
# leave it unset and every chunk is silently dropped.
session.output.transcription = _ParticipantTranscriptionOutput(
    room=ctx.room,
    is_delta_stream=True,
    participant=ctx.room.local_participant.identity,
    json_format=True,
)

await session.start(agent=Assistant(), room=ctx.room)
```

> [!NOTE]
>
> `_ParticipantTranscriptionOutput` is currently an internal LiveKit Agents API. Set the transcription sink **after** `avatar.start()` and **before** `session.start()`.

### Prompt your LLM to emit cues

The transcript sink only forwards cues that are already in the agent's replies — nothing generates them for you. Your LLM has to write the `[cue]` tags inline, so add a short instruction to the agent's system prompt:

```text theme={"system"}
Use Anam cue tags sparingly and naturally to mark emotional shifts in your spoken replies.
Available cue tags are: [happy], [warm], [playful], [curious], [supportive], [concerned], [sad], [surprised], [angry], [distressed], and [laughter].
Place tags inline before the words they should affect.
Only use a tag when the delivery should noticeably change. Do not explain the tags.
```

See [Prompting an LLM to use cues](https://anam.ai/docs/personas/director-notes.md#prompting-an-llm-to-use-cues) for more detail.

## Advanced examples

### Gemini with Vision

Use Gemini Live for multimodal conversations with screen share analysis:

```python theme={"system"}
import os
from livekit.agents import Agent, AgentSession, JobContext, WorkerOptions, cli
from livekit.agents.voice import VoiceActivityVideoSampler, room_io
from livekit.plugins import anam, google

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    llm = google.realtime.RealtimeModel(
        model="gemini-2.0-flash-exp",
        api_key=os.getenv("GEMINI_API_KEY"),
        voice="Aoede",
        instructions="You are a helpful assistant that can see the user's screen.",
    )

    avatar = anam.AvatarSession(
        persona_config=anam.PersonaConfig(
            name="Maya",
            avatarId=os.getenv("ANAM_AVATAR_ID"),
            avatarModel="cara-4",
        ),
        api_key=os.getenv("ANAM_API_KEY"),
    )

    session = AgentSession(
        llm=llm,
        video_sampler=VoiceActivityVideoSampler(
            speaking_fps=0.2,
            silent_fps=0.1,
        ),
    )

    await avatar.start(session, room=ctx.room)
    await session.start(
        agent=Agent(instructions="Help the user with what you see on their screen."),
        room=ctx.room,
        room_input_options=room_io.RoomInputOptions(video_enabled=True),
    )

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
```

### Function tools

Extend your agent with custom tools:

```python theme={"system"}
from livekit.agents import function_tool

@function_tool
async def fill_form_field(field_name: str, value: str) -> str:
    """Fill in a form field on the user's screen.

    Args:
        field_name: The name of the field to fill
        value: The value to enter

    Returns:
        Confirmation message
    """
    await send_command_to_frontend("fill_field", {"field": field_name, "value": value})
    return "Field filled successfully"

session = AgentSession(
    llm=llm,
    tools=[fill_form_field],
)
```

## Running your agent

> [tab] Development
>
> ```bash theme={"system"}
> python agent.py dev
> ```
>
> Connects to your LiveKit server and automatically joins rooms when participants connect.

> [tab] Production
>
> ```bash theme={"system"}
> python agent.py
> ```
>
> Deploy using Docker, Kubernetes, or your preferred container platform. See the [LiveKit Agents deployment guide](https://docs.livekit.io/agents/deployment) for details.

## Optimize latency

Time to first visible speech depends on your STT, LLM, and TTS latency—including your TTS provider's time to first byte (TTFB)—and how quickly your agent delivers audio. At the start of each response, Anam waits for **480 ms of audio** before it starts generating frames. This is measured in audio duration, not elapsed time, so how long the wait takes depends on how quickly your agent delivers audio. You can't change this buffer per account or per session.

* **Reduce TTS TTFB.** Choose a low-latency TTS model, reuse provider connections where supported, and start TTS as soon as response text is available.
* **Stream audio immediately.** Do not wait for the full utterance. Configure your TTS provider to produce small chunks frequently so LiveKit can forward them as they arrive.
* **Deliver audio at least as fast as real time.** Forward audio without pacing it or adding your own buffer. Audio that arrives faster than real time fills the 480 ms buffer sooner; audio streamed at speaking speed takes about 480 ms to fill it. Sustained delivery slower than real time can cause choppy playback.
* **Minimize network distance.** Where your architecture permits, run your LiveKit agent, TTS, and orchestration close to Anam's `eu-south` infrastructure for European traffic or `us-east` infrastructure for US traffic. Keep your LiveKit server close to the agent and users when possible. The final media path still runs through LiveKit to the client, so test latency from the locations your users connect from.

## Troubleshooting

> [accordion] Agent won't connect to LiveKit
>
> * Verify `LIVEKIT_URL`, `LIVEKIT_API_KEY`, and `LIVEKIT_API_SECRET` are correct
> * If session-token creation returns HTTP 400 at `details.environment.livekitUrl`, replace local development URLs such as `ws://localhost:7880` with a LiveKit endpoint reachable from the public internet
> * Check that your LiveKit server is accessible
> * Ensure WebSocket connections aren't blocked by a firewall
> * Test connectivity at [meet.livekit.io](https://meet.livekit.io)

> [accordion] Avatar not appearing
>
> * Verify your `ANAM_API_KEY` is valid
> * Check that `ANAM_AVATAR_ID` matches an existing avatar
> * Review agent logs for Anam connection errors
> * Ensure the avatar session starts before the agent session

> [accordion] No voice response
>
> * Check your LLM API key is valid (OpenAI, Gemini, etc.)
> * Verify microphone permissions in the browser
> * Look for API errors in the agent logs
> * Confirm the agent is receiving audio tracks

> [accordion] High latency or choppy audio
>
> * Check your network connection stability
> * Consider using LiveKit Cloud for optimized routing
> * Reduce video sampling frequency if CPU-bound
> * Monitor your LLM API response times

When a session ends unexpectedly, use the [session termination reasons](https://anam.ai/docs/resources/session-termination-reasons.md) shown in Anam Lab to diagnose LiveKit room and identity errors.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.