JavaScript SDK Talk Commands

Send text for an Anam persona to speak, stream ordered utterances, and control how commands interact with current output.

Basic Usage

During a persona session, you can force a response from the persona using the talk method. This is useful when the user interacts with a UI element or when you use your own LLM instead of Anam’s built-in models.

Both talk() and createTalkMessageStream() require an active streaming session. Call stream() or streamToVideoElement() first.

await anamClient.talk("Hello, how are you?");

This sends the text to the persona, which then speaks it aloud.

To learn more about using the talk method with your own LLM, see the Custom LLMs guide.

Streaming Talk Input

For lower latency, stream messages to the persona in chunks. This works well when streaming output from a custom LLM.

const talkMessageStream = anamClient.createTalkMessageStream();

const chunks = ["He", "l", "lo", ", how are you?"];

for (const chunk of chunks) {
  if (talkMessageStream.isActive()) {
    await talkMessageStream.streamMessageChunk(
      chunk,
      chunk === chunks[chunks.length - 1] // endOfSpeech: true on last chunk
    );
  }
}

Each TalkMessageStream represents one conversation turn. Once the turn ends, create a new stream for the next turn.

Available Methods

The TalkMessageStream object provides these methods:

Method	Description
streamMessageChunk(content, endOfSpeech, utteranceId?)	Send a text chunk. Set endOfSpeech: true on the final chunk. Optionally start a new utterance with utteranceId.
endMessage()	End the stream without sending more content and preserve its latest utterance ID. Alternative to endOfSpeech: true.
isActive()	Returns true if the stream can still accept chunks.
getState()	Returns a TalkMessageStreamState value: UNSTARTED, STREAMING, INTERRUPTED, ENDED, or DELIVERY_FAILED.
getCorrelationId()	Returns the correlation ID for this stream.

Streaming Multiple Utterances

With @anam-ai/js-sdk version 4.26.0 or later, you can divide one TalkMessageStream into ordered utterances. A new utterance waits for the previous one to finish playing while both remain part of the same conversation turn. This lets the persona speak before a tool call and continue when the result is ready.

Multi-utterance talk streams require a Cara 4 avatar. With Cara 3, the persona still speaks the text, but Anam treats the stream as one utterance and does not return your utterance IDs. The SDK does not throw an error or warning.

Pass an utteranceId as the third argument on the first chunk of each utterance. The ID must be a lowercase UUID v4; crypto.randomUUID() returns the required format. The method throws before sending the chunk if the ID is invalid. Omit the ID from later chunks in the same utterance.

const talkMessageStream = anamClient.createTalkMessageStream();
const beforeToolUtteranceId = crypto.randomUUID();

// The first utterance can play while the tool runs.
await talkMessageStream.streamMessageChunk(
  "Let me check ",
  false,
  beforeToolUtteranceId
);
await talkMessageStream.streamMessageChunk("that for you.", false);

const toolResultText = await runToolCall();
const toolResultUtteranceId = crypto.randomUUID();

// A new ID starts the next utterance after the first one.
await talkMessageStream.streamMessageChunk(
  toolResultText,
  true,
  toolResultUtteranceId
);

The IDs are returned in persona MESSAGE_STREAM_EVENT_RECEIVED events as utteranceId. Use them to match streamed captions to the utterances you sent. See Listen for JavaScript SDK Events for the event payload.

Send the next text chunk within 15 seconds. After 15 seconds without a chunk that contains text, the server closes the stream and rejects later chunks with the same correlation ID. Empty chunks do not reset the timeout. For a longer tool call, end the first stream and create a new one for the result.

Ending a Stream

End a conversation turn in one of two ways:

Option 1: Set endOfSpeech on the last chunk

await talkMessageStream.streamMessageChunk("final text", true);

Option 2: Call endMessage() separately

await talkMessageStream.streamMessageChunk("final text", false);
await talkMessageStream.endMessage();

Handling Interruptions

When a user speaks during a stream, the SDK emits AnamEvent.TALK_STREAM_INTERRUPTED and closes the stream:

import { AnamEvent } from "@anam-ai/js-sdk";

anamClient.addListener(AnamEvent.TALK_STREAM_INTERRUPTED, (correlationId) => {
  console.log("Stream interrupted:", correlationId);
  // Handle the interruption - e.g., stop your LLM generation
});

Checking Stream State

Check whether a stream can still accept chunks:

if (talkMessageStream.isActive()) {
  await talkMessageStream.streamMessageChunk(chunk, false);
}

// Or check the specific state
const state = talkMessageStream.getState();
// Returns a TalkMessageStreamState enum value, including DELIVERY_FAILED.

Error Handling

The streamMessageChunk method throws an error if the stream is not active:

try {
  await talkMessageStream.streamMessageChunk("text", false);
} catch (error) {
  // Stream is in INTERRUPTED, ENDED, or DELIVERY_FAILED state
  console.error("Cannot send chunk:", error.message);
}

Handling Delivery Failures

With @anam-ai/js-sdk version 4.29.1 or later, talk streams report chunks that cannot be sent or whose delivery cannot be confirmed. On engines that answer heartbeats, the SDK detects an unresponsive signalling WebSocket and reconnects automatically. If a chunk is not confirmed within about 3 seconds, the stream moves to TalkMessageStreamState.DELIVERY_FAILED and the client emits AnamEvent.MESSAGE_DELIVERY_FAILED. Further calls to streamMessageChunk() reject.

Register the listener once, before starting the session. Use correlationId to match the failure to the answer your application is generating:

import { AnamEvent, type MessageDeliveryFailure } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.MESSAGE_DELIVERY_FAILED,
  (failure: MessageDeliveryFailure) => {
    if (failure.messageType === "talkStream") {
      console.warn("Talk stream delivery unconfirmed:", failure.correlationId);
    }
  }
);

The SDK does not resend chunks. Chunks sent before the signalling connection first opens are queued. After it has opened, sending while it reconnects or stops answering fails the stream instead of queuing chunks for replay. Resolving streamMessageChunk() or endMessage() does not confirm delivery; a failure can arrive after the final chunk, even when the stream was already ENDED.

You can stop generating text for the failed stream, show the answer as text, or use talk() to send the complete answer through REST. If you choose a fallback, keep it limited to the current answer and cancel it when the user interrupts or the session ends.

A delivery failure means receipt was not confirmed. The persona may already have spoken some of the answer, so retrying with talk() can repeat speech. An interruption can also be missed during a disconnection or arrive after recovery has started.

Treat CONNECTION_CLOSED as the end of all open streams. Calling stopStreaming(), or closing the session because the server ended it or the media connection failed, discards pending chunks without a delivery-failure event. If signalling closes permanently on its own, pending chunks fail before CONNECTION_CLOSED.

Correlation IDs

Attach a correlation ID to track streams, especially useful for matching interruption events:

const correlationId = "request-123";
const talkMessageStream = anamClient.createTalkMessageStream(correlationId);

// Later, when handling interruptions:
anamClient.addListener(AnamEvent.TALK_STREAM_INTERRUPTED, (interruptedCorrelationId) => {
  if (interruptedCorrelationId === correlationId) {
    // This specific stream was interrupted
  }
});

Use unique correlation IDs for each TalkMessageStream. The ID appears in TALK_STREAM_INTERRUPTED events, helping you identify which stream was interrupted.

Next Steps

Audio Control

Learn how to control audio input in your Anam sessions