Client Tools and Events

Enable your AI persona to trigger actions and control your application’s user interface

Overview

Client tools allow your AI persona to trigger events in your client application. When the LLM invokes a client tool, the Anam SDK emits an event that your application can listen for, enabling the persona to:

• Navigate to specific pages or sections
• Open modals, dialogs, or overlays
• Update UI state based on conversation
• Trigger animations or visual effects
• Control media playback
• Submit forms or initiate actions

This creates a seamless voice-driven or chat-driven user experience where the AI can guide users through your application.

Beta Feature: Tool calling is currently in beta. You may encounter some issues as we continue to improve the feature. Please report any feedback or issues to help us make it better.

How Client Tools Work

1. User makes a request

   User: “Show me the pricing page”

2. LLM decides to call tool

   The LLM recognizes this requires a client-side action and generates a tool call:

   {
     "name": "navigate_to_page",
     "arguments": {
       "page": "pricing"
     }
   }
   
3. SDK triggers handler

   The Anam SDK triggers the registered handler for the tool, or emits a TOOL_CALL_STARTED event.
4. Your app handles the event

   Your registered handler receives the tool call and executes the action:

   client.registerToolCallHandler("navigate_to_page", {
     onStart: async (payload) => {
       window.location.href = `/${payload.arguments.page}`;
       return `Navigated to ${payload.arguments.page}`;
     },
   });
   
5. LLM continues conversation

   If awaitResult is true, the return value from your handler is sent back to the LLM. The persona uses the result to continue naturally: “I’ve opened the pricing page for you!”

   If awaitResult is false, the tool call is fire-and-forget — the LLM continues without waiting for a result.

   The user experiences a seamless interaction between voice/chat and UI.

Creating Client Tools

Basic Client Tool Structure

A client tool requires four components:

type · string · required

Must be "client" for client-side tools

name · string · required

Unique identifier for the tool (snake_case, 1-64 characters)

description · string · required

Describes when the LLM should use this tool (helps with decision-making)

parameters · object

JSON Schema defining the parameters the tool accepts

awaitResult · boolean · default: "false"

When true, the return value from your onStart handler is sent back to the LLM as the tool result, allowing the persona to incorporate it into its response. When false (default), the tool call is fire-and-forget.

toolTimeoutSeconds · number

How long the engine waits for the client to return a tool result before timing out. Only applies when awaitResult is true. Range: 1–600 seconds. Default: 10 seconds.

Example: Page Navigation

{
  "type": "client",
  "name": "navigate_to_page",
  "description": "Navigate to a specific page when user asks to see pricing, features, documentation, or other sections",
  "parameters": {
    "type": "object",
    "properties": {
      "page": {
        "type": "string",
        "description": "The page to navigate to (pricing, features, docs, contact)",
        "enum": ["pricing", "features", "docs", "contact", "dashboard"]
      },
      "section": {
        "type": "string",
        "description": "Optional section anchor to scroll to"
      }
    },
    "required": ["page"]
  },
  "awaitResult": true,
  "toolTimeoutSeconds": 5
}

Example: Open Modal

{
  "type": "client",
  "name": "open_modal",
  "description": "Open a modal dialog when user wants to perform an action like checkout, signup, or view details",
  "parameters": {
    "type": "object",
    "properties": {
      "modalType": {
        "type": "string",
        "description": "The type of modal to open",
        "enum": ["checkout", "signup", "login", "contact", "product_details"]
      },
      "data": {
        "type": "object",
        "description": "Additional data to pass to the modal",
        "properties": {
          "productId": { "type": "string" },
          "userId": { "type": "string" }
        }
      }
    },
    "required": ["modalType"]
  }
}

Example: Update UI State

{
  "type": "client",
  "name": "update_filters",
  "description": "Update product filters when user describes what they're looking for",
  "parameters": {
    "type": "object",
    "properties": {
      "category": {
        "type": "string",
        "description": "Product category"
      },
      "priceRange": {
        "type": "object",
        "properties": {
          "min": { "type": "number" },
          "max": { "type": "number" }
        }
      },
      "inStock": {
        "type": "boolean",
        "description": "Only show in-stock items"
      }
    }
  }
}

Handling Tool Events in Your Application

Using registerToolCallHandler (Recommended)

Register handlers for specific tools by name. This will automatically emit completed or failed events when the handler completes. For client tools with awaitResult set to true, the return value from onStart is sent back to the LLM as the tool result:

import { AnamClient } from "@anam-ai/js-sdk";

const client = new AnamClient({
  sessionToken: "YOUR_SESSION_TOKEN",
});

// Register handlers for each tool
client.registerToolCallHandler("navigate_to_page", {
  onStart: async (payload) => {
    window.location.href = `/${payload.arguments.page}`;
    return `Navigated to ${payload.arguments.page}`;
  },
  onComplete: async (payload) => {
    console.log(`Navigation completed in ${payload.executionTime}ms`);
  },
  onFail: async (payload) => {
    console.error(`Navigation failed: ${payload.errorMessage}`);
  },
});

client.registerToolCallHandler("open_modal", {
  onStart: async (payload) => {
    openModal(payload.arguments.modalType, payload.arguments.data);
    return `Opened ${payload.arguments.modalType} modal`;
  },
});

// Start the session
await client.streamToVideoElement("video-element-id");

Each call to registerToolCallHandler returns a cancel function you can use for cleanup:

const cancelNavHandler = client.registerToolCallHandler("navigate_to_page", {
  onStart: async (payload) => {
    // handle navigation
  },
});

// Later, when the handler is no longer needed:
cancelNavHandler();

Using Event Listeners

You can also listen to tool call lifecycle events directly for logging, analytics, or handling tools generically:

import { AnamClient, AnamEvent } from "@anam-ai/js-sdk";

client.addListener(AnamEvent.TOOL_CALL_STARTED, (event) => {
  console.log(`Tool started: ${event.toolName}`, event.arguments);
});

client.addListener(AnamEvent.TOOL_CALL_COMPLETED, (event) => {
  console.log(`Tool completed: ${event.toolName} in ${event.executionTime}ms`);
});

client.addListener(AnamEvent.TOOL_CALL_FAILED, (event) => {
  console.error(`Tool failed: ${event.toolName} - ${event.errorMessage}`);
});

See the complete SDK Reference for all available events, type definitions, and methods.

Real-World Examples

E-commerce Shopping Assistant

Create a shopping assistant that can guide users through your product catalog:

const tools = [
  {
    type: "client",
    name: "show_product",
    description: "Display a product when user asks about specific items",
    parameters: {
      type: "object",
      properties: {
        productId: { type: "string" },
        productName: { type: "string" },
      },
      required: ["productId"],
    },
  },
  {
    type: "client",
    name: "add_to_cart",
    description: "Add a product to cart when user wants to purchase",
    parameters: {
      type: "object",
      properties: {
        productId: { type: "string" },
        quantity: { type: "number", default: 1 },
      },
      required: ["productId"],
    },
  },
  {
    type: "client",
    name: "apply_filter",
    description: "Filter products when user describes preferences",
    parameters: {
      type: "object",
      properties: {
        category: { type: "string" },
        maxPrice: { type: "number" },
        inStock: { type: "boolean" },
      },
    },
  },
  {
    type: "client",
    name: "open_checkout",
    description: "Open checkout when user is ready to purchase",
    parameters: {
      type: "object",
      properties: {},
    },
  },
];

// Register handlers for each tool
client.registerToolCallHandler("show_product", {
  onStart: async (payload) => {
    router.push(`/products/${payload.arguments.productId}`);
    return `Showing product ${payload.arguments.productName}`;
  },
});

client.registerToolCallHandler("add_to_cart", {
  onStart: async (payload) => {
    const { productId, quantity } = payload.arguments;
    await cart.addItem(productId, quantity);
    return `Added ${quantity}x to cart`;
  },
  onComplete: async () => {
    showNotification("Added to cart", "success");
  },
});

client.registerToolCallHandler("apply_filter", {
  onStart: async (payload) => {
    productList.filter(payload.arguments);
    return "Filters applied";
  },
});

client.registerToolCallHandler("open_checkout", {
  onStart: async () => {
    router.push("/checkout");
    return "Checkout opened";
  },
});

Example conversation:

• User: “Show me wireless headphones under $100”
• AI: Calls apply_filter with category: “headphones”, maxPrice: 100
• User: “I like the Sony ones”
• AI: Calls show_product with productId: “sony-wh-1000xm4”
• User: “Add them to my cart”
• AI: Calls add_to_cart “Added to your cart! Ready to checkout?”

Customer Support Dashboard

Create a support agent that can navigate your dashboard:

const tools = [
  {
    type: "client",
    name: "show_ticket",
    description: "Display a support ticket when user mentions a ticket number",
    parameters: {
      type: "object",
      properties: {
        ticketId: { type: "string" },
      },
      required: ["ticketId"],
    },
  },
  {
    type: "client",
    name: "open_chat",
    description: "Open live chat with a human agent when issue needs escalation",
    parameters: {
      type: "object",
      properties: {
        reason: { type: "string", description: "Why escalating to human" },
      },
    },
  },
  {
    type: "client",
    name: "show_analytics",
    description: "Show analytics dashboard when user asks for metrics or reports",
    parameters: {
      type: "object",
      properties: {
        dateRange: { type: "string", enum: ["today", "week", "month"] },
      },
    },
  },
];

SaaS Application Navigator

const tools = [
  {
    type: "client",
    name: "navigate_to_feature",
    description: "Navigate to a specific feature or section of the application",
    parameters: {
      type: "object",
      properties: {
        feature: {
          type: "string",
          enum: ["dashboard", "analytics", "settings", "billing", "team", "integrations"],
        },
      },
      required: ["feature"],
    },
  },
  {
    type: "client",
    name: "create_new",
    description: "Open creation modal for new items (project, user, campaign, etc.)",
    parameters: {
      type: "object",
      properties: {
        itemType: {
          type: "string",
          enum: ["project", "campaign", "user", "report"],
        },
        prefill: {
          type: "object",
          description: "Data to prefill in the creation form",
        },
      },
      required: ["itemType"],
    },
  },
  {
    type: "client",
    name: "run_export",
    description: "Export data when user requests a download",
    parameters: {
      type: "object",
      properties: {
        exportType: { type: "string", enum: ["csv", "pdf", "json"] },
        dataType: { type: "string" },
      },
    },
  },
];

Best Practices

Provide Clear Descriptions

The description helps the LLM decide when to use the tool:

// ✅ Good - Specific about when to use
{
  name: 'open_checkout',
  description: 'Open the checkout page when user explicitly says they want to purchase, buy, checkout, or complete their order'
}

// ❌ Bad - Too vague
{
  name: 'open_checkout',
  description: 'Opens checkout'
}

Use Enums for Constrained Values

When parameters have a limited set of valid values, use enums:

{
  parameters: {
    type: 'object',
    properties: {
      page: {
        type: 'string',
        enum: ['home', 'pricing', 'features', 'contact'],
        description: 'The page to navigate to'
      }
    }
  }
}

This prevents the LLM from generating invalid values.

Handle Errors Gracefully

Validate arguments in your handler. Errors thrown in onStart are automatically caught and routed to the onFail callback. If awaitResult is true, the error message is also sent back to the LLM so it can respond appropriately (e.g., “Sorry, I couldn’t find that product”):

client.registerToolCallHandler("show_product", {
  onStart: async (payload) => {
    const { productId } = payload.arguments;
    if (!productId) {
      throw new Error("Missing productId");
    }

    if (!productExists(productId)) {
      throw new Error("Product not found");
    }

    router.push(`/products/${productId}`);
    return `Showing product ${productId}`;
  },
  onFail: async (payload) => {
    console.error("Tool error:", payload.errorMessage);
    showNotification("Something went wrong");
  },
});

Provide User Feedback

Give immediate feedback when tools execute:

client.registerToolCallHandler("add_to_cart", {
  onStart: async (payload) => {
    const { productId, quantity } = payload.arguments;
    await cart.addItem(productId, quantity);

    // Visual feedback
    showNotification(`Added ${quantity}x to cart`, "success");

    // Update cart icon with animation
    cartIcon.classList.add("bounce");
    setTimeout(() => cartIcon.classList.remove("bounce"), 300);

    return `Added ${quantity}x to cart`;
  },
});

Test Tool Execution

Use the tool call events to debug tool execution in the browser console:

client.addListener(AnamEvent.TOOL_CALL_STARTED, (event) => {
  console.group(`Tool Started: ${event.toolName}`);
  console.log("Type:", event.toolType);
  console.log("Arguments:", event.arguments);
  console.log("Timestamp:", event.timestamp);
  console.groupEnd();
});

client.addListener(AnamEvent.TOOL_CALL_COMPLETED, (event) => {
  console.log(`Tool ${event.toolName} completed in ${event.executionTime}ms`);
});

client.addListener(AnamEvent.TOOL_CALL_FAILED, (event) => {
  console.error(`Tool ${event.toolName} failed: ${event.errorMessage}`);
});

Security Considerations

Validate All Parameters

Never trust client tool arguments without validation:

client.registerToolCallHandler("navigate_to_page", {
  onStart: async (payload) => {
    const validPages = ["home", "pricing", "features", "contact"];
    const { page } = payload.arguments;

    if (!validPages.includes(page)) {
      throw new Error(`Invalid page: ${page}`);
    }

    window.location.href = `/${page}`;
    return `Navigated to ${page}`;
  },
});

Avoid Exposing Sensitive Data

Don’t include sensitive information in tool parameters:

// ❌ Bad - Exposes sensitive data
{
  name: 'show_user_profile',
  parameters: {
    userId: { type: 'string' },
    email: { type: 'string' },
    creditCard: { type: 'string' }  // Never expose this!
  }
}

// ✅ Good - Only IDs, fetch sensitive data server-side
{
  name: 'show_user_profile',
  parameters: {
    userId: { type: 'string' }
  }
}

Rate Limit Tool Calls

Prevent abuse by tracking and limiting tool execution:

const toolCallCounts = {};
const MAX_CALLS_PER_MINUTE = 20;

function withRateLimit(handler) {
  return async (payload) => {
    const now = Date.now();
    const name = payload.toolName;
    toolCallCounts[name] = (toolCallCounts[name] || []).filter((t) => now - t < 60000);

    if (toolCallCounts[name].length >= MAX_CALLS_PER_MINUTE) {
      throw new Error("Rate limit exceeded");
    }

    toolCallCounts[name].push(now);
    return handler(payload);
  };
}

client.registerToolCallHandler("navigate_to_page", {
  onStart: withRateLimit(async (payload) => {
    window.location.href = `/${payload.arguments.page}`;
    return `Navigated to ${payload.arguments.page}`;
  }),
});

Next Steps

Cookbook: Client-Side Tools

Build a multi-page app where the avatar navigates users with voice commands

Webhook Tools

Integrate external APIs with webhook tools

Knowledge Tools

Search documents with RAG-powered tools

SDK Reference

Complete SDK documentation for tool events