Tools and Function Calling

Enable your AI personas to interact with external systems, trigger client actions, and search knowledge bases

Overview

Anam’s tool calling system enables AI personas to perform actions beyond conversation. During a session, the LLM can decide when to invoke tools based on user intent, making your personas capable of:

• Triggering client-side actions (opening modals, redirecting pages, updating UI)
• Searching knowledge bases using semantic search (RAG)
• Calling external APIs via webhooks
• Executing custom business logic

Tools make your AI personas agentic, capable of taking actions to help users accomplish their goals.

Beta Feature: Tool calling is currently in beta. You may encounter some issues as we continue to improve the feature. Please report any feedback or issues to help us make it better.

How Tool Calling Works

When a user interacts with your persona, the conversation flows through a decision-making process:

1. User speaks or types

   The user makes a request: “What’s the status of my order 12345?”

2. LLM analyzes intent

   The persona’s LLM analyzes the request and determines it needs external information to respond accurately.

3. Tool invocation

   The LLM selects the appropriate tool and generates a structured function call:

   {
     "name": "check_order_status",
     "arguments": {
       "orderId": "12345"
     }
   }
   
4. Tool execution

   The system executes the tool based on its type: - Client tools: Event sent to your client application - Knowledge tools: Semantic search performed on your documents - Webhook tools: HTTP request sent to your API endpoint

5. Response integration

   The tool result is returned to the LLM, which incorporates it into a natural language response.

   The user hears a complete, informed answer without knowing the technical orchestration behind the scenes.

Tool Types

Anam supports three types of tools, each designed for different use cases:

Client Tools

Client tools trigger events in your client application, enabling the persona to control your user interface.

Common use cases:

• Opening product pages or checkout flows
• Displaying modals or notifications
• Navigating to specific sections of your app
• Updating UI state based on conversation

{
  type: 'client',
  name: 'open_checkout',
  description: 'Opens the checkout page when user wants to purchase',
  parameters: {
    type: 'object',
    properties: {
      productId: {
        type: 'string',
        description: 'The ID of the product to checkout'
      }
    },
    required: ['productId']
  }
}

Client tools work well for creating voice-driven user experiences where the AI can guide users through your application.

Knowledge Tools (RAG)

Knowledge tools enable semantic search across your uploaded documents using Retrieval-Augmented Generation (RAG).

Common use cases:

• Answering questions from product documentation
• Searching company policies or FAQs
• Retrieving information from manuals or guides
• Providing accurate, source-based responses

{
  type: 'server',
  subtype: 'knowledge',
  name: 'search_documentation',
  description: 'Search product documentation when user asks questions',
  documentFolderIds: ['550e8400-e29b-41d4-a716-446655440000', '6ba7b810-9dad-11d1-80b4-00c04fd430c8']
}

Folder IDs are UUIDs. You can find them in the Anam Lab UI at /knowledge or via the API when creating folders.

Knowledge tools handle search automatically:

• They understand the user’s intent, not just keywords.
• They find the most relevant snippets from your documents.
• They provide this information to the AI to form an accurate answer.

Knowledge tools require you to upload and organize documents in knowledge folders before they can be used. Learn more in the Knowledge Base documentation.

Webhook Tools

Webhook tools call external HTTP endpoints, allowing your persona to integrate with any API. This enables your persona to interact with external systems and perform actions.

Common use cases:

• Checking order or shipment status
• Creating support tickets
• Updating CRM records
• Fetching real-time data (weather, stock prices, etc.)
• Triggering workflows in external systems

{
  type: 'server',
  subtype: 'webhook',
  name: 'check_order_status',
  description: 'Check the status of a customer order',
  url: 'https://api.example.com/orders',
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.API_SECRET}`,
    'X-Organization-ID': 'org-uuid'
  },
  parameters: {
    type: 'object',
    properties: {
      orderId: {
        type: 'string',
        description: 'The order ID to check'
      }
    },
    required: ['orderId']
  },
  awaitResponse: true
}

Set awaitResponse: false for fire-and-forget webhooks like logging or notifications where you don’t need the response data.

Attaching Tools to Personas

Tools can be attached to personas in two ways:

Stateful Personas (Database-Stored)

For stateful personas, tools must be created first and then attached by their ID.

1. Create tools

   Create tools via the UI at /tools or via the API. Each tool gets a persistent ID.

   POST /v1/tools
   Content-Type: application/json
   Authorization: Bearer YOUR_API_KEY
   
   {
     "type": "server",
     "subtype": "knowledge",
     "name": "search_docs",
     "description": "Search product documentation",
     "config": {
       "documentFolderIds": ["550e8400-e29b-41d4-a716-446655440000"]
     }
   }
   

   Response includes the tool ID:

   {
     "id": "tool-uuid-123",
     "name": "search_docs",
     ...
   }
   
2. Attach tools to persona

   In the UI at /build/{personaId}, add tools from your organization’s tool library, OR via API when creating/updating a persona:

   PUT /v1/personas/{personaId}
   Content-Type: application/json
   Authorization: Bearer YOUR_API_KEY
   
   {
     "toolIds": ["tool-uuid-123", "tool-uuid-456"]
   }
   
3. Use persona in session

   When you create a session with this persona, all attached tools are automatically loaded:

   POST /v1/auth/session-token
   Content-Type: application/json
   
   {
     "personaConfig": {
       "personaId": "persona-uuid"
     }
   }
   

   The persona’s tools are available during the session without needing to specify them again.

Ephemeral Personas (Session-Only)

For ephemeral personas defined at session creation time, attach tools by their IDs in the toolIds array:

POST /v1/auth/session-token
Content-Type: application/json
Authorization: Bearer YOUR_API_KEY

{
  "personaConfig": {
    "name": "Support Agent",
    "avatarId": "avatar-uuid",
    "voiceId": "voice-uuid",
    "llmId": "llm-uuid",
    "systemPrompt": "You are a helpful support agent...",
    "toolIds": ["tool-uuid-123", "tool-uuid-456"]
  }
}

Tools must be created first via the API or UI before they can be attached to ephemeral personas. The toolIds array references existing tool IDs.

Handling Tool Events

For client tools, use registerToolCallHandler to define per-tool handlers, this will automatically emit completed or failed events when the handler completes.

import { AnamClient } from '@anam-ai/js-sdk';

const client = new AnamClient(sessionToken);

const cancelCheckoutHandler = client.registerToolCallHandler('open_checkout', {
  onStart: async (payload) => {
    window.location.href = `/checkout/${payload.arguments.productId}`;
  },
  onComplete: async(payload) => {
    console.log(`tool call completed ${payload.toolName}`)
  },
  onFail: async(payload) => {
    console.log(`tool call failed ${payload.toolName}`)
  }
});

// you can also register partial handlers
const cancelRegisterHandler = client.registerToolCallHandler('show_notification', {
  onStart: async (payload) => {
    showNotification(payload.arguments.message);
    return 'Notification shown';
  },
});

You can also listen for tool call lifecycle events for logging or analytics:

import { AnamEvent } from '@anam-ai/js-sdk';

client.addListener(AnamEvent.TOOL_CALL_STARTED, (event) => {
  console.log('Tool started:', event.toolName, event.arguments);
});

client.addListener(AnamEvent.TOOL_CALL_COMPLETED, (event) => {
  console.log('Tool completed:', event.toolName, `${event.executionTime}ms`);
});

Register handlers and event listeners before calling streamToVideoElement() to ensure you don’t miss any events.

Tool Design Best Practices

Write Clear Descriptions

The tool description helps the LLM understand when to use the tool. Be specific and include context.

Good

{
  name: 'check_order_status',
  description: 'Check the status of a customer order when they ask about delivery, tracking, or order updates. Use the order ID from the conversation.'
}

Use Semantic Function Names

Follow snake_case naming conventions and make names descriptive:

• search_product_documentation
• create_support_ticket
• open_checkout_page
• search
• doThing
• tool1

Define Clear Parameters

Use JSON Schema to define parameters with detailed descriptions:

parameters: {
  type: 'object',
  properties: {
    orderId: {
      type: 'string',
      description: 'The order ID mentioned by the user, typically in format ORD-12345'
    },
    includeTracking: {
      type: 'boolean',
      description: 'Whether to include detailed tracking information'
    }
  },
  required: ['orderId']
}

The LLM extracts parameter values from the conversation. If a required parameter isn’t available, the LLM may ask the user for clarification or skip the tool call.

Organize Knowledge by Domain

Create separate knowledge folders for different topics and assign them to specific tools for better relevance:

// Instead of one tool searching everything...
{
  name: 'search_all_docs',
  documentFolderIds: [
    '550e8400-e29b-41d4-a716-446655440000', // product docs
    '6ba7b810-9dad-11d1-80b4-00c04fd430c8', // faqs
    '7c9e6679-7425-40de-944b-e07fc1f90ae7'  // policies
  ]
}

// ...use focused tools.
{
  name: 'search_product_docs',
  description: 'Search product documentation for technical questions',
  documentFolderIds: ['550e8400-e29b-41d4-a716-446655440000']
},
{
  name: 'search_faqs',
  description: 'Search frequently asked questions for common inquiries',
  documentFolderIds: ['6ba7b810-9dad-11d1-80b4-00c04fd430c8']
}

Limitations

Tool naming:

• Length: 1-64 characters
• Pattern: ^[a-zA-Z0-9_.-]+$
• No spaces or special characters

Tool descriptions:

• Length: 1-1024 characters

Knowledge tools:

• Require at least one folder ID
• Folders must contain at least one READY document for useful results
• Document uploads subject to size and storage limits
• Supported formats: PDF, TXT, MD, DOCX, CSV, JSON, LOG

Webhook tools:

• 5-second timeout (Ideally much faster)
• Supported methods: GET, POST, PUT, PATCH, DELETE
• Response size limit: 1MB (ideally lower)

Next Steps

Cookbook: Client-Side Tools

Build a multi-page app where the avatar navigates users with voice commands

Getting Started with Tools

Create your first tool in 15 minutes

Knowledge Base Setup

Upload documents and configure RAG search

Webhook Tools

Integrate external APIs with webhook tools