How to Implement Custom Tool Policies in aisuite: A Developer’s Guide

aisuite enables fine-grained control over LLM tool invocations by allowing developers to attach JSON-structured policy rules to clients or sessions, with enforcement occurring at runtime via the BaseProvider class before any tool executes.

The andrewyng/aisuite repository provides a unified abstraction layer for interacting with multiple large language model (LLM) providers. When deploying agents that can call external tools—such as web search APIs or database queries—security requires strict governance over which functions are available to whom and under what conditions. Implementing custom tool policies in aisuite lets you create declarative guardrails that intercept tool calls and validate them against your business rules, user roles, or parameter constraints.

Understanding the Tool Policy Architecture

Tool Definitions in tools.ts

Tool capabilities are declared in aisuite-js/src/types/tools.ts, which exports the ToolSpec interface describing each available function. This specification includes the tool’s name, human-readable description, and a JSON-Schema definition of its parameters. When you implement custom tool policies, the runtime uses these definitions to match incoming LLM requests against your policy rules.

Policy Schema Structure

Policies follow a declarative JSON structure defined implicitly in the codebase (referenced from aisuite-js/src/types/policy.ts). A policy object contains a rules array, where each rule specifies:

• tool: The exact name of the tool the rule governs.
• allow: A boolean indicating whether to permit (true) or block (false) the invocation.
• conditions: Optional predicates—such as user role, timestamp, or argument patterns—that must be satisfied for the rule to apply.

Rules are evaluated in order, and the first matching rule determines the outcome. If no rule matches, the system defaults to deny.

Enforcement Layer in base-provider.ts

The core enforcement logic resides in aisuite-js/src/core/base-provider.ts within the BaseProvider class. When an LLM attempts to invoke a tool, the callTool method receives the request, retrieves the active policy via the client’s internal state, and evaluates the rule set against the current context. Only requests satisfying the policy constraints are forwarded to the actual tool implementation; violations trigger a PolicyError.

Step-by-Step Implementation Guide

Step 1: Define Your Tool Specification

Before enforcing policies, ensure your tool is properly typed. Create a ToolSpec in aisuite-js/src/types/tools.ts or your application code:

// types/tools.ts
import { ToolSpec } from "aisuite";

export const searchTool: ToolSpec = {
  name: "web_search",
  description: "Perform a web search and return the top results.",
  parameters: {
    type: "object",
    properties: {
      query: { 
        type: "string", 
        description: "The search query string" 
      },
      max_results: {
        type: "number",
        description: "Maximum number of results to return"
      }
    },
    required: ["query"],
  },
};

Step 2: Create a Custom Policy Object

Construct a policy that implements your security requirements. This example restricts the web_search tool to users with an admin role while explicitly denying all other requests:

// policies/search-policy.ts
export const adminOnlySearchPolicy = {
  rules: [
    {
      tool: "web_search",
      allow: true,
      conditions: { 
        role: "admin",
        max_results: { $lte: 10 }  // Optional: limit result quantity
      },
    },
    {
      tool: "web_search",
      allow: false,  // Default deny for non-admins
    }
  ],
};

Step 3: Register the Policy with AisuiteClient

Attach the policy during client initialization by passing it to the AisuiteClient constructor in aisuite-js/src/client.ts:

import { AisuiteClient } from "aisuite-js/src/client";
import { adminOnlySearchPolicy } from "./policies/search-policy";

const client = new AisuiteClient({
  provider: "openai",
  model: "gpt-4",
  policy: adminOnlySearchPolicy,  // Global policy applied to all requests
});

// Execute a chat that might invoke tools
const response = await client.chat({
  messages: [{ 
    role: "user", 
    content: "Search for recent developments in quantum computing" 
  }],
});

For temporary overrides, apply a different policy to a specific session using setPolicy (also defined in aisuite-js/src/client.ts):

const session = client.createSession();
session.setPolicy({
  rules: [{ tool: "web_search", allow: true }],  // Permissive for this session only
});

const result = await session.chat({ 
  content: "Find me cat videos" 
});

Step 4: Handle Policy Violations

When a tool call violates the active policy, the runtime throws a PolicyError. Implement error handling to provide graceful degradation:

import { PolicyError } from "aisuite-js/src/errors";

try {
  await client.chat({ 
    messages: [{ role: "user", content: "Search for confidential data" }] 
  });
} catch (error) {
  if (error instanceof PolicyError) {
    console.error(`Tool policy violation: ${error.toolName} blocked`);
    // Return a safe fallback message to the user
  } else {
    throw error;  // Re-throw unexpected errors
  }
}

Complete Working Example

Here is a consolidated implementation that wires together tool definitions, policy creation, and client registration:

// src/index.ts
import { AisuiteClient } from "aisuite-js/src/client";
import { ToolSpec } from "aisuite-js/src/types/tools";

// 1. Define the tool
const codeExecutorTool: ToolSpec = {
  name: "execute_python",
  description: "Execute Python code in a sandboxed environment.",
  parameters: {
    type: "object",
    properties: {
      code: { type: "string" },
      timeout: { type: "number", default: 30 }
    },
    required: ["code"],
  },
};

// 2. Define restrictive policy
const safeExecutionPolicy = {
  rules: [
    {
      tool: "execute_python",
      allow: true,
      conditions: {
        // Only allow if code doesn't contain 'import os'
        code: { $notContains: "import os" }
      }
    },
    { tool: "execute_python", allow: false }
  ]
};

// 3. Initialize client with policy
const client = new AisuiteClient({
  policy: safeExecutionPolicy,
});

// 4. Execute with monitoring
async function runSecureAgent(userCode: string) {
  try {
    return await client.chat({
      messages: [{ 
        role: "user", 
        content: `Please execute this Python: ${userCode}` 
      }],
    });
  } catch (e) {
    if (e.name === "PolicyError") {
      return "Execution blocked by security policy.";
    }
    throw e;
  }
}

Summary

• Tool policies in aisuite are JSON-structured rule sets defined in files like aisuite-js/src/types/tools.ts that specify which functions an LLM may invoke.
• Enforcement occurs in aisuite-js/src/core/base-provider.ts via the BaseProvider class, which evaluates rules before executing any tool call.
• Registration happens through the AisuiteClient constructor in aisuite-js/src/client.ts, with per-session overrides available via session.setPolicy().
• Violations throw PolicyError, allowing your application to handle denied requests gracefully.

Frequently Asked Questions

Can I apply different policies to different chat sessions?

Yes. While you can set a global policy in the AisuiteClient constructor, you can override it for individual sessions. According to the implementation in aisuite-js/src/client.ts, calling session.setPolicy(customPolicy) applies a temporary rule set that exists only for that session’s lifecycle, enabling multi-tenant scenarios where different users require different permission levels.

What happens if no policy rule matches a tool call?

The runtime implements a default-deny strategy. As enforced in aisuite-js/src/core/base-provider.ts, if the callTool method evaluates all rules in the active policy and finds no match for the requested tool name and conditions, the invocation is automatically blocked and a PolicyError is raised.

Can policies inspect the content of tool arguments?

Yes. The conditions field in a policy rule can reference specific parameters defined in the tool’s ToolSpec. When BaseProvider evaluates the policy, it compares the actual argument values provided by the LLM against your condition predicates (such as regex matches, numeric ranges, or enum values), allowing fine-grained control over not just which tools are called, but how they are used.

Where is the default deny logic implemented?

The default deny logic resides in aisuite-js/src/core/base-provider.ts within the BaseProvider class. When processing a tool invocation, if the callTool method iterates through the entire rules array without finding a matching tool name that satisfies the current conditions, it immediately returns a policy violation rather than allowing the call to proceed to the underlying provider implementation.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}