How to Implement Custom Tool Policies for Agent Tool Execution Control in aisuite

aisuite provides a flexible ToolPolicy protocol that allows you to control whether agent tool calls are executed, denied, or require additional approval by implementing an evaluate method that receives a ToolPolicyContext and returns a ToolPolicyDecision or boolean.

The aisuite framework offers a robust mechanism to implement custom tool policies for agent tool execution control, enabling per-call security checks, rate limiting, and human-in-the-loop approval workflows. By leveraging the ToolPolicy protocol and related types defined in aisuite/agents/types.py, developers can inject custom logic at the precise moment before a tool executes. This architecture supports both stateful class-based policies and lightweight callable functions, giving you complete control over tool execution safety and compliance.

Understanding the Tool Policy Architecture

The tool policy system in aisuite revolves around three core abstractions that work together to standardize execution control across the framework.

Core Components

ToolPolicyContext is a dataclass defined at aisuite/agents/types.py#L27-L40 that bundles runtime information passed to every policy evaluation. It contains the agent name, tool name, arguments, tool metadata, and run-specific details, providing the policy with complete visibility into the pending execution context.

ToolPolicyDecision (located at aisuite/agents/types.py#L84-L90) standardizes the return format for policy evaluations. This dataclass includes an allowed boolean, an optional reason string for denials, and optional metadata for audit trails.

ToolPolicy is a Protocol defined at aisuite/agents/types.py#L42-L45 that requires a single evaluate(context) method returning bool | ToolPolicyDecision. This design allows both class-based implementations and plain callables to satisfy the interface.

Built-in Policy Classes

aisuite ships with several ready-made policies in aisuite/agents/policies.py for common scenarios:

• AllowAllToolPolicy – Permits every tool call unconditionally
• DenyAllToolPolicy – Blocks every tool call unconditionally
• AllowToolsPolicy – Whitelists specific tool names
• RequireApprovalPolicy – Delegates decisions to an external approval function

Policy Evaluation Flow

When a tool call is pending, the engine invokes Tools._evaluate_tool_policy (implemented at aisuite/utils/tools.py#L399-L438). This method:

1. Constructs a ToolPolicyContext from the active run context
2. Invokes the policy via policy.evaluate(context) or direct callable invocation
3. Normalizes the result: bool values become ToolPolicyDecision(allowed=bool), while existing ToolPolicyDecision objects pass through unchanged
4. If allowed is False, the engine blocks execution and emits a tool-policy event containing the denial reason

Both the Runner class (aisuite/agents/runner.py#L75-L93) and the high-level Client (aisuite/client.py#L497-L527) accept a tool_policy argument that propagates to this evaluation engine.

Implementing a Custom Tool Policy

You can implement custom tool policies for agent tool execution control by creating a class with an evaluate method or by providing a simple callable function.

Class-Based Implementation

For stateful policies requiring configuration, persistent counters, or external service integration, implement the ToolPolicy protocol as a class:

from aisuite.agents.types import ToolPolicyDecision, ToolPolicyContext

class RiskBasedToolPolicy:
    """Deny high-risk tools while allowing low-risk operations."""
    
    def evaluate(self, context: ToolPolicyContext) -> ToolPolicyDecision:
        # Access tool metadata populated during tool registration

        meta = context.tool_metadata
        
        if meta and hasattr(meta, 'risk_level') and meta.risk_level == "high":
            return ToolPolicyDecision(
                allowed=False,
                reason="High-risk tools are prohibited in production environments"
            )
        
        return ToolPolicyDecision(allowed=True, reason="Risk assessment passed")

Attach this policy when running an agent:

from aisuite.agents.runner import Runner
from aisuite.agents.types import Agent

def delete_database():
    return "Database deleted"

agent = Agent(name="dba", model="openai:gpt-4o", tools=[delete_database])

# High-risk tool calls will be blocked

result = Runner.run_sync(
    agent, 
    "Delete the database", 
    tool_policy=RiskBasedToolPolicy()
)

Using Simple Callables

For straightforward allow/deny logic without state management, pass a lambda or function directly:

# Allow only the 'echo' tool

allow_only_echo = lambda ctx: ctx.tool_name == "echo"

result = Runner.run_sync(
    agent,
    "Say hello",
    tool_policy=allow_only_echo
)

When a callable returns False, the engine automatically generates a denial with the default reason "Tool call denied by policy".

Extending Built-in Policies

Combine built-in policies with custom logic by wrapping them in composite policies:

from aisuite.agents.policies import AllowToolsPolicy
from datetime import datetime

class BusinessHoursPolicy:
    """Allow specific tools only during business hours."""
    
    def __init__(self):
        self.whitelist = AllowToolsPolicy(["read_file", "search_docs"])
    
    def evaluate(self, context: ToolPolicyContext) -> ToolPolicyDecision:
        # First check the whitelist

        decision = self.whitelist.evaluate(context)
        if not decision.allowed:
            return decision
            
        # Additional time-based restriction

        if datetime.now().hour >= 18 or datetime.now().hour < 9:
            return ToolPolicyDecision(
                allowed=False,
                reason="Tool execution restricted to business hours (9 AM - 6 PM)"
            )
            
        return ToolPolicyDecision(allowed=True)

Practical Custom Policy Examples

The following implementations demonstrate common real-world scenarios for controlling tool execution.

Human Approval Policy

Implement a human-in-the-loop workflow by delegating decisions to an external approval interface:

# policies/human_approval.py

from aisuite.agents.types import ToolPolicyDecision, ToolPolicyContext
from typing import Callable

class HumanApprovalPolicy:
    """
    Routes each tool call through a human approval function.
    """
    def __init__(self, approve_fn: Callable[[ToolPolicyContext], bool]):
        self.approve_fn = approve_fn
    
    def evaluate(self, context: ToolPolicyContext) -> ToolPolicyDecision:
        approved = self.approve_fn(context)
        return ToolPolicyDecision(
            allowed=approved,
            reason="human approved" if approved else "human denied"
        )

Usage with the aisuite Client:

from aisuite.client import Client
from policies.human_approval import HumanApprovalPolicy

def request_ui_approval(ctx: ToolPolicyContext) -> bool:
    # In production: Open a modal dialog or send to Slack

    print(f"Approve {ctx.tool_name} with args {ctx.arguments}?")
    return input("Enter y/n: ").lower() == 'y'

client = Client()
response = client.run(
    agent="assistant",
    prompt="Delete the temp files",
    tool_policy=HumanApprovalPolicy(request_ui_approval)
)

# Inspect the audit trail

print(response.tool_policy_events)

Rate Limiting Policy

Prevent abuse by limiting calls per minute using internal state:

# policies/rate_limit.py

import time
from collections import defaultdict
from aisuite.agents.types import ToolPolicyDecision, ToolPolicyContext

class RateLimitPolicy:
    """Enforce maximum calls per minute per tool."""
    
    def __init__(self, calls_per_minute: int = 10):
        self.limit = calls_per_minute
        self.history = defaultdict(list)  # tool_name -> timestamps

    
    def evaluate(self, ctx: ToolPolicyContext) -> ToolPolicyDecision:
        now = time.time()
        tool_name = ctx.tool_name
        calls = self.history[tool_name]
        
        # Remove timestamps older than 60 seconds

        recent_calls = [t for t in calls if now - t < 60]
        self.history[tool_name] = recent_calls
        
        if len(recent_calls) >= self.limit:
            return ToolPolicyDecision(
                allowed=False,
                reason=f"Rate limit exceeded: {self.limit} calls/minute for {tool_name}"
            )
        
        recent_calls.append(now)
        return ToolPolicyDecision(allowed=True, reason="Within rate limit")

Metadata-Based Filtering

Use tool metadata to enforce categorization policies inline:

from aisuite.agents.runner import Runner

# Allow only tools tagged as 'safe' in their metadata

safe_only = lambda ctx: (
    ctx.tool_metadata is not None and 
    getattr(ctx.tool_metadata, 'category', None) == 'safe'
)

result = Runner.run_sync(
    agent,
    "Perform operation",
    tool_policy=safe_only
)

Integrating Policies with Runners and Clients

Both execution entry points in aisuite accept custom tool policies through the tool_policy parameter.

Using the Runner (aisuite/agents/runner.py):

from aisuite.agents.runner import Runner

result = Runner.run_sync(
    agent=agent,
    prompt="Analyze data",
    tool_policy=RiskBasedToolPolicy(),
    tool_policy_context={"user_tier": "premium"}  # Optional context

)

Using the Client (aisuite/client.py):

from aisuite.client import Client

client = Client()
response = client.run(
    agent="data_analyst",
    prompt="Generate report",
    tool_policy=RateLimitPolicy(calls_per_minute=5)
)

The optional tool_policy_context dictionary allows you to inject additional runtime data (such as user tiers, authentication levels, or session IDs) into the ToolPolicyContext available to your policy's evaluate method.

Monitoring Policy Decisions in Run Traces

All policy evaluations generate auditable events stored in the run trace. When a policy denies a tool call, the engine records the decision via Tools.last_policy_events (maintained in aisuite/utils/tools.py).

Both the Runner and Client surfaces these events:

• Access result.tool_policy_events on Runner results
• Access response.tool_policy_events on Client responses

Each event contains the tool name, decision boolean, reason string, and timestamp, enabling compliance auditing and debugging of policy behavior.

Summary

• aisuite implements custom tool policies through the ToolPolicy protocol requiring an evaluate(context) method that accepts a ToolPolicyContext and returns a ToolPolicyDecision or boolean.
• Core types are defined in aisuite/agents/types.py, including ToolPolicyContext (runtime data) and ToolPolicyDecision (standardized return format).
• Policy evaluation occurs in aisuite/utils/tools.py via _evaluate_tool_policy, which normalizes boolean returns and blocks execution when allowed=False.
• Integration points include the Runner class (aisuite/agents/runner.py) and Client class (aisuite/client.py), both accepting tool_policy and optional tool_policy_context parameters.
• Implementation styles range from simple lambdas for basic filtering to stateful classes for rate limiting, human approval workflows, and composite logic.
• Audit trails are automatically generated and accessible via tool_policy_events on execution results for compliance monitoring.

Frequently Asked Questions

What is the ToolPolicy protocol in aisuite?

The ToolPolicy protocol, defined in aisuite/agents/types.py#L42-L45, is a structural interface that requires any implementing object to provide an evaluate(context: ToolPolicyContext) -> bool | ToolPolicyDecision method. This protocol allows aisuite to treat both class instances and simple functions uniformly during tool execution control.

How do I deny specific tools using a custom policy?

Create a policy that inspects context.tool_name or context.tool_metadata and returns ToolPolicyDecision(allowed=False, reason="...") for prohibited tools. For simple name-based filtering, you can use the built-in AllowToolsPolicy with an allowlist, or implement a custom class that checks ctx.tool_name against a denylist.

Can I combine multiple policies in aisuite?

Yes. While aisuite does not provide a built-in policy combinator, you can implement a composite policy class that instantiates multiple policies (such as AllowToolsPolicy for whitelisting and a custom policy for time restrictions) and chains their evaluate methods. The composite policy returns the first denial encountered, or an allowed decision only if all sub-policies permit the action.

Where are policy decisions logged in aisuite?

Policy decisions are recorded in the run trace via the tool_policy_events list available on response objects from both the Client and Runner. Each event dictionary contains the tool name, the allowed boolean, the reason string (if provided), and execution metadata. These events originate from Tools._evaluate_tool_policy in aisuite/utils/tools.py and propagate through the execution context to the final response.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}