Using Artifact Stores to Save Agent Outputs with aisuite: A Complete Guide

To save large agent outputs with aisuite, pass an InMemoryArtifactStore or FileArtifactStore to Runner.run_sync() via the artifact_store parameter; the framework automatically converts oversized strings into lightweight artifact references while storing the full data for later retrieval.

When building AI agents with aisuite, handling large command outputs or file contents can quickly consume expensive LLM context windows. The aisuite framework, developed by Andrew Ng's team, provides a robust artifact store abstraction that persists heavyweight data outside the model's prompt while maintaining traceability. Using artifact stores to save agent outputs with aisuite allows you to cap message sizes, reduce API costs, and retain full output history for debugging or auditing.

How Artifact Stores Work in aisuite

The artifact system centers on ArtifactRef and Artifact classes defined in aisuite/agents/artifact_store.py. These lightweight metadata objects identify stored blobs without burdening the conversation context.

Core Storage Implementations

aisuite offers two concrete store implementations:

• InMemoryArtifactStore – Stores artifacts in a Python dictionary (lines 68-94 of artifact_store.py). Ideal for testing and ephemeral workflows where persistence across restarts is unnecessary.
• FileArtifactStore – Persists artifacts under a configurable directory hierarchy (lines 111-148 of artifact_store.py). Suitable for production traces that must survive process restarts.

Both implementations expose put(), get(), and delete() methods for artifact lifecycle management.

The Automatic Artifactization Flow

When you enable artifact storage, aisuite orchestrates the following flow during agent execution:

1. Initialization – You inject the store via Runner.run_sync(..., artifact_store=store) (lines 49-58 of runner.py). The runner attaches the store to the active run context (lines 68-71).

2. Hydration – Before the first LLM request, hydrate_messages in aisuite/agents/artifacts.py resolves any existing artifact_ref objects back to real text using the store, enabling seamless run resumption.

3. Size Threshold Checking – During tool execution, tools.py (lines 76-84) calls artifactize_value from artifacts.py (lines 45-78). If a string exceeds the default 20,000-character threshold, the framework stores the full value and replaces it with a reference object containing a preview snippet.

4. Trace Emission – Trace events carry the artifact reference instead of raw payloads. The web UI viewer (aisuite/tracing/viewer.py) can later retrieve the full content via artifact_store.get(ref).text().

Saving Agent Outputs with In-Memory Storage

For development and testing, use InMemoryArtifactStore to capture large outputs without disk I/O:

import aisuite as ai
from aisuite.agents import Agent, Runner

# Initialize an in-memory store

artifact_store = ai.InMemoryArtifactStore()

# Create an agent with shell tools

agent = ai.Agent(
    name="demo",
    model="openai:gpt-4o",
    tools=[*ai.toolkits.shell(cwd=".", allowed_commands=["python3"])],
    instructions="Run the requested command and return the full output.",
)

# Execute a command producing >20k characters

result = Runner.run_sync(
    agent,
    "Run `python3 -c 'print(\"x\" * 21001)'`",
    artifact_store=artifact_store,
)

# The trace contains an artifact reference; retrieve the full data

artifact_ref = ai.ArtifactRef.from_dict(
    result.steps[-1].data["result_artifacts"][0]["artifact_ref"]
)
large_output = artifact_store.get(artifact_ref).text()
print(f"Stored artifact size: {len(large_output)}")

This pattern is verified in tests/toolkits/test_shell.py (lines 165-190), which validates that large shell outputs are automatically artifactized and retrievable via artifact_store.get(ref).

Persisting Agent Outputs to Disk

To survive process restarts, switch to FileArtifactStore:

# Create a persistent store at a specific root directory

artifact_store = ai.FileArtifactStore(".aisuite/artifacts")

# Use the same Runner.run_sync() call as before

result = Runner.run_sync(
    agent,
    "Run `python3 -c 'print(\"x\" * 21001)'`",
    artifact_store=artifact_store,
)

The file store writes each artifact to .aisuite/artifacts/<artifact_id>/data with an accompanying metadata.json. The directory layout enables the web UI to serve /api/artifacts/<artifact_id> endpoints for downloading or previewing historical outputs.

Retrieving Artifacts from the Trace

Whether using in-memory or file-based storage, you can access stored data programmatically via the artifact reference:

# Extract the reference from the trace steps

ref_dict = result.steps[-1].data["result_artifacts"][0]["artifact_ref"]
artifact_ref = ai.ArtifactRef.from_dict(ref_dict)

# Retrieve the full content

artifact = artifact_store.get(artifact_ref)
full_text = artifact.text()

The Artifact object provides a .text() method to decode the stored bytes, while the reference itself contains only a preview string and metadata, keeping LLM context windows lean.

Summary

• Inject the store – Pass artifact_store to Runner.run_sync() or Runner.run() to enable automatic output persistence.
• Choose your backend – Use InMemoryArtifactStore for fast, temporary testing; use FileArtifactStore for durable, production-grade storage.
• Automatic thresholding – Outputs exceeding 20,000 characters are automatically converted to ArtifactRef objects via artifactize_value in aisuite/agents/artifacts.py.
• Programmatic access – Retrieve full content anytime using artifact_store.get(ref).text(), while traces store only lightweight references.
• UI integration – The web viewer automatically resolves artifact references served from FileArtifactStore directories.

Frequently Asked Questions

What is the default size threshold for automatic artifactization in aisuite?

The default threshold is 20,000 characters. When a tool returns a string exceeding this length, aisuite's tools.py automatically invokes artifactize_value to store the data and replace it with a reference. This threshold prevents large command outputs from bloating the LLM context window while preserving the data in the artifact store.

Can I use artifact stores with async runners?

Yes. The artifact_store parameter works identically in Runner.run() (async) and Runner.run_sync() (synchronous). The store is attached to the run context in runner.py (lines 68-71) regardless of which execution mode you choose, ensuring consistent hydration and dehydration of messages.

How does the web UI access stored artifacts?

When using FileArtifactStore, artifacts are written to .aisuite/artifacts/<artifact_id>/data with metadata.json sidecars. The viewer implementation in aisuite/tracing/viewer.py exposes these via HTTP endpoints at /api/artifacts/<artifact_id>, allowing the frontend to fetch full content on demand while displaying previews in the trace timeline.

Is it safe to delete artifacts manually from the file system?

While FileArtifactStore provides a delete() method that safely handles missing files, manual deletion is generally safe if you remove the entire <artifact_id> directory. However, this will break trace references pointing to that artifact. Always use the store's delete(ref) method to ensure consistency between the file system and any in-memory indices.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}