Screenshot to Code Agent Tools Architecture: How create_file and edit_file Work

The create_file and edit_file tools in the screenshot-to-code repository follow a declarative schema-driven architecture where AgentToolRuntime dispatches LLM requests to type-safe handlers that manipulate shared AgentFileState, validating inputs and returning structured ToolExecutionResult objects with human-readable summaries.

The abi/screenshot-to-code repository implements a clean, extensible tool framework that lets Large Language Models safely manipulate HTML files through canonical definitions and runtime validation. Understanding this screenshot to code agent tools architecture reveals how the system prevents common LLM coding errors while maintaining state across multi-step file operations.

Core Components of the Tool Framework

The architecture separates concerns across five key modules, each handling a distinct layer of the tool execution pipeline.

Canonical Tool Definitions

All tool specifications live in backend/agent/tools/definitions.py. This file exposes JSON-schema descriptions for each available tool, defining the exact name, human-readable description, and required input parameters. These schemas drive validation for both the LLM (via OpenAI, Gemini, or Anthropic tool descriptions) and the runtime argument parsing.

Typed Data Structures

The backend/agent/tools/types.py file defines the data contracts that govern tool communication:

• ToolCall – Represents an incoming request from the LLM, containing the tool name and arguments dictionary
• ToolExecutionResult – Wraps the outcome with ok (boolean status), result (payload), summary (concise description), and optional updated file content
• CanonicalToolDefinition – The schema definition used to register tools with the LLM provider

Runtime Dispatcher

backend/agent/tools/runtime.py contains the AgentToolRuntime class, which serves as the central dispatcher. It receives a ToolCall, validates the JSON payload against the expected schema, and forwards execution to the concrete implementation (_create_file, _edit_file, etc.). The runtime also injects helper state and optional services like image generation or background removal.

File State Container

Mutable state management lives in backend/agent/state.py via the AgentFileState class. This container holds the current file path and HTML content, shared across tool invocations so that edit_file operates on the file produced by an earlier create_file call. The overall agent controller populates this state from prior tool results.

Summarization Helpers

Before returning results to the LLM, the system generates human-readable descriptions of what occurred. The backend/agent/tools/summaries.py module builds concise summaries—such as previews of created content or the number of replacements applied—enabling the LLM to understand context without parsing raw HTML.

Execution Flow: How create_file Works

When the LLM decides to create a new HTML file, the system processes the request through five distinct stages:

1. Tool invocation. The LLM emits a tool call named "create_file" with a JSON arguments object containing path (defaulting to index.html) and content. This maps to the ToolCall dataclass defined in backend/agent/tools/types.py.

2. Runtime routing. The AgentToolRuntime.execute() method in backend/agent/tools/runtime.py validates the payload and dispatches to the private _create_file implementation.

3. Content extraction and validation. The handler extracts the path and content parameters, validates that content is non-empty, and processes the HTML through extract_html_content to isolate the body. It then stores both the path and content in the shared AgentFileState instance.

4. Result construction. The method builds a ToolExecutionResult with ok=True, a success message in the result field, and a summary containing the file path, content length, and a preview generated via summarize_text (lines 84-88 in runtime.py).

5. LLM feedback. The populated ToolExecutionResult returns to the LLM, which can now reference the newly created file in subsequent tool calls.

Execution Flow: How edit_file Works

The editing tool supports both single replacements and batch operations through a more complex validation pipeline:

1. Tool invocation. The LLM emits "edit_file" with either a single edit object (old_text, new_text, count) or an edits array containing multiple such objects.

2. Runtime routing. AgentToolRuntime.execute() routes the call to _edit_file in backend/agent/tools/runtime.py.

3. State validation. The handler first verifies that a file already exists by checking file_state.content. If no file exists, it immediately returns a ToolExecutionResult with ok=False and an error summary (lines 11-17).

4. Payload normalization. The runtime normalizes the input into a list of edit dictionaries, validating that each contains the required old_text field (lines 19-25).

5. Safe string replacement. For each edit, _apply_single_edit (lines 91-108) performs a safe replacement:

   • Locates old_text in the current content
   • Determines how many occurrences to replace (default 1, -1 for all)
   • Executes str.replace with the specified count

6. Error handling. If any old_text is not found, the tool aborts immediately with an error indicating the missing snippet (lines 46-55), preventing partial edits or accidental content loss.

7. State update and summarization. After all edits succeed, the updated HTML stores back into AgentFileState. The ToolExecutionResult includes a summary previewing each edit's old/new text and the count replaced (lines 66-79).

8. Continuation. The LLM receives the updated file content and can issue further edits or proceed to other actions.

Key Architectural Design Principles

The screenshot to code agent tools architecture adheres to several critical design patterns that ensure reliability and extensibility:

• Declarative schema-driven contracts. Both create_file and edit_file expose strict JSON schemas. create_file expects content (required) and optional path, while edit_file supports either a single edit or a list of edits, each with old_text, new_text, and optional count parameters.

• Stateless dispatcher with shared state. AgentToolRuntime holds no historical context; it only maintains the current AgentFileState and service flags. This design prevents memory leaks while ensuring file operations remain atomic and traceable.

• Error-first validation. Every tool validates arguments early and returns a ToolExecutionResult with ok=False and a concise summary describing the problem. This feedback loop allows the LLM to self-correct in subsequent calls.

• Safety guardrails. The edit_file tool refuses to replace text that does not exist, preventing silent failures. Additionally, the system prompt in backend/prompts/system_prompt.py enforces that the LLM must use these tools rather than outputting raw HTML.

• Extensible plugin model. New capabilities (such as image generation) are added by defining a new CanonicalToolDefinition in definitions.py and implementing a corresponding private method (e.g., _generate_images) in AgentToolRuntime. The dispatcher automatically exposes new tools to the LLM without modifying core routing logic.

Practical Code Examples

Building a ToolCall for File Creation

from agent.tools.types import ToolCall

create_call = ToolCall(
    id="1",
    name="create_file",
    arguments={
        "path": "index.html",
        "content": "<!DOCTYPE html><html><body><h1>Hello</h1></body></html>"
    },
)

The schema for this call is defined in backend/agent/tools/definitions.py under name="create_file".

Dispatching Calls Through the Runtime

from agent.state import AgentFileState
from agent.tools.runtime import AgentToolRuntime

state = AgentFileState()               # starts empty

runtime = AgentToolRuntime(
    file_state=state,
    should_generate_images=False,
    openai_api_key=None,
    openai_base_url=None,
)

result = await runtime.execute(create_call)

print(result.ok)          # True

print(result.result)      # {"content": "Successfully created file at index.html.", ...}

print(result.summary)     # {"path": "index.html", "contentLength": 57, "preview": "..."}

All heavy lifting occurs inside runtime._create_file (lines 58-84 in backend/agent/tools/runtime.py).

Executing a Single Text Replacement

edit_call = ToolCall(
    id="2",
    name="edit_file",
    arguments={
        "old_text": "Hello",
        "new_text": "Welcome",
        "count": 1,               # replace only the first occurrence

    },
)

edit_result = await runtime.execute(edit_call)

print(edit_result.ok)          # True

print(edit_result.result["content"])  # Updated HTML with "Welcome"

The edit logic lives in runtime._edit_file (lines 111-180) and runtime._apply_single_edit (lines 91-108).

Batch Editing with the edits Array

batch_edit = ToolCall(
    id="3",
    name="edit_file",
    arguments={
        "edits": [
            {"old_text": "<h1>", "new_text": "<h2>", "count": -1},
            {"old_text": "</h1>", "new_text": "</h2>", "count": -1},
        ]
    },
)

batch_result = await runtime.execute(batch_edit)

The runtime automatically normalizes the payload and runs each edit sequentially.

Generating Tool Summaries for LLM Feedback

from agent.tools.summaries import summarize_tool_input

summary = summarize_tool_input(edit_call, state)

# → {"path": "index.html", "edits": [{"old_text": "Hello", "new_text": "Welcome", "count": 1}]}

Summaries are defined in backend/agent/tools/summaries.py.

Summary

The screenshot to code agent tools architecture provides a robust framework for LLM-driven file manipulation:

• Schema definitions in definitions.py establish strict contracts that both the LLM and runtime must follow
• Type-safe data classes (ToolCall, ToolExecutionResult) ensure reliable communication between components
• AgentToolRuntime serves as a stateless dispatcher that routes requests to specific handlers like _create_file and _edit_file
• AgentFileState maintains mutable file content across tool invocations, enabling iterative editing workflows
• Safety validations prevent common errors by verifying file existence before edits and requiring exact text matches for replacements
• Summarization layers convert technical results into human-readable feedback that the LLM can interpret for subsequent actions

Frequently Asked Questions

What happens if edit_file cannot find the old_text snippet?

If _apply_single_edit cannot locate the specified old_text in the current file content, the tool immediately aborts and returns a ToolExecutionResult with ok=False. The result includes a concise error message indicating which snippet was missing, allowing the LLM to adjust its next edit call. This safety guard prevents partial replacements or accidental content deletion.

How does the runtime handle multiple edits in a single call?

When the LLM provides an edits array rather than a single edit object, AgentToolRuntime normalizes the payload into a list of edit dictionaries. It then iterates through each edit sequentially, applying _apply_single_edit for every item. If any individual edit fails validation or cannot find its target text, the entire operation aborts without applying partial changes.

Where is the file content stored between tool calls?

Mutable file state resides in the AgentFileState class defined in backend/agent/state.py. This object holds both the file path and the current HTML content. The AgentToolRuntime receives a reference to this state during initialization, allowing create_file to establish the initial content and edit_file to mutate it across multiple invocations.

Can new tools be added without modifying the core runtime?

Yes. The architecture supports extensibility through the CanonicalToolDefinition system. To add a new tool, define its schema in definitions.py and implement a corresponding private method in AgentToolRuntime (e.g., _generate_images). The dispatcher automatically exposes the new capability to the LLM, provided the method name matches the tool definition's name field.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}