How the Screenshot-to-Code Agent Manages File State and Tracks Code Changes

The agent uses a single mutable AgentFileState object shared across all tool runtimes to track the current file path and content, with changes applied atomically by create_file and edit_file operations.

The abi/screenshot-to-code repository implements a centralized state management pattern that allows the LLM agent to maintain a consistent view of the file being edited throughout a generation session. Instead of scattering file data across multiple variables or external storage, the system relies on one shared dataclass instance that every tool runtime mutates in place.

Core State Architecture

The AgentFileState Dataclass

In backend/agent/state.py, the system defines a lightweight dataclass that serves as the single source of truth for the current file:

@dataclass
class AgentFileState:
    path: str = "index.html"
    content: str = ""

This object stores only two fields: the file path and the file content. The default path of "index.html" ensures that even when the agent starts fresh, it has a sensible fallback target for HTML generation.

Seeding State from Conversation History

When resuming a conversation or continuing from a previous assistant message, the helper function seed_file_state_from_messages pre-populates the AgentFileState instance. This function scans the message history for HTML content the model may have already emitted and injects it into the state, preventing the agent from losing progress between turns.

How Tools Mutate Shared State

All file-related operations receive the same AgentFileState instance via the AgentToolRuntime constructor in backend/agent/tools/runtime.py. This ensures that every tool call—whether creating, editing, or deleting—operates on the identical memory reference.

Creating Files

The _create_file method handles new file generation by extracting HTML content and updating both fields of the shared state:

def _create_file(self, args):
    path = ensure_str(args.get("path") or self.file_state.path or "index.html")
    content = ensure_str(args.get("content"))
    extracted = extract_html_content(content)
    self.file_state.path = path
    self.file_state.content = extracted or content

Each invocation overwrites file_state.path and file_state.content atomically, ensuring the runtime always reflects the latest file snapshot.

Editing Files

The _edit_file method implements safe string replacement logic to track incremental changes. It reads the current file_state.content, applies one or more edits via _apply_single_edit, and writes the result back:

def _edit_file(self, args):
    if not self.file_state.content:
        # handle missing file case

        pass
    for edit in edits:
        old_text = ensure_str(edit.get("old_text"))
        new_text = ensure_str(edit.get("new_text"))
        content, replaced = self._apply_single_edit(content, old_text, new_text, count)
    self.file_state.content = content

This approach guarantees that every modification is recorded in the shared state before the next tool executes, preventing race conditions or lost updates.

Engine Orchestration

The AgentEngine class in backend/agent/engine.py instantiates the state container and wires it into the tool runtime:

self.file_state = AgentFileState()
if initial_file_state and initial_file_state.get("content"):
    self.file_state.path = initial_file_state.get("path") or "index.html"
    self.file_state.content = initial_file_state["content"]

self.tool_runtime = AgentToolRuntime(
    file_state=self.file_state,
    # ... other dependencies

)

When the agent completes a generation turn, the engine returns self.file_state.content as the current code snapshot. This design keeps the state lifecycle tightly coupled to the agent session, creating a fresh instance for each request while preserving continuity within that request.

API Synchronization

The HTTP layer in backend/routes/generate_code.py bridges the internal state with the frontend client. The endpoint accepts an optional fileState payload in the request body, converts it into a dictionary, and passes it as initial_file_state to the Agent constructor. After generation completes, the same dictionary structure is returned in the response, allowing the UI to synchronize its client-side view with the server's internal AgentFileState.

Summary

• Single mutable object: The AgentFileState dataclass in backend/agent/state.py stores the canonical path and content for the active file.
• Shared reference: All tool runtimes receive the same instance via AgentToolRuntime, ensuring every operation mutates the same memory location.
• Atomic updates: create_file and edit_file methods overwrite state fields immediately, tracking every change made by the LLM.
• Engine lifecycle: AgentEngine creates the state, seeds it from history or API payloads, and returns the final content when the turn ends.
• Client synchronization: The REST API exposes fileState payloads to keep frontend and backend views consistent.

Frequently Asked Questions

How does the agent prevent losing file content between conversation turns?

The seed_file_state_from_messages function in backend/agent/state.py extracts HTML content from previous assistant messages and pre-populates the AgentFileState before the new turn begins. This ensures the agent resumes work from the exact state it left off.

What happens if the agent tries to edit a file that does not exist yet?

The _edit_file method checks self.file_state.content before processing edits. If the content is empty, the tool can either raise an error or initialize a blank file depending on the runtime configuration, preventing silent failures on missing state.

Can the agent work with multiple files simultaneously?

The current implementation maintains a single AgentFileState instance per agent session, which tracks one active file at a time. While the create_file tool can change the path to a new file, the system does not maintain a collection of file states concurrently; each generation turn focuses on one primary file.

How are edit conflicts handled when multiple string replacements target the same text?

The _apply_single_edit method performs safe string replacements with an optional count parameter to limit replacements. If an edit cannot be applied because the old_text is not found, the system can report the failure back to the LLM, allowing it to retry with corrected parameters rather than silently skipping the change.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}