# How the Screenshot-to-Code Agent Manages File State and Tracks Code Changes

> Learn how the screenshot-to-code agent manages file state and tracks code changes using a shared mutable object for atomic updates. Explore the abi/screenshot-to-code repository.

- Repository: [Abi Raja/screenshot-to-code](https://github.com/abi/screenshot-to-code)
- Tags: internals
- Published: 2026-03-02

---

**The agent uses a single mutable `AgentFileState` object shared across all tool runtimes to track the current file path and content, with changes applied atomically by `create_file` and `edit_file` operations.**

The `abi/screenshot-to-code` repository implements a centralized state management pattern that allows the LLM agent to maintain a consistent view of the file being edited throughout a generation session. Instead of scattering file data across multiple variables or external storage, the system relies on one shared dataclass instance that every tool runtime mutates in place.

## Core State Architecture

### The AgentFileState Dataclass

In [`backend/agent/state.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/agent/state.py), the system defines a lightweight dataclass that serves as the single source of truth for the current file:

```python
@dataclass
class AgentFileState:
    path: str = "index.html"
    content: str = ""

```

This object stores only two fields: the **file path** and the **file content**. The default path of `"index.html"` ensures that even when the agent starts fresh, it has a sensible fallback target for HTML generation.

### Seeding State from Conversation History

When resuming a conversation or continuing from a previous assistant message, the helper function `seed_file_state_from_messages` pre-populates the `AgentFileState` instance. This function scans the message history for HTML content the model may have already emitted and injects it into the state, preventing the agent from losing progress between turns.

## How Tools Mutate Shared State

All file-related operations receive the same `AgentFileState` instance via the `AgentToolRuntime` constructor in [`backend/agent/tools/runtime.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/agent/tools/runtime.py). This ensures that every tool call—whether creating, editing, or deleting—operates on the identical memory reference.

### Creating Files

The `_create_file` method handles new file generation by extracting HTML content and updating both fields of the shared state:

```python
def _create_file(self, args):
    path = ensure_str(args.get("path") or self.file_state.path or "index.html")
    content = ensure_str(args.get("content"))
    extracted = extract_html_content(content)
    self.file_state.path = path
    self.file_state.content = extracted or content

```

Each invocation overwrites `file_state.path` and `file_state.content` atomically, ensuring the runtime always reflects the latest file snapshot.

### Editing Files

The `_edit_file` method implements safe string replacement logic to track incremental changes. It reads the current `file_state.content`, applies one or more edits via `_apply_single_edit`, and writes the result back:

```python
def _edit_file(self, args):
    if not self.file_state.content:
        # handle missing file case

        pass
    for edit in edits:
        old_text = ensure_str(edit.get("old_text"))
        new_text = ensure_str(edit.get("new_text"))
        content, replaced = self._apply_single_edit(content, old_text, new_text, count)
    self.file_state.content = content

```

This approach guarantees that every modification is recorded in the shared state before the next tool executes, preventing race conditions or lost updates.

## Engine Orchestration

The `AgentEngine` class in [`backend/agent/engine.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/agent/engine.py) instantiates the state container and wires it into the tool runtime:

```python
self.file_state = AgentFileState()
if initial_file_state and initial_file_state.get("content"):
    self.file_state.path = initial_file_state.get("path") or "index.html"
    self.file_state.content = initial_file_state["content"]

self.tool_runtime = AgentToolRuntime(
    file_state=self.file_state,
    # ... other dependencies

)

```

When the agent completes a generation turn, the engine returns `self.file_state.content` as the current code snapshot. This design keeps the state lifecycle tightly coupled to the agent session, creating a fresh instance for each request while preserving continuity within that request.

## API Synchronization

The HTTP layer in [`backend/routes/generate_code.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/routes/generate_code.py) bridges the internal state with the frontend client. The endpoint accepts an optional `fileState` payload in the request body, converts it into a dictionary, and passes it as `initial_file_state` to the `Agent` constructor. After generation completes, the same dictionary structure is returned in the response, allowing the UI to synchronize its client-side view with the server's internal `AgentFileState`.

## Summary

- **Single mutable object**: The `AgentFileState` dataclass in [`backend/agent/state.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/agent/state.py) stores the canonical path and content for the active file.
- **Shared reference**: All tool runtimes receive the same instance via `AgentToolRuntime`, ensuring every operation mutates the same memory location.
- **Atomic updates**: `create_file` and `edit_file` methods overwrite state fields immediately, tracking every change made by the LLM.
- **Engine lifecycle**: `AgentEngine` creates the state, seeds it from history or API payloads, and returns the final content when the turn ends.
- **Client synchronization**: The REST API exposes `fileState` payloads to keep frontend and backend views consistent.

## Frequently Asked Questions

### How does the agent prevent losing file content between conversation turns?

The `seed_file_state_from_messages` function in [`backend/agent/state.py`](https://github.com/abi/screenshot-to-code/blob/main/backend/agent/state.py) extracts HTML content from previous assistant messages and pre-populates the `AgentFileState` before the new turn begins. This ensures the agent resumes work from the exact state it left off.

### What happens if the agent tries to edit a file that does not exist yet?

The `_edit_file` method checks `self.file_state.content` before processing edits. If the content is empty, the tool can either raise an error or initialize a blank file depending on the runtime configuration, preventing silent failures on missing state.

### Can the agent work with multiple files simultaneously?

The current implementation maintains a single `AgentFileState` instance per agent session, which tracks one active file at a time. While the `create_file` tool can change the path to a new file, the system does not maintain a collection of file states concurrently; each generation turn focuses on one primary file.

### How are edit conflicts handled when multiple string replacements target the same text?

The `_apply_single_edit` method performs safe string replacements with an optional count parameter to limit replacements. If an edit cannot be applied because the `old_text` is not found, the system can report the failure back to the LLM, allowing it to retry with corrected parameters rather than silently skipping the change.