How the Disk Memory and File Store System Persist Project State Between Sessions in GPT-Engineer

GPT-Engineer persists project state between sessions by using DiskMemory to store logs and metadata in a hidden .gpteng/memory directory, while FileStore writes generated source files directly to the project folder, ensuring all data survives process restarts.

The AntonOsika/gpt-engineer repository implements a lightweight, filesystem-based persistence layer that allows AI-assisted coding sessions to be resumed after the process exits. By treating the disk as both a key-value store for metadata and a flat file repository for source code, the disk memory and file store system eliminates the need for external databases while keeping project history intact.

Architecture of the Persistence Layer

DiskMemory: The Metadata Archive

DiskMemory is a dictionary-like abstraction that serialises arbitrary data to ordinary files inside a dedicated memory folder. It is defined in [gpt_engineer/core/default/disk_memory.py](https://github.com/AntonOsika/gpt-engineer/blob/main/gpt_engineer/core/default/disk_memory.py#L36) and instantiated via the helper memory_path(), which returns <project_path>/.gpteng/memory.

The class implements the standard mapping protocol:

• __setitem__ (lines 63‑72) writes a value to a file named after the key.
• __getitem__ (lines 44‑52) reads the file back.
• log() (lines 88‑107) appends timestamped entries to a log file, creating an immutable audit trail.
• archive_logs() rotates old logs into a sub‑directory, preventing unbounded growth.

Because the backing store is a plain directory, the data survives process termination and is automatically available the next time the CLI starts.

FileStore: The Project File Interface

While DiskMemory handles ephemeral metadata, FileStore manages the actual source code that the AI generates. Located in [gpt_engineer/core/default/file_store.py](https://github.com/AntonOsika/gpt-engineer/blob/main/gpt_engineer/core/default/file_store.py#L10), the class treats a directory on disk as a mutable file collection.

Key behaviours:

• Constructor (lines 16‑25) accepts an optional path. If None, it creates a temporary directory via tempfile.mkdtemp; otherwise it uses the supplied path (the project root).
• push() (lines 31‑45) writes every entry of a FilesDict to disk, creating parent directories as needed.
• pull() (lines 47‑58) reads the entire directory tree back into a FilesDict, allowing the AI to inspect existing code.

In the CLI, FileStore is instantiated with the concrete project path, meaning every push() writes directly into the user’s repository. Consequently, the generated code is immediately persisted and can be committed, edited, or resumed in a later session without any additional export step.

Session Lifecycle: From Init to Resume

The persistence mechanism is wired together in the CLI entry point [gpt_engineer/applications/cli/main.py](https://github.com/AntonOsika/gpt-engineer/blob/main/gpt_engineer/applications/cli/main.py#L80-L83):

from gpt_engineer.core.default.disk_memory import DiskMemory
from gpt_engineer.core.default.file_store import FileStore
from gpt_engineer.core.default.paths import memory_path

# Inside the main CLI handler

memory = DiskMemory(memory_path(project_path))   # line ~480

files  = FileStore(project_path)                 # line ~482

1. First run – The .gpteng/memory directory is created automatically when DiskMemory writes its first key. FileStore writes generated files into the project root.
2. Process exit – Both helpers hold no open file handles; the OS flushes the data to disk. The .gpteng folder and the source files remain.
3. Resume – On the next invocation, memory_path() returns the same hidden directory, and DiskMemory reloads the existing files. FileStore sees the previously written source files via pull() or simply continues writing new versions.

Because the state is materialised as ordinary files, users can inspect, version‑control, or manually edit the memory logs and the generated code between runs.

Implementation Deep Dive

DiskMemory: File‑System Key‑Value Store

The DiskMemory class implements the MutableMapping interface, allowing it to behave like a Python dictionary while backing every operation to the filesystem.

Writing a value (__setitem__):

def __setitem__(self, key: str, val: str) -> None:
    # key is sanitized to a safe filename

    file_path = self.path / key
    file_path.write_text(val, encoding="utf-8")

This method is located at lines 63‑72.

Reading a value (__getitem__):

def __getitem__(self, key: str) -> str:
    file_path = self.path / key
    if not file_path.exists():
        raise KeyError(key)
    return file_path.read_text(encoding="utf-8")

Found at lines 44‑52.

Appending logs (log):

def log(self, key: str, value: str) -> None:
    file_path = self.path / key
    timestamp = datetime.datetime.now().isoformat()
    with file_path.open("a", encoding="utf-8") as f:
        f.write(f"[{timestamp}] {value}\n")

Located at lines 88‑107.

FileStore: Bridging AI Output to Disk

FileStore abstracts the transition from the in‑memory FilesDict (a dictionary of filenames to contents) to the actual project directory.

Constructor (lines 16‑25):

def __init__(self, path: Optional[Union[str, Path]] = None):
    if path is None:
        # Ephemeral storage for one‑off runs

        self.working_dir = Path(tempfile.mkdtemp())
    else:
        # Persistent project directory

        self.working_dir = Path(path)

Pushing files (push, lines 31‑45):

def push(self, files_dict: FilesDict) -> None:
    for file_name, content in files_dict.items():
        file_path = self.working_dir / file_name
        file_path.parent.mkdir(parents=True, exist_ok=True)
        file_path.write_text(content, encoding="utf-8")

Pulling files (pull, lines 47‑58):

def pull(self) -> FilesDict:
    files_dict = FilesDict()
    for file_path in self.working_dir.rglob("*"):
        if file_path.is_file():
            relative_path = file_path.relative_to(self.working_dir)
            files_dict[str(relative_path)] = file_path.read_text(encoding="utf-8")
    return files_dict

Summary

• DiskMemory provides a dictionary-like interface backed by the filesystem, storing logs, chat history, and AI metadata inside .gpteng/memory.
• FileStore translates in-memory file dictionaries into real project files on disk, using the project root as its working directory when persistence is required.
• Both helpers are instantiated together in the CLI entry point, ensuring that every session—whether fresh or resumed—has immediate access to the previous state.
• No external database is required; ordinary file I/O guarantees durability, and users can inspect or version-control the .gpteng folder alongside their code.

Frequently Asked Questions

Where exactly does GPT-Engineer store session data?

Session data is stored in two locations: (1) the hidden directory .gpteng/memory inside your project root, managed by DiskMemory, and (2) the project files themselves (e.g., src/main.py), managed by FileStore. The memory path is computed by memory_path() and passed to the DiskMemory constructor.

Can I manually edit the memory files between sessions?

Yes. Because DiskMemory uses plain text files, you can edit, delete, or add files inside .gpteng/memory using any text editor. The next time the CLI starts, __getitem__ will read your modified values just as it would any other file.

What happens if I delete the .gpteng directory?

Deleting .gpteng removes all logs, chat history, and AI metadata stored by DiskMemory. The project source files remain untouched because they live in the project root, not inside .gpteng. On the next run, GPT-Engineer will recreate the .gpteng/memory folder automatically, but previous session context will be lost.

How does FileStore handle temporary vs. permanent storage?

FileStore checks the path argument in its constructor. If path is None, it creates a temporary directory via tempfile.mkdtemp() that disappears when the process ends. If a concrete project_path is supplied (as in the CLI), FileStore uses that directory directly, making every push() operation permanent.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}