# How Data Is Managed and Stored in dotnet/skills: SQLite Session Persistence Explained

> Discover how dotnet/skills uses SQLite session persistence to manage and store data. Learn about reproducible skill validation and re-judging with the SessionDatabase class.

- Repository: [.NET Platform/skills](https://github.com/dotnet/skills)
- Tags: internals
- Published: 2026-07-08

---

**The dotnet/skills repository uses an embedded SQLite database managed by the `SessionDatabase` class to persist evaluation sessions, agent results, and metadata, enabling reproducible skill validation and re-judging across runs.**

The **dotnet/skills** repository is a skill-evaluation framework that records every AI agent interaction to disk for later analysis and re-evaluation. Understanding how data storage in dotnet/skills works is essential for contributors who need to debug evaluation pipelines, compare baseline results, or resume crashed validation runs.

## Core Data Storage: The SQLite Session Database

The primary mechanism for data management in dotnet/skills centers on a lightweight **SQLite** file that tracks the complete lifecycle of evaluation sessions.

In [`eng/skill-validator/src/Evaluate/SessionDatabase.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Evaluate/SessionDatabase.cs), the `SessionDatabase` class encapsulates all database operations. This class creates and manages a local `sessions.db` file that stores:

- **Session metadata** – timestamps, configuration hashes, and agent collections
- **Input fingerprints** – SHA-256 hashes of input files and directories via `ComputeFileSha()` and `ComputeDirectorySha()` methods
- **Agent results** – generated outputs, execution times, and performance metrics
- **Baseline references** – links to golden result files used for regression testing

The database schema is initialized automatically when `EvaluateCommand` instantiates a new session. Unlike external database servers, this embedded approach requires no configuration and works identically across Windows, macOS, and Linux environments.

## How Evaluation Data Flows Through the System

Data persistence follows a predictable pipeline during skill validation, ensuring that every intermediate state can be reconstructed or re-evaluated.

### Starting an Evaluation Session

When [`EvaluateCommand.cs`](https://github.com/dotnet/skills/blob/main/EvaluateCommand.cs) initiates a run, it creates a timestamped output directory and initializes the SQLite database:

```csharp
var timestampedResultsDir = Path.Combine(outputRoot, DateTime.UtcNow.ToString("yyyyMMdd_HHmmss"));
Directory.CreateDirectory(timestampedResultsDir);
var dbPath = Path.Combine(timestampedResultsDir, "sessions.db");

using var sessionDb = new SessionDatabase(dbPath);

```

This pattern ensures each evaluation run receives an isolated database file, preventing cross-contamination between sessions while maintaining a complete audit trail.

### Recording Agent Results

During execution, the validator captures each agent's output along with computational fingerprints. The `SessionDatabase` writes these records to normalized tables:

```csharp
var result = await agent.RunAsync(input);
sessionDb.InsertResult(
    sessionId: currentSessionId,
    agentId: agent.Id,
    inputSha: SessionDatabase.ComputeFileSha(inputPath),
    outputJson: JsonSerializer.Serialize(result),
    elapsedMs: stopwatch.ElapsedMilliseconds);

```

This storage strategy preserves not just the generated results, but also the exact input state that produced them, enabling deterministic reproduction.

### Re-judging Stored Sessions

The [`RejudgeCommand.cs`](https://github.com/dotnet/skills/blob/main/RejudgeCommand.cs) component demonstrates the durability of this storage model by loading historical sessions for re-evaluation:

```csharp
using var sessionDb = new SessionDatabase(existingDbPath);
var previousSession = sessionDb.GetSession(sessionId);

foreach (var input in previousSession.Inputs)
{
    var newResult = await updatedAgent.RunAsync(input);
    sessionDb.InsertResult(
        sessionId: sessionId,
        agentId: updatedAgent.Id,
        outputJson: JsonSerializer.Serialize(newResult),
        // ... additional metadata
    );
}

```

By reading the original input hashes from SQLite, the system ensures that re-judgments operate on identical data, even if the underlying source files have changed.

## Supporting Data Components

Beyond the core SQLite engine, the repository employs specialized storage strategies for specific use cases.

### Baseline Storage via JSON

While session data lives in SQLite, **baseline results** (golden reference outputs) are persisted as JSON files managed by [`BaselineStore.cs`](https://github.com/dotnet/skills/blob/main/BaselineStore.cs). The SQLite database maintains foreign key references to these files, creating a hybrid storage model where:

- **SQLite** holds the relational metadata and execution context
- **JSON files** store large, schema-evolving result payloads

This separation prevents database bloat while maintaining queryable relationships between sessions and their expected outputs.

### Test Fixture Databases

Several test projects utilize **Entity Framework Core** for demonstration scenarios, though these are distinct from the evaluation storage:

- [`BookStoreContext.cs`](https://github.com/dotnet/skills/blob/main/BookStoreContext.cs) in the dotnet-upgrade tests demonstrates EF Core migrations with ASP.NET Core web APIs
- [`SchoolContext.cs`](https://github.com/dotnet/skills/blob/main/SchoolContext.cs) in the dotnet-test fixtures models university data for code-testing scenarios

These `DbContext` implementations support the repository's sample applications but do not participate in the skill validation pipeline's data management strategy.

## Practical Code Examples

### Querying Historical Sessions

Developers can inspect stored evaluation data using the `SessionDatabase` API:

```csharp
using var db = new SessionDatabase("path/to/sessions.db");

// Retrieve all sessions with agent counts
var sessions = db.GetAllSessions();
foreach (var session in sessions)
{
    Console.WriteLine($"{session.Id}: {session.StartTime:u} – {session.AgentCount} agents");
}

// Fetch detailed results for analysis
var results = db.GetResultsForSession(targetSessionId);
foreach (var result in results)
{
    Console.WriteLine($"Agent {result.AgentId} completed in {result.ElapsedMs}ms");
}

```

### Computing Input Fingerprints

The utility methods in `SessionDatabase` enable consistent hashing of evaluation inputs:

```csharp
// Hash a single file
string fileHash = SessionDatabase.ComputeFileSha("/path/to/input.cs");

// Hash an entire directory tree
string dirHash = SessionDatabase.ComputeDirectorySha("/path/to/project");

```

These hashes serve as cache keys and integrity checks, ensuring that re-judgments and baseline comparisons reference identical input states.

## Summary

- **dotnet/skills** persists evaluation data using an **embedded SQLite database** managed by the `SessionDatabase` class in [`eng/skill-validator/src/Evaluate/SessionDatabase.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Evaluate/SessionDatabase.cs)
- **Each evaluation run** creates an isolated `sessions.db` file in a timestamped output directory, ensuring complete session isolation
- **Agent results, input hashes, and metadata** are stored relationally, enabling `RejudgeCommand` to reload and re-execute historical evaluations
- **Baseline comparisons** use a hybrid model where SQLite stores references and JSON files store golden result payloads
- **Test fixtures** include EF Core contexts like `BookStoreContext`, but these support sample applications rather than the core validation storage layer

## Frequently Asked Questions

### What database engine does dotnet/skills use for data storage?

The repository uses **SQLite** as its primary storage engine for skill evaluation data. The `SessionDatabase` class in [`eng/skill-validator/src/Evaluate/SessionDatabase.cs`](https://github.com/dotnet/skills/blob/main/eng/skill-validator/src/Evaluate/SessionDatabase.cs) manages all database operations, creating local `.db` files that require no external server configuration.

### Can I resume an evaluation if the process crashes?

Yes. Because `EvaluateCommand` commits session data to SQLite incrementally during the evaluation loop, you can inspect partial results in the `sessions.db` file. While the repository does not provide automatic resume logic, the persistent storage allows manual recovery and re-judging of incomplete sessions via `RejudgeCommand`.

### How does the system handle baseline comparisons?

Baseline storage uses a hybrid approach: [`BaselineStore.cs`](https://github.com/dotnet/skills/blob/main/BaselineStore.cs) persists golden results as JSON files on disk, while the SQLite database maintains relational references linking these files to specific evaluation sessions. This structure keeps the database lightweight while preserving baseline integrity through file hashing.

### Are the EF Core contexts in the test directories part of the evaluation storage?

No. Contexts like [`BookStoreContext.cs`](https://github.com/dotnet/skills/blob/main/BookStoreContext.cs) and [`SchoolContext.cs`](https://github.com/dotnet/skills/blob/main/SchoolContext.cs) exist solely within test fixtures to demonstrate skill upgrades for ASP.NET Core applications. The evaluation pipeline's data management relies exclusively on the `SessionDatabase` SQLite implementation, not Entity Framework Core.