# Backfilling Signals in AgentsView: Process, Implementation, and Best Practices

> Learn the AgentsView process for backfilling signals by scanning stale sessions, recomputing values in-memory, and persisting completion markers for idempotent retries.

- Repository: [Kenn Software/agentsview](https://github.com/kenn-io/agentsview)
- Tags: how-to-guide
- Published: 2026-06-20

---

**AgentsView backfills session quality signals by scanning the SQLite database for stale sessions, recomputing signal values in-memory via `RecomputeSignals`, and persisting a completion marker to ensure idempotent retries.**

AgentsView stores quality signals—such as tool-failure counts, edit-churn, and health scores—for every session in its SQLite database. When new signal fields are added through schema migrations or legacy sessions require processing, the system must **backfill** these values by recomputing them in bulk.

## How the Backfill Process Works

The backfill is orchestrated by three core components defined in [`internal/db/signals.go`](https://github.com/kenn-io/agentsview/blob/main/internal/db/signals.go) and [`internal/sync/engine.go`](https://github.com/kenn-io/agentsview/blob/main/internal/sync/engine.go):

- **BackfillSignals**: Scans the database and manages the backfill loop.
- **RecomputeSignals**: Performs the pure in-memory signal calculation.
- **MarkSignalsBackfillDone**: Records completion to prevent redundant processing.

### Detecting the Need for Backfill

The process begins by checking the `stats` table for the sentinel marker `session_quality_signals_v1`. In [`internal/db/signals.go`](https://github.com/kenn-io/agentsview/blob/main/internal/db/signals.go), the code queries:

```go
var done int
db.getWriter().QueryRow(
    `SELECT count(*) FROM stats WHERE key = ? AND value != 0`,
    signalsBackfillMarker,
).Scan(&done)

```

- If `done == 0`, no backfill has ever run.
- If `done > 0`, the system triggers a **partial** backfill by filtering on `quality_signal_version < CurrentQualitySignalVersion`.

### Selecting Candidate Sessions

The function constructs a dynamic SQL query to fetch sessions requiring updates:

```go
query := `SELECT id FROM sessions WHERE message_count > 0`
if done > 0 {
    query += ` AND quality_signal_version < ?`
    args = append(args, CurrentQualitySignalVersion)
}
rows, _ := db.getReader().QueryContext(ctx, query, args...)

```

Only sessions containing at least one message are selected. If a previous backfill marker exists, the process targets only sessions with stale signal versions.

### Per-Session Signal Recomputation

For each candidate session ID, `BackfillSignals` invokes the provided `computeFn`. In production, this delegates to `engine.RecomputeSignals` from [`internal/sync/engine.go`](https://github.com/kenn-io/agentsview/blob/main/internal/sync/engine.go):

```go
func(bCtx context.Context, id string) error {
    return engine.RecomputeSignals(bCtx, id)
}

```

`RecomputeSignals` performs the following steps:

1. Loads full session metadata via `GetSessionFull`.
2. Retrieves message history via `GetAllMessages`.
3. Executes `computeSignalsAndSecrets` to derive the `SessionSignalUpdate` struct and secret findings.
4. Persists results via `UpdateSessionSignals` and `ReplaceSessionSecretFindings`.

If any step fails, the error propagates back to `BackfillSignals`, which skips the completion marker to enable retry on the next startup.

### Completion Markers and Idempotency

After processing all candidates successfully, `BackfillSignals` calls `MarkSignalsBackfillDone`, which upserts the marker:

```go
func (db *DB) MarkSignalsBackfillDone() error {
    _, err := db.getWriter().Exec(
        `INSERT INTO stats (key, value) VALUES (?, 1)
         ON CONFLICT(key) DO UPDATE SET value = excluded.value`,
        signalsBackfillMarker,
    )
    return err
}

```

This design ensures **partial runs are safe**: if any session fails, the marker remains unset, and the next execution retry only processes remaining stale sessions.

## Code Implementation and Key Functions

You can trigger a backfill manually using the database and engine instances:

```go
func backfillAll(db *db.DB, eng *sync.Engine) error {
    return db.BackfillSignals(context.Background(),
        func(ctx context.Context, id string) error {
            return eng.RecomputeSignals(ctx, id)
        })
}

```

In the production server ([`cmd/agentsview/main.go`](https://github.com/kenn-io/agentsview/blob/main/cmd/agentsview/main.go)), the backfill runs asynchronously after HTTP server startup:

```go
go func() {
    if err := database.BackfillSignals(
        ctx,
        func(bCtx context.Context, id string) error {
            return engine.RecomputeSignals(bCtx, id)
        },
    ); err != nil && ctx.Err() == nil {
        log.Printf("signals backfill: %v", err)
    }
}()

```

### Interaction with Full Resyncs

When a full resync rewrites every session through the inline-signal path, the code short-circuits unnecessary backfill work by pre-marking the completion marker in [`cmd/agentsview/main.go`](https://github.com/kenn-io/agentsview/blob/main/cmd/agentsview/main.go) (lines 66-73). This prevents walking a large database after a clean resync has already computed current signal values.

## Summary

- **Backfilling signals in AgentsView** recomputes quality metrics for legacy or migrated sessions using SQLite markers for state tracking.
- The `BackfillSignals` function in [`internal/db/signals.go`](https://github.com/kenn-io/agentsview/blob/main/internal/db/signals.go) orchestrates detection, selection, and batch processing via a user-supplied compute function.
- `RecomputeSignals` in [`internal/sync/engine.go`](https://github.com/kenn-io/agentsview/blob/main/internal/sync/engine.go) handles the pure in-memory calculation of signal values and secret findings.
- The system uses the `session_quality_signals_v1` marker in the `stats` table to ensure idempotent, retry-safe execution.
- Partial failures leave the marker unset, guaranteeing automatic retry on the next startup without duplicating work.
- Full resyncs can pre-mark completion to skip unnecessary backfill passes.

## Frequently Asked Questions

### What triggers a signal backfill in AgentsView?

A backfill triggers when the `stats` table lacks the `session_quality_signals_v1` marker, or when existing sessions have a `quality_signal_version` lower than the current system version. This typically occurs after schema migrations add new signal fields or when legacy sessions have never been processed by the inline signal path.

### Is the backfill process safe to interrupt and restart?

Yes. The backfill is idempotent because `MarkSignalsBackfillDone` only writes the completion marker after **all** sessions process successfully. If the process interrupts or fails on any session, the marker remains unset, and the next startup will retry only the remaining stale sessions without duplicating previous work.

### How does AgentsView handle concurrent writes during backfill?

The backfill runs in a separate goroutine while the file-watcher and periodic sync continue writing new sessions. Because new writes use the normal incremental path and the backfill only targets existing stale sessions via version filtering, concurrent operations remain safe and consistent.

### Can I manually trigger a backfill without restarting the server?

Yes. You can manually invoke `BackfillSignals` from any context with access to the database and engine instances. Pass a closure wrapping `engine.RecomputeSignals` as the compute function, and handle errors appropriately to retry or log failures without affecting the server's main operations.