AgentsView Full Resync vs Incremental Sync: How the Sync Engine Handles Data Updates

AgentsView uses three distinct synchronization modes—per-file incremental sync via SyncPaths, full-discovery incremental sync via SyncAll, and complete database rebuild via ResyncAll—triggered respectively by file system events, periodic timers, and explicit CLI or HTTP API requests.

AgentsView is an open-source session management tool that maintains a synchronized SQLite database of agent conversations. Understanding how the kenn-io/agentsview repository handles full resync vs incremental sync is critical for developers integrating with the API or customizing the sync behavior for large-scale deployments.

Understanding AgentsView's Three Sync Modes

The sync engine in internal/sync/engine.go implements three operational modes that balance performance against thoroughness. Each mode uses different strategies for reading agent session files and writing to the SQLite backend.

Per-File Incremental Sync (SyncPaths)

The per-file incremental sync is the lightest-weight operation, designed to react to specific file system changes. When invoked via func (e *Engine) SyncPaths(paths []string) (lines 76-112), the engine:

• Parses only the file paths provided in the slice
• Updates the SQLite database records for affected sessions exclusively
• Refreshes the in-memory skip-cache to reflect the new state

This method uses syncWriteDefault for persisting results and runs workers in parallel to process the limited path set. It avoids directory walking overhead by trusting that the caller has identified exactly which files changed.

Full Discovery Incremental Sync (SyncAll)

The full discovery incremental sync performs a comprehensive scan while still skipping unchanged files. Implemented in func (e *Engine) SyncAll(ctx context.Context, prog ProgressFunc) (lines 4109-4140), this mode:

• Walks all configured agent directories to discover every session file
• Hashes each file and compares size, modification time, and data-version against the skip-cache
• Skips files that match the cache, processing only new or updated sessions
• Processes batches of 100 files using up to 8 concurrent workers via e.syncAllLocked

This approach ensures database consistency with the file system while minimizing I/O through the skip-cache mechanism. It writes results using syncWriteDefault and updates the skip-cache upon completion.

Complete Database Rebuild (ResyncAll)

The full resync operation rebuilds the entire database from scratch to ensure absolute consistency. The func (e *Engine) ResyncAll(ctx context.Context, onProgress ProgressFunc) implementation (lines 1466-1670) performs an atomic swap:

1. Creates a temporary database file with the suffix -resync (origPath + "-resync")
2. Clears the skip-cache entirely to force reprocessing of all files
3. Reparses every session file using bulk write semantics (syncWriteBulk)
4. Copies insights, orphaned data rows, and user-managed metadata to the new database
5. Rebuilds the Full-Text Search (FTS) index
6. Atomically renames the temporary file over the original database

This atomic rename ensures that readers never see a partially constructed database, making it safe for production use even with large datasets.

What Triggers Full Resync vs Incremental Sync in AgentsView

The kenn-io/agentsview source code defines specific triggers for each sync mode based on event types and explicit user requests.

File System Events Trigger Per-File Sync

The file watcher in internal/sync/watcher.go (lines 125-138) monitors agent directories using fsnotify. When the watcher detects create, modify, or delete events, it collects the affected paths and calls engine.SyncPaths(changedPaths):

// internal/sync/watcher.go
func (w *Watcher) handleEvent(event fsnotify.Event) {
    // collect changed paths …
    w.engine.SyncPaths(changedPaths)
}

This provides near-real-time synchronization with minimal resource consumption when individual files change.

Periodic Timer Triggers Full Discovery

The CLI entry point in cmd/agentsview/main.go (lines 210-218) initializes a background ticker that runs full discovery incremental sync on a configurable interval (defaulting to 15 minutes):

// cmd/agentsview/main.go
for range time.NewTicker(cfg.SyncInterval).C {
    engine.SyncAll(context.Background(), nil)
}

This timer ensures that the database eventually reflects any changes that might have been missed by the file watcher or performed outside the application's monitoring.

Explicit Requests Trigger Full Resync

Administrators can force a full resync through two interfaces:

1. CLI: Running agentsview sync --resync
2. HTTP API: Sending POST /api/v1/sync?mode=resync

The HTTP handler in internal/server/huma_routes_sync.go (lines 78-88) routes these requests:

// internal/server/huma_routes_sync.go
func (s *Server) postSync(ctx huma.Context) {
    mode := ctx.Param("mode") // "full", "incremental", "resync"
    var stats syncpkg.SyncStats
    switch mode {
    case "resync":
        stats = s.engine.ResyncAll(ctx, nil)
    default:
        stats = s.engine.SyncAll(ctx, nil)
    }
    // return stats …
}

Use this mode when the database might contain corruption, stale skip-cache entries, or after schema migrations that require complete data reconstruction.

Code Examples and Implementation Details

You can trigger these sync modes programmatically or via command-line interfaces:

Run per-file incremental sync from CLI:

$ agentsview sync --paths /home/me/.cache/agentsview/claude/session.jsonl

Trigger full resync via HTTP:

curl -X POST http://localhost:8080/api/v1/sync?mode=resync

Programmatic usage in Go:

engine := sync.NewEngine(db, cfg)

// incremental sync for specific changed files
stats := engine.SyncPaths([]string{"/path/to/session.jsonl"})

// full discovery incremental sync
stats = engine.SyncAll(context.Background(), nil)

// full resync with atomic database replacement
stats = engine.ResyncAll(context.Background(), nil)

Summary

• Per-file incremental sync (SyncPaths) processes only specified changed files, triggered by the fsnotify watcher in internal/sync/watcher.go or direct API calls with path lists.
• Full discovery incremental sync (SyncAll) walks all directories but skips unchanged files using the skip-cache, triggered by the periodic timer in cmd/agentsview/main.go (default 15-minute intervals).
• Full resync (ResyncAll) creates a new SQLite database, copies all data including insights and metadata, performs an atomic file swap, and is triggered only by explicit CLI --resync flags or HTTP mode=resync requests.
• The skip-cache mechanism optimizes incremental operations by comparing file size, modification time, and data-version to avoid unnecessary parsing.
• All database writes in ResyncAll use bulk operations, while incremental modes use default batch processing with 100-file batches and up to 8 workers.

Frequently Asked Questions

When should I use a full resync instead of incremental sync?

Use full resync when you suspect database corruption, after modifying the session file schema, or when the skip-cache might be stale (for example, if files were modified while AgentsView was not running). According to the kenn-io/agentsview source code, ResyncAll atomically rebuilds the database, ensuring consistency that incremental modes cannot guarantee when metadata tables become desynchronized.

How does AgentsView ensure data consistency during a full resync?

The ResyncAll function in internal/sync/engine.go writes to a temporary database file (suffixed with -resync) and only performs an atomic rename operation after successfully copying insights, orphaned rows, and rebuilding the FTS index. This ensures that readers never encounter a partially constructed database, maintaining ACID guarantees throughout the operation.

What is the performance impact of the periodic full discovery sync?

The full discovery incremental sync (SyncAll) minimizes overhead by consulting the skip-cache before parsing files. When processing unchanged files, it compares size, modification time, and data-version hashes, skipping files that match the cache. With batch processing of 100 files and up to 8 concurrent workers, typical overhead is minimal unless thousands of files have actually changed since the last sync.

Can I disable the automatic incremental sync and trigger updates manually?

Yes. While the default configuration in cmd/agentsview/main.go starts a ticker for periodic sync, you can configure the application to disable automatic timers and rely exclusively on the HTTP API (POST /api/v1/sync) or CLI commands. This is useful for environments where you want strict control over when the sync engine consumes CPU and I/O resources.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}