How to Troubleshoot Sync Failures and File Watcher Issues in agentsview

Sync failures in agentsview typically stem from OS inotify limits, watcher budget exhaustion, or file classification mismatches, diagnosable through the Watcher.WatchRecursiveBudgeted results and SyncStats logs in internal/sync/watcher.go and internal/sync/engine.go.

The kenn-io/agentsview repository maintains a real-time SQLite database synchronized with on-disk agent session files through a tightly coupled pipeline. When you need to troubleshoot sync failures and file watcher issues in agentsview, you are typically debugging interactions between the file watcher (internal/sync/watcher.go) and the sync engine (internal/sync/engine.go). Understanding the specific failure modes of these components allows you to resolve resource exhaustion, parsing errors, and missed file events efficiently.

Understanding the Sync Architecture

agentsview relies on two primary components to keep its database current:

• File watcher (internal/sync/watcher.go): Monitors agent directories for changes using filesystem events, applying a debounce period before triggering sync runs.
• Sync engine (internal/sync/engine.go): Discovers session files, classifies them by agent type, parses their contents, and writes data to SQLite (or performs bulk-loads during full resyncs).

When synchronization breaks, the failure originates in one of these five distinct categories.

Common Root Causes of Sync Failures

Resource Exhaustion and inotify Limits

The most common infrastructure failure occurs when the OS hits its limit for inotify watches or file descriptors. In watcher.go, the function Watcher.WatchRecursiveBudgeted detects this via isWatchResourceExhaustion (lines 25‑27). When the system returns EMFILE (too many open files) or ENOSPC (inotify watch limit exceeded), the watcher sets RecursiveWatchResult.ResourceExhausted = true (lines 9‑12) and falls back to shallow watching, potentially missing deep directory changes.

Watcher Budget Exhaustion

Even without OS limits, the watcher operates within a configured budget. If Watcher.WatchRecursiveBudgeted exhausts its budget (defaulting to math.MaxInt in WatchRecursive) before watching the entire tree, it returns result.BudgetExhausted (lines 102‑104). This leaves portions of your agent directories unwatched and unsynchronized.

File Path Classification Failures

The engine silently ignores files it does not recognize as valid agent sessions. In internal/sync/engine.go, the method Engine.classifyOnePath contains a large switch statement (starting at line 511) that determines file eligibility. If a file path fails to match any supported agent pattern, the sync engine discards it without error, making this failure mode particularly difficult to detect.

Parsing and Database Write Errors

When the engine encounters corrupt session files or database constraints, Engine.syncAllLocked reports these through SyncStats.Failed. These errors bubble up through the sync run but may not stop the watcher from continuing to trigger new sync attempts, creating a loop of failed updates.

Resync Abort Conditions

During a full rebuild initiated by Engine.ResyncAll, the engine evaluates an abortSwap condition (around line 220 of engine.go). If the new database would be "worse" than the existing one—due to empty discovery, excessive parse failures, or other validity checks—the engine aborts the swap, leaving the old (potentially stale) data in place.

Step-by-Step Troubleshooting Workflow

1. Inspect Watcher Logs

Check for lines like watcher error: … or watcher: X file(s) changed, triggering sync in the console output. The watcher logs via log.Printf in the watcher.loop function (lines 63‑76). Run the binary with DEBUG=1 or AGENTSVIEW_LOG=debug to increase verbosity.

2. Verify inotify Limits

Check your system limits to confirm resource exhaustion:

cat /proc/sys/fs/inotify/max_user_watches

If the value is too low (commonly 8192), raise it permanently:

echo "fs.inotify.max_user_watches=524288" | sudo tee /etc/sysctl.d/99-agentsview.conf
sudo sysctl -p /etc/sysctl.d/99-agentsview.conf

After increasing limits, restart agentsview; the ResourceExhausted flag should no longer trigger.

3. Confirm Directory Watch Status

Verify that the watcher actually added your directories by logging the return value of Watcher.WatchRecursiveBudgeted. This function returns Watched and Unwatched counts. If Unwatched is greater than zero, the watcher missed directories due to budget constraints or permission issues.

4. Check Sync Statistics

After a sync run, examine SyncStats printed by engine.SyncPaths or ResyncAll. The CLI (cmd/agentsview/main.go) outputs log.Printf("sync: %d file(s) updated", stats.Synced). Enable verbose mode (-v) to see full statistics including Failed and Skipped counts.

5. Inspect Skip Cache Behavior

Files that repeatedly fail parsing are cached in Engine.skipCache. The engine persists this cache via Engine.persistSkipCache and loads it via Engine.loadSkippedFiles (located in internal/db/db.go). To force a clean run, delete the skipped_files table:

agentsview db exec "DELETE FROM skipped_files"

6. Run a Manual Full Resync

Force a fresh discovery that bypasses watcher limits:

agentsview sync --full

This invokes Engine.ResyncAll. If the resync aborts, logs will contain resync: aborting swap, X synced / Y failed / Z total, indicating why the new database was rejected.

7. Validate Configuration

Verify your environment variables in internal/config/config.go. Confirm that AGENTSVIEW_DATA_DIR and agent-specific paths (e.g., AGENTSVIEW_CLAUDE_DIR) resolve to the correct directories. Run agentsview config dump to inspect the resolved AgentDirs map.

8. Examine File Classification

If specific files refuse to sync, they may fail classification. Add temporary instrumentation to Engine.classifyOnePath just before the final return parser.DiscoveredFile{}, false (after the switch statement ending near line 1090):

log.Printf("unmatched: %s", path)

This reveals which files slip through the classification logic.

Common Failure Patterns and Solutions

Symptom	Root Cause	Fix
"watcher error: too many open files"	OS file descriptor limit (EMFILE) or inotify limit (ENOSPC).	Increase fs.inotify.max_user_watches and per-process limits (ulimit -n 65536).
No sync after editing a session file	Directory not watched (budget exhausted) or file excluded by pattern.	Check WatchRecursiveBudgeted results; raise the budget or thin the directory tree.
Sync runs but files remain "skipped"	Skip cache contains entries from previous parse failures.	Clear the skipped_files table or run a full resync.
Resync aborts with "empty discovery"	AgentDirs configuration points to wrong paths or all files fail classification.	Verify AGENTSVIEW_DATA_DIR and run agentsview config dump to check resolved paths.
Parser errors for specific agents	File name does not match expected patterns (e.g., missing .jsonl suffix).	Consult the agent-specific classification logic in engine.go (search for AgentCodex, AgentClaude, etc.) and rename files accordingly.

Debugging Techniques and Code Examples

Starting the Watcher with Custom Debounce

Use this pattern to instrument the watcher and verify budget results:

import (
    "log"
    "time"
    
    "go.kenn.io/agentsview/internal/sync"
)

func main() {
    // 500ms debounce, ignore .git directories
    w, err := sync.NewWatcher(500*time.Millisecond,
        func(paths []string) {
            log.Printf("paths changed: %v", paths)
        },
        []string{".git"})
    if err != nil {
        log.Fatalf("watcher init: %v", err)
    }

    // Watch with a budget of 10,000 directories
    result := w.WatchRecursiveBudgeted("/home/user/.agentsview", 10000)
    log.Printf("watched=%d unwatched=%d budgetExhausted=%t resourceExhausted=%t",
        result.Watched, result.Unwatched,
        result.BudgetExhausted, result.ResourceExhausted)

    w.Start()
    // Shutdown with w.Stop()
}

Forcing a Single-File Sync

Isolate classification and database write issues for a specific file:

agentsview sync /home/user/.agentsview/claude/project1/session123.jsonl

If this returns "0 file(s) updated", the file failed classification in Engine.classifyOnePath. Check the Claude-specific patterns in internal/sync/engine.go.

Running a Full Resync with Debug Output

Capture detailed logs during a full rebuild:

AGENTSVIEW_LOG=debug agentsview sync --full

Look for:

• watcher: X file(s) changed… – watcher event triggers
• resync: drop temp fts… – full-text search maintenance
• resync: aborting swap… – swap failure reasons

Raising inotify Watch Limits

For Linux systems where agentsview monitors large directory trees:

# Check current limit

cat /proc/sys/fs/inotify/max_user_watches

# Set permanently

echo "fs.inotify.max_user_watches=524288" | sudo tee /etc/sysctl.d/99-agentsview.conf
sudo sysctl -p /etc/sysctl.d/99-agentsview.conf

This prevents the ENOSPC error detected in Watcher.isWatchResourceExhaustion.

Summary

• Resource exhaustion manifests as ResourceExhausted=true in Watcher.WatchRecursiveBudgeted results; fix by raising OS inotify and file descriptor limits.
• Budget exhaustion leaves directories unwatched; monitor the Unwatched count and BudgetExhausted flag.
• Classification failures cause silent skips; instrument Engine.classifyOnePath or force single-file syncs to identify pattern mismatches.
• Skip cache persists transient parse errors; clear it via SQL delete or full resync when files are fixed.
• Resync aborts protect against bad database states; verify AgentDirs configuration when encountering "empty discovery" errors.

Frequently Asked Questions

Why does agentsview stop detecting file changes after running for a while?

This typically indicates inotify resource exhaustion. When the OS hits fs.inotify.max_user_watches or the process hits its file descriptor limit (EMFILE), the watcher in internal/sync/watcher.go sets ResourceExhausted=true and falls back to shallow monitoring. Check cat /proc/sys/fs/inotify/max_user_watches and increase it to 524288 or higher, then restart the application.

How do I force agentsview to re-scan all files from scratch?

Run agentsview sync --full from the CLI. This invokes Engine.ResyncAll in internal/sync/engine.go, which creates a fresh database, discovers all files regardless of the watcher state, and swaps it in atomically. If you suspect cache corruption, also run agentsview db exec "DELETE FROM skipped_files" to clear the skip cache before the full resync.

Why are some session files ignored during sync?

Files are ignored when they fail classification in Engine.classifyOnePath (line 511 of engine.go). Each agent type (Claude, Codex, etc.) has specific filename patterns. If a file lacks the expected extension (e.g., .jsonl) or resides in an unrecognized directory structure, the engine silently skips it. Use agentsview sync <path> on the specific file to test classification; if it returns zero updates, check the classification patterns for that agent type.

What does "resync: aborting swap" mean in the logs?

This indicates the Engine.ResyncAll function aborted the database swap because the new discovery would result in a "worse" state than the existing database. As implemented around line 220 of engine.go, the engine checks abortSwap conditions including empty discovery (zero valid sessions), excessive parse failures, or other validity checks. Verify your AGENTSVIEW_DATA_DIR configuration and check that session files are parseable to prevent the abort.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}