# How AgentsView Detects and Syncs New Agent Sessions: The File Watcher Mechanism Explained

> Learn how AgentsView uses a debounced file watcher with fsnotify in Go. Discover how it detects new agent sessions and syncs them to the database via callbacks.

- Repository: [Kenn Software/agentsview](https://github.com/kenn-io/agentsview)
- Tags: internals
- Published: 2026-07-04

---

**AgentsView uses a debounced file watcher built on Go's `fsnotify` library to monitor session directories, automatically detect new agent sessions, and trigger database synchronization through a callback-driven pipeline.**

The `kenn-io/agentsview` repository maintains a real-time SQLite store by watching directories that contain agent session files. The file watcher mechanism lives in [`internal/sync/watcher.go`](https://github.com/kenn-io/agentsview/blob/main/internal/sync/watcher.go) and wraps the underlying `fsnotify` package with recursive directory monitoring, resource exhaustion handling, and intelligent debouncing to batch rapid filesystem changes.

## Core Architecture in [`internal/sync/watcher.go`](https://github.com/kenn-io/agentsview/blob/main/internal/sync/watcher.go)

The watcher implementation centers on a struct that manages an `fsnotify.Watcher` instance alongside configuration for debounce intervals and exclusion patterns. When initialized via `NewWatcher` (lines 47-69), the system stores a change callback function (`onChange`) that bridges filesystem events to the sync engine. This design keeps the watcher agnostic of agent-specific logic while providing the hooks necessary to trigger database updates.

The architecture supports both deep recursive watching and shallow monitoring. For massive directory trees, `WatchRecursiveBudgeted` (lines 80-124) walks the filesystem and adds each sub-directory to the underlying watch list, tracking root paths for exclusion checks. Conversely, `WatchShallow` (lines 30-38) registers only the top-level directory, relying on the auto-watch feature to catch dynamically created subdirectories.

## Step-by-Step Detection Flow

### Initializing the Watcher with `NewWatcher`

The `NewWatcher` function accepts three parameters: a **debounce duration**, an **onChange callback**, and a slice of **exclude patterns**. It constructs the underlying `fsnotify.Watcher`, normalizes exclusion globs (stripping leading "./" and deduplicating entries), and returns a configured watcher ready for directory registration.

```go
// Create a watcher with 200ms debounce, ignoring version control and temp folders
watcher, err := sync.NewWatcher(
    200*time.Millisecond,
    func(paths []string) {
        // This callback receives batched changed paths
        engine.SyncPaths(paths)
    },
    []string{".git", "node_modules", "**/tmp/**"},
)

```

### Registering Directories: Recursive and Shallow Modes

The watcher provides two registration strategies. **`WatchRecursive`** traverses the entire directory tree, while **`WatchRecursiveBudgeted`** stops when the OS limit for file descriptors is reached, setting a `ResourceExhausted` flag for the sync engine to handle via fallback polling. For large but shallow trees, **`WatchShallow`** monitors only the specified root, though the watcher will still auto-add subdirectories that belong to recursive roots to avoid shadowing.

### The Event Loop and `handleEvent`

After calling `Start()`, a dedicated goroutine executes the `loop` function (lines 55-80). This loop selects across three channels: filesystem events, errors, and stop signals. When `fsnotify` reports a write, create, remove, or rename operation, `handleEvent` (lines 83-99) validates the event type and records the affected path in a thread-safe pending map with a timestamp.

### Auto-Watching New Subdirectories

When a `Create` event targets a directory, the `watchIfDir` helper (lines 105-119) automatically adds it to the monitoring list. This mechanism catches dynamically created session-date folders such as `sessions/2024/09/15/` without requiring manual re-registration. The function consults `shouldExclude` to skip paths matching user-defined glob patterns and avoids adding directories under shallow-only roots.

### Debouncing and Batch Flushing

To prevent sync storms during rapid file modifications, the watcher implements a **debounce mechanism**. A ticker fires at the configured interval, triggering `flush` (lines 130-165). This function scans the pending map, selects paths whose timestamps exceed the debounce duration, removes them from the map, and invokes the `onChange` callback with the complete batch of changed paths. This coalesces multiple rapid events into a single database synchronization operation.

### Triggering the Sync via Callback

The `onChange` callback closure is registered by the sync engine in [`cmd/agentsview/main.go`](https://github.com/kenn-io/agentsview/blob/main/cmd/agentsview/main.go). When invoked, it receives a slice of absolute paths that changed during the debounce window. The engine parses these session files, updates SQLite tables, and optionally pushes changes to PostgreSQL. The watcher itself remains unaware of agent semantics; it purely reports filesystem changes.

## Key Design Features

**Resource-Aware Recursion**: The `WatchRecursiveBudgeted` method tracks OS file descriptor limits. When exhaustion occurs, it marks the error condition and allows the sync engine to fall back to periodic polling for the remaining subtree.

**Exclusion Pattern Matching**: Users specify glob-style patterns (e.g., `**/tmp/**`) that the watcher normalizes and validates via `shouldExclude` before adding directories to the monitor list.

**Thread-Safe Bookkeeping**: Two mutexes protect critical state: `rootsMu` guards the list of watched roots, while `mu` protects the pending change map. This ensures safe concurrent access between the `fsnotify` goroutine and the debounce ticker.

**Shallow Root Handling**: The watcher distinguishes between recursive and shallow roots. When auto-watching new directories, it checks against the shallow root list to prevent over-monitoring massive subtrees that the user explicitly marked as shallow.

## Integration Example

The following example demonstrates setting up the watcher to monitor agent session directories and trigger synchronization:

```go
package main

import (
    "log"
    "time"
    
    "github.com/kenn-io/agentsview/internal/sync"
)

func main() {
    // Initialize with 200ms debounce and exclusions
    watcher, err := sync.NewWatcher(
        200*time.Millisecond,
        func(paths []string) {
            log.Printf("Syncing %d changed paths", len(paths))
            // Engine processes session files and updates SQLite
            syncEngine.SyncPaths(paths)
        },
        []string{".git", "node_modules", "*.tmp"},
    )
    if err != nil {
        log.Fatal(err)
    }
    
    // Watch session directory recursively
    err = watcher.WatchRecursive("/home/user/.agentsview/sessions")
    if err != nil {
        log.Fatal(err)
    }
    
    watcher.Start()
    defer watcher.Stop()
    
    // Block or run other services...
    select {}
}

```

## Summary

- The file watcher mechanism resides in [`internal/sync/watcher.go`](https://github.com/kenn-io/agentsview/blob/main/internal/sync/watcher.go) and wraps Go's `fsnotify` package.
- **`NewWatcher`** initializes the system with debounce configuration and exclusion patterns.
- **`WatchRecursive`** and **`WatchShallow`** provide flexible directory monitoring strategies.
- The event loop processes filesystem events and populates a pending map with timestamps.
- **`watchIfDir`** auto-registers newly created subdirectories, enabling dynamic session folder detection.
- The **`flush`** method debounces rapid changes, batching them into single sync operations.
- An **`onChange`** callback bridges filesystem events to the sync engine in [`cmd/agentsview/main.go`](https://github.com/kenn-io/agentsview/blob/main/cmd/agentsview/main.go).
- Thread-safe mutexes and resource exhaustion handling ensure robust production operation.

## Frequently Asked Questions

### How does the watcher handle newly created session directories?

When the filesystem reports a `Create` event for a directory, the `watchIfDir` function (lines 105-119 in [`internal/sync/watcher.go`](https://github.com/kenn-io/agentsview/blob/main/internal/sync/watcher.go)) automatically adds that directory to the `fsnotify` watch list. This enables the system to detect agent sessions written to date-organized folders like `sessions/2024/09/15/` without requiring manual restarts or re-registration of watch paths.

### What happens when the OS file descriptor limit is reached?

The `WatchRecursiveBudgeted` method tracks the number of directories being watched and stops recursion when the OS limit approaches. It sets a `ResourceExhausted` flag on the watcher, which the sync engine checks in [`cmd/agentsview/main.go`](https://github.com/kenn-io/agentsview/blob/main/cmd/agentsview/main.go). The engine then falls back to periodic polling for the unwatched portion of the directory tree, ensuring no sessions are missed while respecting system limits.

### How does the debounce mechanism improve performance?

The watcher accumulates filesystem events in a pending map and only invokes the sync callback after the configured debounce interval (typically 200ms) has elapsed without new changes. Implemented in the `flush` method (lines 130-165), this coalesces rapid bursts of file operations—such as atomic rewrites or batch session imports—into a single database transaction, reducing SQLite lock contention and PostgreSQL sync overhead.

### Can the watcher exclude specific directories from monitoring?

Yes. The `NewWatcher` constructor accepts a slice of glob patterns (e.g., `".git"`, `"node_modules"`, `"**/tmp/**"`). These patterns are normalized and stored in the watcher struct. Before adding any directory to the `fsnotify` watch list, the `shouldExclude` method checks against these patterns, allowing users to skip version control folders, temporary directories, or large asset caches that would otherwise consume file descriptors and trigger unnecessary sync operations.