How Wigolo's Watch and Diff Tool Detect and Report Changes to Monitored URLs

Wigolo detects changes by comparing SHA-256 hashes of fetched content in src/watch/scheduler.ts, then generates detailed line-wise diffs via the LCS-based engine in src/cache/diff-engine.ts when changes are confirmed.

Wigolo provides a robust change-monitoring system for URLs through its watch and diff capabilities. This open-source tool from KnockOutEZ/wigolo uses cryptographic hashing and lazy background execution to track content modifications efficiently. Understanding how wigolo's watch and diff tool detect and report changes to monitored URLs reveals a two-stage architecture that separates lightweight detection from comprehensive diff generation.

Core Architecture Overview

Wigolo implements change detection through two cooperating components:

Component	Responsibility	Primary Source File
Watch scheduler	Periodically fetches monitored URLs, hashes content, compares hashes, and surfaces change reports	src/watch/scheduler.ts
Diff engine	Computes line-wise or word-wise diffs and formats them as unified diffs, hunk lists, or concise summaries	src/cache/diff-engine.ts

The watch scheduler handles the heavy lifting of periodic monitoring, while the diff engine provides detailed text comparison when users invoke the diff tool directly or when a change requires detailed inspection.

Change Detection Workflow in the Watch Scheduler

Registering Watch Jobs

When you create a watch job using wigolo watch create, the request flows through src/tools/watch.ts to the handleWatch function. This stores a job record in the SQLite-backed store (src/watch/store.ts) containing:

• The target URL
• Optional CSS selector for partial content monitoring
• Notification endpoint for webhooks
• Last known content_hash
• Check interval in seconds

Before storage, guardUrl in src/watch/ssrf.ts validates the URL to prevent server-side request forgery attacks.

The runCheck Detection Logic

The core detection logic resides in the runCheck function exported from src/watch/scheduler.ts:

export async function runCheck(job: WatchJob, router: SmartRouter): Promise<ChangeReport> {
  // SSRF guard - re-validate the URL
  guardUrl(job.url);
  
  // Fetch the URL
  const fetched = await handleFetch(job.url, router);
  
  // Compute current hash
  const currentHash = fetched.data.content_hash ??
                     createHash('sha256').update(fetched.data.markdown ?? '').digest('hex');
  
  // Compare with stored hash
  if (!job.last_content_hash) {
    // First check - establish baseline only
    await recordCheck(job.id, currentHash, false);
    return { changed: false, current_hash: currentHash };
  }
  
  if (currentHash === job.last_content_hash) {
    return { changed: false, current_hash: currentHash };
  }
  
  // Change detected
  const report: ChangeReport = {
    changed: true,
    previous_hash: job.last_content_hash,
    current_hash: currentHash,
    diff_summary: computeDiffSummary('', fetched.data.markdown)
  };
  
  await recordCheck(job.id, currentHash, true);
  return report;
}

Hashing strategy: Wigolo prefers the content_hash produced by the fetch tool (src/tools/fetch.ts), which represents a SHA-256 hash of the entire extracted HTML body. If legacy cache entries lack this field, the scheduler falls back to hashing the raw markdown content.

Baseline handling: The first successful check stores the hash as a baseline without reporting a change. Subsequent checks compare against this stored value.

Background Execution Strategy

Rather than using a dedicated daemon, Wigolo implements lazy background execution through scheduleOverdueCheck. Every non-watch tool call triggers this function, which schedules triggerOverdueJobs to:

1. Retrieve all jobs where next_check_at is in the past via getOverdueJobs (src/watch/store.ts)
2. Re-read each job to verify it hasn't been paused or deleted
3. Execute runCheck for each overdue job
4. Swallow exceptions to ensure caller latency remains unaffected

This design eliminates the need for a persistent background process while ensuring regular monitoring.

Diff Generation and Reporting

The Diff Engine Implementation

When changes are detected or when users manually invoke wigolo diff, the system delegates to computeDiffEnvelope in src/cache/diff-engine.ts. This function implements a longest-common-subsequence (LCS) algorithm supporting three granularity levels:

• line - Line-by-line comparison
• word - Word-level diffing for prose
• section - Structural section comparison

The engine returns a structured envelope containing:

• changed - Boolean flag indicating modification
• summary - Human-readable statistics ("+12 lines, -3 lines")
• unified_diff - Classic unified diff string with --- and +++ headers
• hunks - Array of diff hunks for programmatic consumption
• truncated - Boolean indicating when input exceeds size limits

For quick comparisons within the scheduler, computeDiffSummary in src/cache/diff-summary.ts generates approximate line counts without computing the full LCS.

Notification Delivery

When runCheck detects a change and the job defines a notification URL, it POSTs a JSON payload using Node's native fetch with redirect: 'manual' to prevent SSRF bypasses:

{
  "url": "https://example.com/blog/post",
  "changed": true,
  "previous_hash": "a1b2c3...",
  "current_hash": "d4e5f6...",
  "diff_summary": "Added 12 lines, removed 3 lines (≈ 450 chars)"
}

Failures are logged but not retried to prevent blocking the scheduler.

Practical Usage Examples

Create a Monitored Watch Job

wigolo watch add https://example.com/blog/post --interval=3600 --notify=https://hooks.example.com/wigolo

This stores a job checking every hour with webhook notifications.

List Active Monitors

wigolo watch list

Displays job IDs, URLs, intervals, and last check timestamps.

Force an Immediate Check

wigolo watch check 42

Returns immediate status for job ID 42:

{
  "url": "https://example.com/blog/post",
  "changed": true,
  "previous_hash": "a1b2c3...",
  "current_hash": "d4e5f6...",
  "diff_summary": "Added 12 lines, removed 3 lines (≈ 450 chars)"
}

Generate Full Unified Diff

wigolo diff https://example.com/blog/post --output=unified

Produces output compatible with git apply.

Compare Inline Text

wigolo diff --old="Hello world\nLine A" --new="Hello world\nLine B"

Returns a diff envelope without network requests.

Summary

• Hash-based detection: Wigolo uses SHA-256 hashes in src/watch/scheduler.ts to detect changes without storing full page content.
• Two-stage architecture: Lightweight hash comparison runs automatically; detailed LCS-based diffing runs on-demand via src/cache/diff-engine.ts.
• Lazy execution: The scheduleOverdueCheck mechanism triggers checks during regular tool usage, eliminating the need for a background daemon.
• SSRF protection: URL validation via guardUrl occurs at registration and every check cycle.
• Flexible reporting: Change reports include hashes, summaries, and optional webhook delivery with manual redirect handling.

Frequently Asked Questions

How does Wigolo handle the first check of a new watch job?

The first successful check establishes a baseline by storing the content hash via recordCheck without setting changed: true. This prevents false positives when initially monitoring a URL, ensuring you only receive notifications for actual subsequent modifications.

What hashing algorithm does Wigolo use for change detection?

Wigolo uses SHA-256 hashing throughout the detection pipeline. The fetch tool in src/tools/fetch.ts generates the primary content_hash, while the scheduler in src/watch/scheduler.ts computes a fallback hash from markdown content using Node's crypto.createHash('sha256') when legacy data lacks the pre-computed hash.

How does Wigolo prevent SSRF attacks during URL monitoring?

Wigolo implements SSRF protection through guardUrl in src/watch/ssrf.ts, which validates URLs at registration time and re-validates them during every runCheck execution. Additionally, webhook notifications use redirect: 'manual' to prevent attackers from exploiting open redirects to hit internal services.

What's the difference between the diff summary and the full diff output?

The diff summary generated by computeDiffSummary in src/cache/diff-summary.ts provides quick line-count statistics ("+N lines, -M lines") during the watch check to minimize overhead. The full diff computed by computeDiffEnvelope in src/cache/diff-engine.ts provides complete LCS-based comparison with unified diff formatting, hunk arrays, and word-level granularity options when you invoke the diff tool directly.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}