How wigolo REST API Authentication and Resource Limits Work: A Deep Dive into the Implementation

The wigolo REST API enforces a fail-closed security model through bind-time gates that prevent public exposure without tokens, and request-time validation that verifies bearer tokens or restricts access to loopback hosts, while external resource protection relies on mandatory polling intervals and per-domain throttling.

The KnockOutEZ/wigolo repository provides a web-intelligence daemon that exposes its functionality via a RESTful JSON-over-HTTP interface. Understanding how wigolo REST API authentication and resource limits are implemented reveals a defense-in-depth strategy designed to prevent accidental exposure and respect external rate limits.

Bind-Time Security: The evaluateBindGate Firewall

Before the server accepts any connections, the system performs a critical safety check in src/daemon/rest/auth.ts. The evaluateBindGate function (lines 60‑77) acts as a circuit breaker that prevents the daemon from starting on non-loopback addresses unless proper authentication is configured.

This function checks two conditions:

• Whether the bind address is loopback (isLoopbackBind)
• Whether an API token is configured

If you attempt to bind to 0.0.0.0 or any external interface without setting the WIGOLO_API_TOKEN environment variable, the daemon exits immediately with an error. This ensures that exposing the service to the network requires an explicit opt-in via token configuration, eliminating the risk of accidental public exposure during development or deployment.

Request-Time Authentication with checkAuth

Once the server passes the bind-time gate, every incoming request undergoes validation through the checkAuth function in src/daemon/rest/auth.ts (lines 21‑62). This middleware distinguishes between two operational modes:

Token Mode: When WIGOLO_API_TOKEN or WIGOLO_API_TOKEN_FILE is set, the server requires every request to include an Authorization: Bearer <token> header. The validation uses a constant-time comparison function (tokenMatches) to prevent timing attacks against the secret.

Open Mode: If no token is configured, the server operates in a restricted loopback-only mode. The checkAuth function enforces this through the isAllowedHost helper (lines 12‑18), which maintains an allow-list containing localhost, 127.0.0.1, ::1, and any explicitly bound host.

Additionally, when running in open mode, the server rejects requests carrying browser-origin headers (lines 52‑60) to prevent CSRF attacks from malicious websites targeting local services.

Token Resolution and Normalization

The authentication system supports flexible secret management through resolveApiToken in src/daemon/rest/auth.ts (lines 15‑34):

1. Direct environment variable: Set WIGOLO_API_TOKEN to the secret string.
2. File-based secrets: Set WIGOLO_API_TOKEN_FILE to a path containing the token, enabling integration with Docker secrets or Kubernetes vaults.

The normalizeToken function (lines 4‑13) sanitizes input by converting whitespace-only strings to null, ensuring that empty environment variables or files containing only newlines do not accidentally enable authentication or bypass security checks.

Resource Limit Enforcement and Rate Limiting

Beyond authentication, wigolo implements resource limits to protect both the host system and external web services from abuse.

Watch Tool Interval Guards

The watch tool, defined in src/tools/watch.ts, enforces a hard minimum polling interval to prevent aggressive scraping that could trigger remote rate limits. At line 89, the code validates that interval_seconds is at least 60 seconds. Attempting to configure a lower value produces the error: "Raise interval_seconds to at least 60 to respect target-site rate limits."

Per-Domain Throttling in the Fetch Router

The fetch router (src/fetch/router.ts) implements sophisticated per-domain rate limiting to respect external service constraints. According to comments at lines 799 and 1072, the router:

• Parses and respects robots.txt directives
• Learns site-specific behavior patterns
• Throttles requests based on observed response times and 429 (Too Many Requests) responses

This prevents a single wigolo instance from overwhelming external APIs and ensures sustainable access to third-party resources.

Configuration and Usage Examples

Starting the daemon with authentication enabled:

# Export a token (or point to a secret file) before starting wigolo

export WIGOLO_API_TOKEN=super-secret-token   # or WIGOLO_API_TOKEN_FILE=/run/secrets/token

# Bind to a remote address – the token is required

wigolo serve --host 0.0.0.0   # Refuses to start without the token (see evaluateBindGate)

Making an authenticated REST call:

curl -X POST http://127.0.0.1:3333/v1/search \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer super-secret-token" \
     -d '{"query":"local‑first web search","max_results":5}'

Running in open (loopback-only) mode:

# No token set → open mode, but only loopback hosts are allowed

wigolo serve                     # Binds to 127.0.0.1 by default

# Successful request from the same machine (no Authorization header needed)

curl -s http://127.0.0.1:3333/v1/search \
     -H "Content-Type: application/json" \
     -d '{"query":"local‑first web search","max_results":5}'

Using the watch tool with compliant intervals:

wigolo watch https://example.com \
     --interval_seconds 120   # Must be ≥ 60 seconds or the server returns an error

Summary

• evaluateBindGate in src/daemon/rest/auth.ts prevents binding to non-loopback addresses without authentication tokens, creating a fail-closed default.
• checkAuth validates every request using bearer tokens in token mode, or restricts access to loopback hosts and non-browser origins in open mode.
• Token resolution supports both direct environment variables (WIGOLO_API_TOKEN) and file-based secrets (WIGOLO_API_TOKEN_FILE), with whitespace normalization to prevent configuration errors.
• Rate limiting is enforced through a 60-second minimum interval in the watch tool (src/tools/watch.ts) and per-domain throttling in the fetch router (src/fetch/router.ts), protecting external services from abuse.

Frequently Asked Questions

How do I configure authentication for the wigolo REST API?

Set the WIGOLO_API_TOKEN environment variable to a secure random string before starting the daemon, or use WIGOLO_API_TOKEN_FILE to point to a file containing the token. When configured, all requests must include the header Authorization: Bearer <your-token>. According to the source code in src/daemon/rest/auth.ts, empty or whitespace-only values are treated as unset, preventing accidental authentication bypass.

What happens if I try to bind wigolo to 0.0.0.0 without a token?

The daemon will refuse to start. The evaluateBindGate function specifically checks if the bind address is non-loopback and whether a token is configured. This prevents accidental exposure of the REST API to the public internet during development or deployment without explicit security configuration.

How does wigolo prevent abuse of external APIs?

The system implements multiple layers of protection. The watch tool enforces a minimum polling interval of 60 seconds (src/tools/watch.ts, line 89) to prevent aggressive scraping. Additionally, the fetch router in src/fetch/router.ts applies per-domain rate limiting, respects robots.txt directives, and throttles requests based on observed site behavior and 429 responses, ensuring wigolo instances remain good citizens of the web ecosystem.

Can I run wigolo without authentication?

Yes, but only in open mode restricted to loopback addresses. Without a token configured, the server allows only requests from localhost, 127.0.0.1, or ::1, and rejects requests containing browser-origin headers to prevent CSRF attacks. This mode is safe for local development but prevents remote access to the API.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}