# How wigolo REST API Authentication and Resource Limits Work: A Deep Dive into the Implementation

> Explore wigolo REST API authentication and resource limits. Learn about bind-time gates, token validation, and external resource protection through polling and throttling.

- Repository: [Towhid Khan/wigolo](https://github.com/KnockOutEZ/wigolo)
- Tags: deep-dive
- Published: 2026-07-20

---

**The wigolo REST API enforces a fail-closed security model through bind-time gates that prevent public exposure without tokens, and request-time validation that verifies bearer tokens or restricts access to loopback hosts, while external resource protection relies on mandatory polling intervals and per-domain throttling.**

The [KnockOutEZ/wigolo](https://github.com/KnockOutEZ/wigolo) repository provides a web-intelligence daemon that exposes its functionality via a RESTful JSON-over-HTTP interface. Understanding how wigolo REST API authentication and resource limits are implemented reveals a defense-in-depth strategy designed to prevent accidental exposure and respect external rate limits.

## Bind-Time Security: The `evaluateBindGate` Firewall

Before the server accepts any connections, the system performs a critical safety check in [`src/daemon/rest/auth.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/daemon/rest/auth.ts). The `evaluateBindGate` function (lines 60‑77) acts as a circuit breaker that prevents the daemon from starting on non-loopback addresses unless proper authentication is configured.

This function checks two conditions:
- **Whether the bind address is loopback** (`isLoopbackBind`)
- **Whether an API token is configured**

If you attempt to bind to `0.0.0.0` or any external interface without setting the `WIGOLO_API_TOKEN` environment variable, the daemon exits immediately with an error. This ensures that exposing the service to the network requires an explicit opt-in via token configuration, eliminating the risk of accidental public exposure during development or deployment.

## Request-Time Authentication with `checkAuth`

Once the server passes the bind-time gate, every incoming request undergoes validation through the `checkAuth` function in [`src/daemon/rest/auth.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/daemon/rest/auth.ts) (lines 21‑62). This middleware distinguishes between two operational modes:

**Token Mode:** When `WIGOLO_API_TOKEN` or `WIGOLO_API_TOKEN_FILE` is set, the server requires every request to include an `Authorization: Bearer <token>` header. The validation uses a constant-time comparison function (`tokenMatches`) to prevent timing attacks against the secret.

**Open Mode:** If no token is configured, the server operates in a restricted loopback-only mode. The `checkAuth` function enforces this through the `isAllowedHost` helper (lines 12‑18), which maintains an allow-list containing `localhost`, `127.0.0.1`, `::1`, and any explicitly bound host.

Additionally, when running in open mode, the server rejects requests carrying browser-origin headers (lines 52‑60) to prevent CSRF attacks from malicious websites targeting local services.

## Token Resolution and Normalization

The authentication system supports flexible secret management through `resolveApiToken` in [`src/daemon/rest/auth.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/daemon/rest/auth.ts) (lines 15‑34):

1. **Direct environment variable:** Set `WIGOLO_API_TOKEN` to the secret string.
2. **File-based secrets:** Set `WIGOLO_API_TOKEN_FILE` to a path containing the token, enabling integration with Docker secrets or Kubernetes vaults.

The `normalizeToken` function (lines 4‑13) sanitizes input by converting whitespace-only strings to `null`, ensuring that empty environment variables or files containing only newlines do not accidentally enable authentication or bypass security checks.

## Resource Limit Enforcement and Rate Limiting

Beyond authentication, wigolo implements resource limits to protect both the host system and external web services from abuse.

### Watch Tool Interval Guards

The **watch** tool, defined in [`src/tools/watch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/watch.ts), enforces a hard minimum polling interval to prevent aggressive scraping that could trigger remote rate limits. At line 89, the code validates that `interval_seconds` is at least 60 seconds. Attempting to configure a lower value produces the error: *"Raise interval_seconds to at least 60 to respect target-site rate limits."*

### Per-Domain Throttling in the Fetch Router

The fetch router ([`src/fetch/router.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/fetch/router.ts)) implements sophisticated per-domain rate limiting to respect external service constraints. According to comments at lines 799 and 1072, the router:
- Parses and respects [`robots.txt`](https://github.com/KnockOutEZ/wigolo/blob/main/robots.txt) directives
- Learns site-specific behavior patterns
- Throttles requests based on observed response times and 429 (Too Many Requests) responses

This prevents a single wigolo instance from overwhelming external APIs and ensures sustainable access to third-party resources.

## Configuration and Usage Examples

Starting the daemon with authentication enabled:

```bash

# Export a token (or point to a secret file) before starting wigolo

export WIGOLO_API_TOKEN=super-secret-token   # or WIGOLO_API_TOKEN_FILE=/run/secrets/token

# Bind to a remote address – the token is required

wigolo serve --host 0.0.0.0   # Refuses to start without the token (see evaluateBindGate)

```

Making an authenticated REST call:

```bash
curl -X POST http://127.0.0.1:3333/v1/search \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer super-secret-token" \
     -d '{"query":"local‑first web search","max_results":5}'

```

Running in open (loopback-only) mode:

```bash

# No token set → open mode, but only loopback hosts are allowed

wigolo serve                     # Binds to 127.0.0.1 by default

# Successful request from the same machine (no Authorization header needed)

curl -s http://127.0.0.1:3333/v1/search \
     -H "Content-Type: application/json" \
     -d '{"query":"local‑first web search","max_results":5}'

```

Using the watch tool with compliant intervals:

```bash
wigolo watch https://example.com \
     --interval_seconds 120   # Must be ≥ 60 seconds or the server returns an error

```

## Summary

- **`evaluateBindGate`** in [`src/daemon/rest/auth.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/daemon/rest/auth.ts) prevents binding to non-loopback addresses without authentication tokens, creating a fail-closed default.
- **`checkAuth`** validates every request using bearer tokens in token mode, or restricts access to loopback hosts and non-browser origins in open mode.
- **Token resolution** supports both direct environment variables (`WIGOLO_API_TOKEN`) and file-based secrets (`WIGOLO_API_TOKEN_FILE`), with whitespace normalization to prevent configuration errors.
- **Rate limiting** is enforced through a 60-second minimum interval in the watch tool ([`src/tools/watch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/watch.ts)) and per-domain throttling in the fetch router ([`src/fetch/router.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/fetch/router.ts)), protecting external services from abuse.

## Frequently Asked Questions

### How do I configure authentication for the wigolo REST API?

Set the `WIGOLO_API_TOKEN` environment variable to a secure random string before starting the daemon, or use `WIGOLO_API_TOKEN_FILE` to point to a file containing the token. When configured, all requests must include the header `Authorization: Bearer <your-token>`. According to the source code in [`src/daemon/rest/auth.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/daemon/rest/auth.ts), empty or whitespace-only values are treated as unset, preventing accidental authentication bypass.

### What happens if I try to bind wigolo to 0.0.0.0 without a token?

The daemon will refuse to start. The `evaluateBindGate` function specifically checks if the bind address is non-loopback and whether a token is configured. This prevents accidental exposure of the REST API to the public internet during development or deployment without explicit security configuration.

### How does wigolo prevent abuse of external APIs?

The system implements multiple layers of protection. The **watch** tool enforces a minimum polling interval of 60 seconds ([`src/tools/watch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/watch.ts), line 89) to prevent aggressive scraping. Additionally, the fetch router in [`src/fetch/router.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/fetch/router.ts) applies per-domain rate limiting, respects [`robots.txt`](https://github.com/KnockOutEZ/wigolo/blob/main/robots.txt) directives, and throttles requests based on observed site behavior and 429 responses, ensuring wigolo instances remain good citizens of the web ecosystem.

### Can I run wigolo without authentication?

Yes, but only in **open mode** restricted to loopback addresses. Without a token configured, the server allows only requests from `localhost`, `127.0.0.1`, or `::1`, and rejects requests containing browser-origin headers to prevent CSRF attacks. This mode is safe for local development but prevents remote access to the API.