# How Wigolo Fetch Tier Escalation Works: From Plain HTTP to Headless Browser

> Discover how Wigolo fetch tier escalation moves from plain HTTP to headless browser only when necessary. Optimize your web scraping efficiently.

- Repository: [Towhid Khan/wigolo](https://github.com/KnockOutEZ/wigolo)
- Tags: internals
- Published: 2026-07-19

---

**Wigolo always attempts the cheap plain-HTTP tier first and only escalates to the expensive headless-browser tier when the page requires JavaScript, returns a shell capture, or produces a plain-text error body.**

The `fetch` tool in **KnockOutEZ/wigolo** implements an intelligent **wigolo fetch tier escalation** strategy that balances cost efficiency with content completeness. By starting with lightweight HTTP requests and conditionally upgrading to Playwright-driven browser automation, the system minimizes resource consumption while ensuring dynamic web applications are fully rendered.

## The Tier Escalation Pipeline

The escalation logic resides in the **SmartRouter** invoked by `handleFetch` in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts). The following steps describe the exact decision flow implemented in the source code.

### Step 1: Mode Resolution and SSRF Protection

Every fetch request begins with validation and mode resolution. The `resolveMode` function in [`src/util/mode.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/util/mode.ts) (lines 13-17) processes the requested mode—`default`, `cache`, `stealth`, or others—while `validateFetchUrl` in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 31-68) and `guardFetchUrl` in [`src/watch/ssrf.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/watch/ssrf.ts) (lines 85-100) enforce URL syntax and SSRF safety before any network tier is engaged.

### Step 2: Cache Shortcut

When the mode is not `stealth` and `force_refresh` is false, Wigolo checks for cached content via `getCachedContent` → `isCacheUsable` in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 7-12). A fresh, non-shell cache entry returns immediately, bypassing both the HTTP and browser tiers entirely.

### Step 3: Plain-HTTP Tier Attempt

On cache miss, the router receives the fetch request with options including `{render_js: "auto", …}`. The first attempt uses the **plain-HTTP tier**, a fast TLS request that never spawns a browser. This tier returns a raw result containing `method: "http"` and notably **no** `contentCompleteness` field, as seen in the `router.fetch` call in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 44-51).

### Step 4: Escalation Triggers

The SmartRouter escalates to the Playwright tier when any of three conditions are met:

- **JavaScript Required**: The caller explicitly sets `render_js: "always"` or the router detects that JavaScript is necessary to generate meaningful markup.
- **Shell Capture**: The HTTP response contains only a minimal HTML skeleton, indicated by `contentCompleteness.level === "shell"`.
- **Plain-Text Error Bodies**: The HTTP tier returns a 4xx/5xx status with a body whose `Content-Type` is plain-text, Markdown, JSON, or XML. The router treats this as a failure and retries with the browser tier to capture richer error pages.

This decision logic is handled within the router and observed in `handleFetch` in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 66-68) through inspection of `raw.contentCompleteness`.

### Step 5: Headless Browser Execution

Upon escalation, the system invokes `fetchWithPlaywright` in [`src/fetch/playwright-tier.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/fetch/playwright-tier.ts) (lines 66-71). This tier spawns a Playwright instance, loads the target URL, optionally executes user-provided **actions** (clicks, scrolls, etc.), and returns a result with `method: "browser"` and a `contentCompleteness` object describing the render quality (`full`, `shell`, etc.).

### Step 6: Result Normalization and Auditability

After obtaining the raw tier result, `handleFetch` constructs the public `FetchOutput`. It copies the router-chosen method into `fetch_method` via `out.fetch_method = raw.method` in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 63-66), allowing callers to audit which tier served the content. For browser-tier results, the `content_completeness` label is also propagated (lines 66-68).

### Stealth Mode Bypass

When `mode: "stealth"` is requested, the logic in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 4-7) forces a fresh fetch with no cache lookup and treats any browser-tier failure as a hard error. This prevents anti-bot blocks from being masked by stale cached shells.

## Configuring Fetch Behavior

Developers control the **wigolo fetch tier escalation** behavior through specific parameters that influence the router's tier selection.

### Automatic vs Forced JavaScript Rendering

Setting `render_js: "auto"` allows the SmartRouter to decide whether JavaScript is needed based on initial HTTP responses. Conversely, `render_js: "always"` bypasses the HTTP tier completely, forcing immediate Playwright execution for sites known to require heavy client-side rendering.

### Stealth Mode for Anti-Bot Protection

Use `mode: "stealth"` when fetching from sites with aggressive bot detection. This guarantees a fresh browser context with no cache fallback, ensuring that blocks or CAPTCHAs are captured accurately rather than returning a cached shell or HTTP error.

## Practical Code Examples

The following examples demonstrate how to interact with the tier escalation system:

```typescript
// 1. Simple fetch - tries HTTP first, escalates only if needed
const result = await wigolo.fetch({
  url: 'https://example.com/docs',
  render_js: 'auto',          // "auto" = use browser only when required
});
console.log(result.fetch_method);   // "http" (or "browser" after escalation)

```

```typescript
// 2. Force full JS rendering - bypasses the HTTP tier completely
const jsResult = await wigolo.fetch({
  url: 'https://react-app.example',
  render_js: 'always',        // forces Playwright tier
});
console.log(jsResult.fetch_method); // "browser"

```

```typescript
// 3. Stealth mode - always fresh, never uses cached shells
const stealth = await wigolo.fetch({
  url: 'https://private-site.internal',
  mode: 'stealth',
});

```

## Summary

- **Fast-first strategy**: The plain-HTTP tier in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) is always attempted first unless `force_refresh` or `stealth` mode dictates otherwise.
- **Conditional escalation**: The SmartRouter upgrades to the Playwright tier in [`src/fetch/playwright-tier.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/fetch/playwright-tier.ts) only for JavaScript requirements, shell captures, or plain-text error responses.
- **Full auditability**: The `fetch_method` field in the output reveals which tier served the request (`"http"` or `"browser"`), while `content_completeness` indicates render quality for browser results.
- **Cache awareness**: In `mode: "cache"`, the system never escalates to the browser tier, returning a `cache_miss` error instead of spawning Playwright.

## Frequently Asked Questions

### What triggers wigolo to switch from HTTP to headless browser?

Wigolo escalates from the HTTP tier to the headless-browser tier when the `render_js` parameter is set to `"always"`, when the HTTP response indicates a shell capture (`contentCompleteness.level === "shell"`), or when the HTTP tier returns a plain-text error body (4xx/5xx). The SmartRouter evaluates these conditions after the initial `router.fetch` call in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts).

### How can I force the browser tier to always execute?

Pass `render_js: "always"` in your fetch options. This configuration bypasses the plain-HTTP tier entirely and immediately invokes `fetchWithPlaywright` from [`src/fetch/playwright-tier.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/fetch/playwright-tier.ts), ensuring full JavaScript execution regardless of the initial HTTP response.

### What is the difference between default mode and stealth mode?

Default mode allows cache hits and graceful degradation, potentially returning cached content or HTTP-tier results. Stealth mode (`mode: "stealth"`), implemented in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 4-7), forces a fresh fetch with no cache lookup and treats browser-tier failures as hard errors, making it ideal for auditing anti-bot protections or ensuring completely fresh renders.

### Does wigolo cache headless browser results?

Yes, but with safeguards. The `isCacheUsable` check in [`src/tools/fetch.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/tools/fetch.ts) (lines 7-12) considers cached browser results valid only if they are not shell captures and the mode is not `stealth`. This prevents incomplete renders from being served repeatedly while allowing fully rendered pages to benefit from caching.