# wigolo | Towhid Khan | Knowledge Base | Instagit

The go-to web for your AI coding agent — local-first search, fetch, crawl & research over MCP. No API keys, no cloud, $0/query. Public beta.

GitHub Stars: 1.8k

Repository: https://github.com/KnockOutEZ/wigolo

---

## Articles

### [How Wigolo Per-Domain Fetch Learning Works: Inspection and Reset Guide](/KnockOutEZ/wigolo/how-does-wigolo-s-per-domain-fetch-learning-work-and-how-to-inspect-or-clear-it)

Learn how Wigolo per-domain fetch learning optimizes strategies using TLS, cookies, and timers. Inspect and reset these settings with the wigolo tune CLI tool.

- Tags: how-to-guide
- Published: 2026-07-29

### [How Wigolo's CLI Shell Mode Supports NDJSON Piping for Data Processing](/KnockOutEZ/wigolo/how-does-wigolo-s-cli-shell-mode-support-ndjson-piping-for-data-processing)

Wigolo's CLI shell mode supports NDJSON piping for efficient data processing. Route machine-readable data to stdout and messages to stderr for seamless pipeline integration.

- Tags: how-to-guide
- Published: 2026-07-29

### [How to Troubleshoot Browser Launch Failures When Using wigolo on Linux](/KnockOutEZ/wigolo/how-to-troubleshoot-common-browser-launch-failures-when-using-wigolo-on-linux)

Troubleshoot wigolo browser launch failures on Linux. Run wigolo warmup and wigolo doctor to solve common issues and ensure smooth browser automation.

- Tags: how-to-guide
- Published: 2026-07-29

### [Privacy and Security Implications of Using Wigolo: A Local‑First Web Intelligence Tool](/KnockOutEZ/wigolo/what-are-the-privacy-and-security-implications-of-using-a-local-first-web-intelligence-tool-like-wigolo)

Explore the privacy and security of Wigolo, a local-first web intelligence tool. Discover how Wigolo's architecture ensures your data remains private and secure by default.

- Tags: deep-dive
- Published: 2026-07-29

### [How Wigolo's Tool Scoring and Evidence System Ensure Explainable Results](/KnockOutEZ/wigolo/how-does-wigolo-s-tool-scoring-and-evidence-system-ensure-explainable-results)

Discover how Wigolo's tool scoring and evidence system ensures explainable results with parallel scores and structured evidence objects for complete ranking transparency.

- Tags: deep-dive
- Published: 2026-07-29

### [Wigolo Performance Metrics and Resource Requirements: Optimization Guide](/KnockOutEZ/wigolo/what-are-the-typical-performance-metrics-and-resource-requirements-for-running-wigolo)

Discover Wigolo performance metrics and resource needs. Learn how to optimize with environment caps for efficient deployment on modest VPS with this guide from KnockOutEZ.

- Tags: performance
- Published: 2026-07-29

### [How the Wigolo MCP Server Communicates with AI Agents: A Technical Deep Dive](/KnockOutEZ/wigolo/how-does-the-wigolo-mcp-server-communicate-with-ai-agents-like-claude-code-and-cursor)

Discover how the wigolo MCP server communicates with AI agents like Claude Code and Cursor using a stdio-based JSON protocol. Learn about child process spawning and MCP message exchange over stdin/stdout.

- Tags: deep-dive
- Published: 2026-07-29

### [How wigolo's find_similar Tool Implements 3-Way Fusion for Intelligent Document Retrieval](/KnockOutEZ/wigolo/explain-the-data-flow-and-mechanism-of-wigolo-s-find-similar-tool-using-3-way-fusion)

Discover how wigolo's find_similar tool uses 3-way fusion including FTS5, semantic search, and web results for intelligent document retrieval. Get unified, scaled rankings.

- Tags: deep-dive
- Published: 2026-07-29

### [How to Use the Wigolo Extract Tool for Structured Data Extraction (JSON Schema & Tables)](/KnockOutEZ/wigolo/how-to-utilize-wigolo-s-extract-tool-for-structured-data-extraction-json-schema-tables)

Learn how to use the Wigolo extract tool to get structured data from HTML. Convert unstructured content into JSON Schema objects or markdown tables easily.

- Tags: how-to-guide
- Published: 2026-07-29

### [How Wigolo's Crawl Tool Manages BFS/DFS Traversal, Sitemaps, and Robots.txt Compliance](/KnockOutEZ/wigolo/how-does-wigolo-s-crawl-tool-manage-bfs-dfs-sitemaps-and-robots-txt-compliance)

Discover how Wigolo's crawl tool manages BFS DFS traversal sitemaps and robots txt compliance. Learn about its configurable queue system and RateLimiter for efficient web crawling.

- Tags: how-to-guide
- Published: 2026-07-29

### [Complete Guide to Wigolo Environment Variables for Configuration and Performance](/KnockOutEZ/wigolo/which-environment-variables-are-available-to-customize-wigolo-s-behavior-and-performance)

Discover over 30 WIGOLO_* environment variables to customize Wigolo's behavior data directories TLS impersonation API security and SDK performance for complete runtime control.

- Tags: how-to-guide
- Published: 2026-07-29

### [Deploying wigolo using Docker for Self-Hosting: Complete Setup Guide](/KnockOutEZ/wigolo/what-are-the-steps-for-deploying-wigolo-using-docker-for-self-hosting)

Self-host wigolo easily with Docker. Follow our complete setup guide to deploy wigolo using Docker Compose, ensuring secure API tokens and persistent data for your application.

- Tags: how-to-guide
- Published: 2026-07-29

### [How to Use Wigolo’s Diff and Watch Tools to Track Website Changes](/KnockOutEZ/wigolo/how-to-use-wigolo-s-diff-and-watch-tool-to-track-website-changes)

Learn how to use wigolo diff and watch tools to effectively track website changes. Discover how these MCP tools compare page versions and monitor sites for real time updates.

- Tags: how-to-guide
- Published: 2026-07-29

### [Supported LLM Providers for Synthesis in Wigolo: A Complete Configuration Guide](/KnockOutEZ/wigolo/what-llm-providers-are-supported-by-wigolo-for-synthesis-and-how-to-configure-them)

Configure supported LLM providers like Anthropic, OpenAI, Gemini, and Groq for synthesis in Wigolo. Learn setup with environment variables, keystore, or JSON config.

- Tags: how-to-guide
- Published: 2026-07-29

### [How Wigolo's Research Tool Decomposes Questions and Generates Synthesized Reports](/KnockOutEZ/wigolo/how-does-wigolo-s-research-tool-function-question-decomposition-and-synthesized-reports)

Discover how Wigolo's research tool decomposes questions, executes parallel searches, and synthesizes results into citation-rich markdown reports with LLM generation or deterministic fallback.

- Tags: deep-dive
- Published: 2026-07-29

### [How to Create Custom Search Engine Plugins for wigolo](/KnockOutEZ/wigolo/how-can-developers-create-custom-search-engine-plugins-for-wigolo)

Learn how to create custom search engine plugins for wigolo. Implement a Node module, follow the SearchEngine interface, and install it with the wigolo CLI.

- Tags: how-to-guide
- Published: 2026-07-29

### [How to Integrate wigolo with Popular AI Agent Frameworks: LangChain, CrewAI, and LlamaIndex](/KnockOutEZ/wigolo/how-to-integrate-wigolo-with-langchain-crewai-and-llamaindex)

Integrate wigolo with LangChain, CrewAI, and LlamaIndex effortlessly. Utilize official wrappers for zero-cost, private web operations with wigolo's AI tools. No API keys needed.

- Tags: how-to-guide
- Published: 2026-07-29

### [How Wigolo Implements Keyword and Vector Hybrid Indexing for Local Cache Search](/KnockOutEZ/wigolo/how-does-wigolo-s-local-cache-manage-data-with-keyword-and-vector-hybrid-indexing)

Discover how Wigolo delivers lightning-fast local cache search with hybrid keyword and vector indexing. Learn about its RRF fusion and graceful degradation for optimal performance.

- Tags: internals
- Published: 2026-07-29

### [Architecture of Wigolo's On-Device ML Components for Embeddings and Reranking](/KnockOutEZ/wigolo/what-is-the-architecture-of-wigolo-s-on-device-ml-components-for-embeddings-and-reranking)

Explore Wigolo's on-device ML architecture for embeddings and reranking. Discover its modular, privacy-first RAG stack powered by Fastembed and Transformers.js.

- Tags: architecture
- Published: 2026-07-29

### [How to Set Up and Configure wigolo's REST API Server with Token Authentication](/KnockOutEZ/wigolo/how-to-set-up-and-configure-wigolo-s-rest-api-server-with-token-authentication)

Learn to set up and configure wigolo's REST API server with token authentication. Secure your API endpoints using token authorization for enhanced security.

- Tags: how-to-guide
- Published: 2026-07-29

### [How Wigolo Implements Multi-Engine Search with Rank Fusion and ML Reranking](/KnockOutEZ/wigolo/explain-wigolo-s-multi-engine-search-implementation-and-rank-fusion-with-ml-reranking)

Discover how Wigolo implements multi-engine search using Rank Fusion and ML reranking. Optimize your search results today.

- Tags: deep-dive
- Published: 2026-07-29

### [How Wigolo Handles Anti-Bot Challenges and SPA Pages in Its Fetch Router](/KnockOutEZ/wigolo/how-does-wigolo-handle-anti-bot-challenges-and-spa-pages-with-its-fetch-router)

Discover how Wigolo's fetch router tackles anti-bot challenges and SPA pages using a multi-tier strategy from HTTP to Playwright browser automation.

- Tags: how-to-guide
- Published: 2026-07-29

### [How wigolo REST API Authentication and Resource Limits Work: A Deep Dive into the Implementation](/KnockOutEZ/wigolo/how-does-wigolo-rest-api-implement-authentication-resource-limits)

Explore wigolo REST API authentication and resource limits. Learn about bind-time gates, token validation, and external resource protection through polling and throttling.

- Tags: deep-dive
- Published: 2026-07-20

### [How the Wigolo Research Tool Decomposes Complex Questions into Sub‑Queries](/KnockOutEZ/wigolo/how-does-wigolo-research-decompose-complex-questions-sub-queries)

Learn how the Wigolo research tool decomposes complex questions into sub-queries using LLMs, templates, and heuristics for comprehensive search and efficient computation.

- Tags: how-to-guide
- Published: 2026-07-20

### [How wigolo Implements Reciprocal-Rank Fusion: A Deep Dive into the RRF Search Algorithm](/KnockOutEZ/wigolo/how-does-wigolo-search-implement-reciprocal-rank-fusion)

Discover how wigolo implements reciprocal-rank fusion in its search algorithm. Learn how ranked lists are combined for a unified relevance ranking with the 1/(k + rank) formula.

- Tags: deep-dive
- Published: 2026-07-20

### [Framework Integrations for wigolo: LangChain Tools and SDK Compatibility](/KnockOutEZ/wigolo/wigolo-framework-integrations-langchain-tools-mapping)

Explore wigolo framework integrations like LangChain, CrewAI, and LlamaIndex. Discover how wigolo's LangChain tools map to its MCP surface with full semantics preserved.

- Tags: framework-integrations
- Published: 2026-07-19

### [How Wigolo Fetch Tier Escalation Works: From Plain HTTP to Headless Browser](/KnockOutEZ/wigolo/wigolo-fetch-tier-escalation-plain-http-headless-browser)

Discover how Wigolo fetch tier escalation moves from plain HTTP to headless browser only when necessary. Optimize your web scraping efficiently.

- Tags: internals
- Published: 2026-07-19

### [How to Configure Wigolo for Self-Hosting with Docker and Secure Token Authentication](/KnockOutEZ/wigolo/configure-wigolo-self-hosting-docker-secure-token-authentication)

Self-host Wigolo using Docker. Configure token authentication, bind the daemon to 0.0.0.0, and persist data for secure, reliable operation.

- Tags: how-to-guide
- Published: 2026-07-19

### [Wigolo Warmup Process: How to Download Browser Binaries, Embeddings, and Reranker Models](/KnockOutEZ/wigolo/wigolo-warmup-process-manually-trigger-downloads-browser-embeddings-reranker)

Learn the Wigolo warmup process to download browsers embeddings and reranker models for offline operation. Ensure your Wigolo installation is ready with this essential guide.

- Tags: how-to-guide
- Published: 2026-07-19

### [How to Integrate Wigolo with Vercel AI SDK for Edge-Friendly Tool Usage](/KnockOutEZ/wigolo/integrate-wigolo-vercel-ai-sdk-edge-friendly-tool-usage)

Integrate Wigolo with Vercel AI SDK for edge-friendly tool usage. Run LLM-agnostic search fetch and extract operations efficiently on Vercel’s edge network.

- Tags: how-to-guide
- Published: 2026-07-19

### [Wigolo Local-Only Data Storage Security Model: Complete Technical Guide](/KnockOutEZ/wigolo/security-model-wigolo-local-only-data-storage-home-wigolo)

Explore Wigolo's local-only data storage security model within ~/.wigolo/. Learn how Wigolo secures your secrets via OS keychain or AES-256-GCM with zero external transmission.

- Tags: technical-guide
- Published: 2026-07-19

### [How Wigolo's Watch and Diff Tool Detect and Report Changes to Monitored URLs](/KnockOutEZ/wigolo/wigolo-watch-diff-tool-detect-report-changes-monitored-urls)

Learn how Wigolo's watch and diff tool detects URL changes using SHA-256 hashes and generates detailed line-by-line diffs with its LCS-based engine. Stay informed about web content updates.

- Tags: how-to-guide
- Published: 2026-07-19

### [How to Debug Wigolo Components Using the Doctor Command and Tune Inspect](/KnockOutEZ/wigolo/debug-wigolo-components-doctor-command-tune-inspect)

Debug Wigolo components efficiently using wigolo doctor for health checks and wigolo tune inspect for per-domain self-tuning data. Optimize your fetch routing decisions now.

- Tags: how-to-guide
- Published: 2026-07-19

### [How Wigolo's On-Device Embeddings and Reranker Avoid API Costs While Maintaining Quality](/KnockOutEZ/wigolo/wigolo-on-device-embeddings-reranker-avoid-api-costs-maintain-quality)

Discover how Wigolo's on-device embeddings and reranker slash API costs. Learn how BGE models run locally with ONNX Runtime for efficient, high-quality inference.

- Tags: deep-dive
- Published: 2026-07-19

### [Wigolo Byte-Offset Source Span Tracking: How It Enables Verifiable Citations](/KnockOutEZ/wigolo/wigolo-byte-offset-source-span-tracking-verifiable-citations)

Discover Wigolo's byte-offset source span tracking for verifiable citations. Learn how exact character ranges and SHA-1 hashes ensure independent verification of your sources.

- Tags: how-to-guide
- Published: 2026-07-19

### [How to Configure Wigolo Behind a Corporate Proxy with TLS Inspection](/KnockOutEZ/wigolo/configure-wigolo-corporate-proxy-tls-inspection)

Configure Wigolo behind a corporate proxy with TLS inspection. Set PROXY_URL, secure credentials, and export CA cert via NODE_EXTRA_CA_CERTS to establish trust.

- Tags: how-to-guide
- Published: 2026-07-19

### [How Wigolo Handles Rate Limiting, robots.txt Compliance, and Polite Crawling](/KnockOutEZ/wigolo/wigolo-handle-rate-limiting-robots-txt-compliance-polite-crawling)

Learn how Wigolo ensures polite crawling with robots.txt compliance, Crawl-delay parsing, and strict rate limiting for efficient web scraping and data collection.

- Tags: how-to-guide
- Published: 2026-07-19

### [Core vs Hybrid vs Legacy Search in Wigolo: Backend Architecture Explained](/KnockOutEZ/wigolo/difference-wigolo-core-search-backend-hybrid-legacy-search-modes)

Understand Wigolo's core hybrid and legacy search backends. Explore the differences in ranking sophistication content processing and failover for your technical needs.

- Tags: architecture
- Published: 2026-07-19

### [How Wigolo Decomposes Questions and Synthesizes Cited Reports: A Technical Deep Dive](/KnockOutEZ/wigolo/wigolo-research-tool-decompose-questions-synthesize-cited-reports)

Explore how Wigolo decomposes questions and synthesizes reports. Discover its research pipeline for structured, citation-rich outputs using parallel searches and LLM sampling.

- Tags: deep-dive
- Published: 2026-07-19

### [How to Set Up wigolo with LangChain, CrewAI, or LlamaIndex for RAG Applications](/KnockOutEZ/wigolo/setup-wigolo-langchain-crewai-llamaindex-rag-applications)

Learn how to set up wigolo with LangChain, CrewAI, or LlamaIndex for RAG applications. Install wigolo, initialize the client, and ingest web content into your vector index easily.

- Tags: how-to-guide
- Published: 2026-07-19

### [Wigolo Environment Variables: Configuring Search, Caching, and Browser Behavior](/KnockOutEZ/wigolo/environment-variables-configure-wigolo-search-caching-browser-behavior)

Discover Wigolo environment variables to easily configure search caching, eviction rules, and headless browser automation without code changes. Optimize your Wigolo setup now.

- Tags: configuration
- Published: 2026-07-19

### [How Wigolo's Autonomous Agent Plans, Searches, Fetches, Extracts, and Synthesizes Data](/KnockOutEZ/wigolo/wigolo-autonomous-agent-plan-search-fetch-extract-synthesize-data)

Discover how Wigolo's autonomous agent plans, searches, fetches, extracts, and synthesizes data using a Python SDK and Rust server daemon for efficient orchestration.

- Tags: how-to-guide
- Published: 2026-07-19

### [How to Write Custom Plugins for Wigolo: Adding Search Engines and Site Extractors](/KnockOutEZ/wigolo/write-custom-plugins-wigolo-add-search-engines-site-extractors)

Learn to write custom Wigolo plugins by creating Node modules for search engines and site extractors. Follow simple steps to extend Wigolo's capabilities and discover new data sources.

- Tags: how-to-guide
- Published: 2026-07-19

### [Wigolo MCP Server Mode vs REST API Mode: Key Differences and When to Use Each](/KnockOutEZ/wigolo/differences-wigolo-mcp-server-mode-rest-api-mode)

Explore Wigolo MCP server mode vs REST API: understand authentication needs, use cases, and choose the best mode for your project. Optimize your integration today.

- Tags: comparison
- Published: 2026-07-19

### [How Wigolo's Local Cache Handles Keyword and Semantic Search](/KnockOutEZ/wigolo/wigolo-local-cache-function-keyword-semantic-search)

Explore how Wigolo's cache handles keyword and semantic search. Discover its SQLite FTS5 integration, hybrid RRF pipeline, and automatic fallback for uninterrupted performance.

- Tags: internals
- Published: 2026-07-19

### [How to Configure Wigolo to Use Different LLM Providers (Gemini, Anthropic, OpenAI, Ollama)](/KnockOutEZ/wigolo/configure-wigolo-llm-providers-gemini-anthropic-openai-ollama-synthesis)

Configure Wigolo to use Gemini, Anthropic, OpenAI, or Ollama for synthesis. Just set the WIGOLO LLM PROVIDER environment variable and your API key. Get started easily.

- Tags: how-to-guide
- Published: 2026-07-19

### [Wigolo's Tiered Fetch Router Architecture: How It Handles Anti-Bot Challenges](/KnockOutEZ/wigolo/wigolo-tiered-fetch-router-architecture-handle-anti-bot-challenges)

Discover Wigolo's tiered fetch router architecture. Learn how it bypasses anti-bot challenges using TLS impersonation, HTTP, and Playwright browser tiers automatically.

- Tags: architecture
- Published: 2026-07-19

### [How Wigolo Implements Multi-Engine Search with Rank Fusion and ML Reranking](/KnockOutEZ/wigolo/how-wigolo-multi-engine-search-rank-fusion-ml-reranking-work)

Discover how Wigolo implements multi-engine search using rank fusion and ML reranking for superior relevance from diverse data sources.

- Tags: deep-dive
- Published: 2026-07-19