How to Write Custom Plugins for Wigolo: Adding Search Engines and Site Extractors

To write custom plugins for Wigolo, create a Node module that exports a searchEngine or extractor object matching the TypeScript contracts defined in src/types.ts, place it in ~/.wigolo/plugins, and restart the daemon to auto-register the functionality.

Wigolo is an extensible open-source search and extraction tool developed by KnockOutEZ. Its lightweight plugin architecture allows developers to integrate custom search engines or site-specific extractors by dropping standard Node modules into a designated directory, without modifying the core codebase.

Understanding the Wigolo Plugin Architecture

Wigolo’s plugin system relies on three coordinated components that handle discovery, validation, and runtime registration.

Plugin Discovery and Loading

When the daemon initializes, the loadPlugins() function in src/plugins/loader.ts scans the plugins directory (defaulting to ~/.wigolo/plugins or the path specified by WIGOLO_PLUGINS_DIR). For each subdirectory containing a package.json, the loader dynamically imports the entry file specified in the main field and wraps the result in a PluginLoadResult object.

Validation and Registration

The imported module is immediately passed to validatePluginExports from src/plugins/validate.ts. This validator ensures the module exports at least one valid contract: either a searchEngine object or an extractor object that conforms to the interfaces declared in src/types.ts.

Upon successful validation, the loader forwards the plugin to the global PluginRegistry defined in src/plugins/registry.ts. The registry exposes two primary methods:

• registerSearchEngine adds the engine to the multi-engine dispatcher for rank-fusion search.
• registerExtractor inserts the extractor at the front of the extraction pipeline.

The registry maintains a PluginRegistryState object that tracks loaded plugins and guards against duplicate names, logging a warning and ignoring later definitions if collisions occur.

Creating a Custom Search Engine Plugin

A search engine plugin must export an object with a unique name and an async search method that returns an array of results.

Directory Structure

Create a new folder in the plugins directory with the following layout:

my-search-plugin/
├─ package.json
└─ index.mjs

package.json Configuration

{
  "name": "my-wigolo-search-plugin",
  "version": "0.1.0",
  "main": "index.mjs"
}

Implementing the Search Engine

Create index.mjs with the following structure:

export const searchEngine = {
  // The name must be unique across all loaded engines
  name: 'my-search-engine',

  /**
   * Execute a search query.
   * @param {string} query - The user query
   * @param {object} [options] - Optional SearchEngineOptions (e.g., locale)
   * @returns {Promise<Array<{title:string,url:string,snippet:string,relevance_score:number,engine:string}>>}
   */
  async search(query, options) {
    // Replace with actual HTTP call to your search service
    const results = [
      {
        title: `Result for "${query}" from My Engine`,
        url: 'https://example.com/result',
        snippet: 'A short description of the result.',
        relevance_score: 1.0,
        engine: 'my-search-engine',
      },
    ];
    return results;
  },
};

Once placed in ~/.wigolo/plugins and restarted, Wigolo automatically incorporates this engine into its search workflow.

Creating a Custom Site Extractor Plugin

Extractors handle site-specific HTML parsing before the generic extraction pipeline runs.

Directory Structure

my-extractor-plugin/
├─ package.json
└─ index.mjs

Implementing the Extractor

The extractor must export an object with name, canHandle, and extract methods:

export const extractor = {
  // Unique identifier for this extractor
  name: 'my-site-extractor',

  /**
   * Determine if this extractor should handle the given URL.
   * @param {string} url - The target URL
   * @param {string} html - The raw HTML content
   * @returns {boolean}
   */
  canHandle(url, html) {
    // Example: handle only URLs under https://docs.mycompany.com
    return url.startsWith('https://docs.mycompany.com');
  },

  /**
   * Transform raw HTML into structured data.
   * @param {string} html - The raw HTML content
   * @param {string} url - The source URL
   * @returns {object|null} ExtractionResult or null to fall back to built-in extractor
   */
  extract(html, url) {
    // Simple example: extract the first <h1> as title
    const titleMatch = html.match(/<h1[^>]*>([^<]+)<\/h1>/i);
    const title = titleMatch ? titleMatch[1].trim() : 'Untitled';
    
    return {
      title,
      content: html,  // Process further (strip tags, convert to markdown, etc.)
      url,
    };
  },
};

When Wigolo fetches a page where canHandle returns true, this extractor executes first, allowing you to fix site-specific quirks or extract custom metadata fields.

Installing and Managing Plugins

You can install plugins manually by copying the directory to ~/.wigolo/plugins, or use the CLI command wigolo plugin add <git-url> to clone directly from a repository.

Security Warning: Because plugins run inside the same Node process as the host Wigolo instance, they inherit full network and filesystem permissions. Only install plugins from trusted sources. The wigolo plugin add command prompts for confirmation before completing the installation.

After adding or modifying plugins, restart the Wigolo daemon to trigger the discovery and registration cycle defined in src/plugins/loader.ts.

Summary

• Plugin Location: Place Node modules in ~/.wigolo/plugins or the directory specified by WIGOLO_PLUGINS_DIR.
• Validation: Wigolo validates exports against TypeScript contracts in src/types.ts via validatePluginExports in src/plugins/validate.ts.
• Search Engines: Export a searchEngine object with a unique name and async search method to participate in rank-fusion search.
• Extractors: Export an extractor object with canHandle and extract methods to intercept specific URLs before generic processing.
• Registration: The PluginRegistry in src/plugins/registry.ts manages runtime state and prevents duplicate name collisions.
• Security: Plugins execute with the same privileges as the host process; verify trustworthiness before installation.

Frequently Asked Questions

What file should I export my plugin code from?

Wigolo reads the main field in your package.json to determine the entry point. You can use .mjs, .js, or .ts files (if transpiled), but ensure the path correctly points to the file exporting searchEngine or extractor.

Can a single plugin provide both a search engine and an extractor?

Yes. A single module can export both searchEngine and extractor objects simultaneously. The validator in src/plugins/validate.ts accepts modules that export either or both contracts, registering each with the appropriate registry method.

How do I debug why my plugin is not loading?

Check the daemon logs for warnings from loadPlugins() in src/plugins/loader.ts. Common failures include missing package.json, invalid JSON syntax, or validation errors from validatePluginExports indicating that your exported object does not match the required interface in src/types.ts.

Do I need to restart Wigolo after adding a plugin?

Yes. The plugin discovery process runs once during daemon initialization. After placing a new plugin in the directory or running wigolo plugin add, you must restart the Wigolo daemon to trigger loadPlugins() and register the new functionality in the PluginRegistry.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}