# How to Write Custom Plugins for Wigolo: Adding Search Engines and Site Extractors

> Learn to write custom Wigolo plugins by creating Node modules for search engines and site extractors. Follow simple steps to extend Wigolo's capabilities and discover new data sources.

- Repository: [Towhid Khan/wigolo](https://github.com/KnockOutEZ/wigolo)
- Tags: how-to-guide
- Published: 2026-07-19

---

**To write custom plugins for Wigolo, create a Node module that exports a `searchEngine` or `extractor` object matching the TypeScript contracts defined in [`src/types.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/types.ts), place it in `~/.wigolo/plugins`, and restart the daemon to auto-register the functionality.**

Wigolo is an extensible open-source search and extraction tool developed by KnockOutEZ. Its lightweight plugin architecture allows developers to integrate custom search engines or site-specific extractors by dropping standard Node modules into a designated directory, without modifying the core codebase.

## Understanding the Wigolo Plugin Architecture

Wigolo’s plugin system relies on three coordinated components that handle discovery, validation, and runtime registration.

### Plugin Discovery and Loading

When the daemon initializes, the `loadPlugins()` function in **[`src/plugins/loader.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/loader.ts)** scans the plugins directory (defaulting to `~/.wigolo/plugins` or the path specified by `WIGOLO_PLUGINS_DIR`). For each subdirectory containing a [`package.json`](https://github.com/KnockOutEZ/wigolo/blob/main/package.json), the loader dynamically imports the entry file specified in the `main` field and wraps the result in a `PluginLoadResult` object.

### Validation and Registration

The imported module is immediately passed to `validatePluginExports` from **[`src/plugins/validate.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/validate.ts)**. This validator ensures the module exports at least one valid contract: either a **`searchEngine`** object or an **`extractor`** object that conforms to the interfaces declared in **[`src/types.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/types.ts)**.

Upon successful validation, the loader forwards the plugin to the global **`PluginRegistry`** defined in **[`src/plugins/registry.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/registry.ts)**. The registry exposes two primary methods:
- `registerSearchEngine` adds the engine to the multi-engine dispatcher for rank-fusion search.
- `registerExtractor` inserts the extractor at the front of the extraction pipeline.

The registry maintains a `PluginRegistryState` object that tracks loaded plugins and guards against duplicate names, logging a warning and ignoring later definitions if collisions occur.

## Creating a Custom Search Engine Plugin

A search engine plugin must export an object with a unique `name` and an async `search` method that returns an array of results.

### Directory Structure

Create a new folder in the plugins directory with the following layout:

```

my-search-plugin/
├─ package.json
└─ index.mjs

```

### package.json Configuration

```json
{
  "name": "my-wigolo-search-plugin",
  "version": "0.1.0",
  "main": "index.mjs"
}

```

### Implementing the Search Engine

Create `index.mjs` with the following structure:

```javascript
export const searchEngine = {
  // The name must be unique across all loaded engines
  name: 'my-search-engine',

  /**
   * Execute a search query.
   * @param {string} query - The user query
   * @param {object} [options] - Optional SearchEngineOptions (e.g., locale)
   * @returns {Promise<Array<{title:string,url:string,snippet:string,relevance_score:number,engine:string}>>}
   */
  async search(query, options) {
    // Replace with actual HTTP call to your search service
    const results = [
      {
        title: `Result for "${query}" from My Engine`,
        url: 'https://example.com/result',
        snippet: 'A short description of the result.',
        relevance_score: 1.0,
        engine: 'my-search-engine',
      },
    ];
    return results;
  },
};

```

Once placed in `~/.wigolo/plugins` and restarted, Wigolo automatically incorporates this engine into its search workflow.

## Creating a Custom Site Extractor Plugin

Extractors handle site-specific HTML parsing before the generic extraction pipeline runs.

### Directory Structure

```

my-extractor-plugin/
├─ package.json
└─ index.mjs

```

### Implementing the Extractor

The extractor must export an object with `name`, `canHandle`, and `extract` methods:

```javascript
export const extractor = {
  // Unique identifier for this extractor
  name: 'my-site-extractor',

  /**
   * Determine if this extractor should handle the given URL.
   * @param {string} url - The target URL
   * @param {string} html - The raw HTML content
   * @returns {boolean}
   */
  canHandle(url, html) {
    // Example: handle only URLs under https://docs.mycompany.com
    return url.startsWith('https://docs.mycompany.com');
  },

  /**
   * Transform raw HTML into structured data.
   * @param {string} html - The raw HTML content
   * @param {string} url - The source URL
   * @returns {object|null} ExtractionResult or null to fall back to built-in extractor
   */
  extract(html, url) {
    // Simple example: extract the first <h1> as title
    const titleMatch = html.match(/<h1[^>]*>([^<]+)<\/h1>/i);
    const title = titleMatch ? titleMatch[1].trim() : 'Untitled';
    
    return {
      title,
      content: html,  // Process further (strip tags, convert to markdown, etc.)
      url,
    };
  },
};

```

When Wigolo fetches a page where `canHandle` returns `true`, this extractor executes first, allowing you to fix site-specific quirks or extract custom metadata fields.

## Installing and Managing Plugins

You can install plugins manually by copying the directory to `~/.wigolo/plugins`, or use the CLI command `wigolo plugin add <git-url>` to clone directly from a repository.

**Security Warning:** Because plugins run inside the same Node process as the host Wigolo instance, they inherit full network and filesystem permissions. Only install plugins from trusted sources. The `wigolo plugin add` command prompts for confirmation before completing the installation.

After adding or modifying plugins, restart the Wigolo daemon to trigger the discovery and registration cycle defined in **[`src/plugins/loader.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/loader.ts)**.

## Summary

- **Plugin Location:** Place Node modules in `~/.wigolo/plugins` or the directory specified by `WIGOLO_PLUGINS_DIR`.
- **Validation:** Wigolo validates exports against TypeScript contracts in [`src/types.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/types.ts) via `validatePluginExports` in [`src/plugins/validate.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/validate.ts).
- **Search Engines:** Export a `searchEngine` object with a unique `name` and async `search` method to participate in rank-fusion search.
- **Extractors:** Export an `extractor` object with `canHandle` and `extract` methods to intercept specific URLs before generic processing.
- **Registration:** The `PluginRegistry` in [`src/plugins/registry.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/registry.ts) manages runtime state and prevents duplicate name collisions.
- **Security:** Plugins execute with the same privileges as the host process; verify trustworthiness before installation.

## Frequently Asked Questions

### What file should I export my plugin code from?

Wigolo reads the `main` field in your [`package.json`](https://github.com/KnockOutEZ/wigolo/blob/main/package.json) to determine the entry point. You can use `.mjs`, `.js`, or `.ts` files (if transpiled), but ensure the path correctly points to the file exporting `searchEngine` or `extractor`.

### Can a single plugin provide both a search engine and an extractor?

Yes. A single module can export both `searchEngine` and `extractor` objects simultaneously. The validator in [`src/plugins/validate.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/validate.ts) accepts modules that export either or both contracts, registering each with the appropriate registry method.

### How do I debug why my plugin is not loading?

Check the daemon logs for warnings from `loadPlugins()` in [`src/plugins/loader.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/plugins/loader.ts). Common failures include missing [`package.json`](https://github.com/KnockOutEZ/wigolo/blob/main/package.json), invalid JSON syntax, or validation errors from `validatePluginExports` indicating that your exported object does not match the required interface in [`src/types.ts`](https://github.com/KnockOutEZ/wigolo/blob/main/src/types.ts).

### Do I need to restart Wigolo after adding a plugin?

Yes. The plugin discovery process runs once during daemon initialization. After placing a new plugin in the directory or running `wigolo plugin add`, you must restart the Wigolo daemon to trigger `loadPlugins()` and register the new functionality in the `PluginRegistry`.