# How to Integrate External Search Providers (Brave, Baidu, and SearXNG) in OpenMAIC

> Learn how to integrate external search providers like Brave Baidu and SearXNG into OpenMAIC using the unified searchWeb API for seamless web searching across multiple platforms.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-13

---

**OpenMAIC unifies web search behind a single `searchWeb` API that routes requests to provider-specific implementations in `lib/web-search/`, supporting Brave (API or scrape), Baidu (multi-source), and SearXNG (self-hosted) through a consistent TypeScript interface.**

OpenMAIC provides a modular web-search abstraction layer that makes it straightforward to integrate external search providers without modifying core application logic. The architecture centers on a unified dispatcher in [`lib/web-search/index.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/index.ts) that delegates to provider-specific modules, each conforming to a standard contract for querying and result formatting.

## Understanding the Unified Search Architecture

The entry point for all search operations is the `searchWeb` function exported from [`lib/web-search/index.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/index.ts). This dispatcher accepts a configuration object that includes a `providerId` and switches execution to the appropriate implementation based on this identifier (lines 38-74).

Each provider module exports a `searchWith<Provider>` function adhering to this contract:

```typescript
export async function searchWith<Provider>(params: {
  query: string;
  apiKey?: string;
  maxResults?: number;
  baseUrl?: string;
  signal?: AbortSignal;
}): Promise<WebSearchResult>

```

When `searchWeb` is invoked, it routes to the corresponding provider implementation and returns a standardized `WebSearchResult` containing an array of `WebSearchSource` objects with `title`, `url`, `content`, and `score` properties.

## Integrating Brave Search

The Brave Search integration resides in [`lib/web-search/brave.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/brave.ts) and offers two operational modes: official API access and fallback HTML scraping.

### API Mode vs. Scrape Mode

When an `apiKey` is provided, the system uses `searchWithBraveApi` to send authenticated GET requests to `https://api.search.brave.com/res/v1/web/search`. The API key must be passed in the `X-Subscription-Token` header. Without an API key, `searchWithBrave` automatically falls back to `searchWithBraveScrape`, which fetches the public Brave search page and parses the HTML response.

### Key Implementation Functions

- **`buildBraveSearchUrl`** (lines 25-31): Constructs the search endpoint URL, optionally incorporating a custom `baseUrl`.
- **`parseBraveSearchHtml`** (lines 60-86): Extracts `title`, `url`, and `snippet` from HTML responses while filtering out Brave-owned URLs.
- **`searchWithBrave`** (lines 83-98): The exported entry point that selects between API and scrape modes based on credential availability.

## Integrating Baidu Search

Located in [`lib/web-search/baidu.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/baidu.ts), the Baidu provider aggregates three distinct sources: Qianfan web search, Baidu Baike, and Baidu Scholar. The `searchWithBaidu` function merges results from selected sub-sources into a single `WebSearchResult`.

### Sub-Source Architecture

The integration uses `baiduSubSources` to control which backends to query:

- **`fetchWebSearch`** (lines 58-95): Performs POST requests to the Qianfan endpoint and maps the `references` array to `WebSearchSource` objects.
- **`fetchBaike`** (lines 105-135): Queries the Baike endpoint, extracting abstracts for encyclopedic entries.
- **`fetchScholar`** (lines 143-180): Retrieves academic paper metadata from Baidu Scholar.

The `baiduHeaders` helper (lines 23-29) injects the bearer token and required custom headers for authentication.

## Integrating SearXNG

For organizations running self-hosted search aggregators, [`lib/web-search/searxng.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/searxng.ts) provides SearXNG integration via its JSON API.

### Implementation Details

The `searchWithSearxng` function requires a `baseUrl` pointing to your SearXNG instance. It constructs requests to the `/search` endpoint with `format=json` appended to the query string.

- **`buildSearxngSearchUrl`** (lines 20-26): Normalizes the base URL and appends query parameters.
- **`mapSearxngResult`** (lines 29-48): Transforms raw SearXNG entries into `WebSearchSource` objects, handling missing titles and optional score overrides.

The function validates the `baseUrl`, executes the fetch, logs unmappable results for debugging, and returns the standardized result set.

## Implementation Examples

To integrate these providers in your application, import the unified dispatcher and specify the desired provider:

```typescript
import { searchWeb } from '@/lib/web-search';

// Brave Search with API key
const braveResult = await searchWeb({
  providerId: 'brave',
  query: 'open source AI assistants',
  apiKey: process.env.BRAVE_API_KEY,
  maxResults: 7,
});

// Baidu with multiple sub-sources
const baiduResult = await searchWeb({
  providerId: 'baidu',
  query: 'large language models',
  apiKey: process.env.BAIDU_API_KEY,
  baiduSubSources: { webSearch: true, baike: false, scholar: true },
});

// Self-hosted SearXNG
const searxResult = await searchWeb({
  providerId: 'searxng',
  query: 'vector databases',
  baseUrl: 'https://my.searxng.instance',
  maxResults: 5,
});

```

All calls return a `WebSearchResult` containing an array of `WebSearchSource` objects with `title`, `url`, `content`, and `score` properties, plus metadata including `responseTime`.

## Summary

- OpenMAIC abstracts web search through the `searchWeb` dispatcher in [`lib/web-search/index.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/index.ts), routing requests based on the `providerId` parameter.
- **Brave Search** supports both authenticated API access (using `X-Subscription-Token`) and credential-free HTML scraping via [`lib/web-search/brave.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/brave.ts).
- **Baidu Search** combines Qianfan, Baike, and Scholar sources in [`lib/web-search/baidu.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/baidu.ts), controlled through the `baiduSubSources` configuration.
- **SearXNG** integration in [`lib/web-search/searxng.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/web-search/searxng.ts) enables self-hosted search aggregation using the JSON API format.
- All providers conform to the same TypeScript contract, returning standardized `WebSearchResult` objects regardless of the backend.

## Frequently Asked Questions

### Do I need an API key to use Brave Search in OpenMAIC?

No. While OpenMAIC supports authenticated API access via the `X-Subscription-Token` header in `searchWithBraveApi`, the `searchWithBrave` function automatically falls back to HTML scraping mode when no `apiKey` is provided. The scrape mode parses public Brave results using `parseBraveSearchHtml`, though it may be less stable than the official API.

### Can I query multiple Baidu sources simultaneously?

Yes. The `searchWithBaidu` function accepts a `baiduSubSources` object that lets you enable or disable Qianfan web search, Baidu Baike, and Baidu Scholar in a single request. The results from each enabled source are merged into one unified array of `WebSearchSource` objects returned in the `WebSearchResult`.

### What is the minimum configuration required for SearXNG integration?

You must provide a `baseUrl` pointing to your SearXNG instance endpoint. Unlike Brave or Baidu, SearXNG does not require an API key, but the `baseUrl` parameter is mandatory for `searchWithSearxng` to construct the correct JSON API request to `/search?q={query}&format=json`.

### How does OpenMAIC handle differences between provider response formats?

Each provider module in `lib/web-search/` implements a specific mapping function—such as `mapSearxngResult` or `parseBraveSearchHtml`—that normalizes raw API responses into the standard `WebSearchSource` structure. This abstraction ensures that the `searchWeb` dispatcher returns consistent data regardless of which provider you use.