# How to Add a New Website to Maigret's Database: A Complete Guide

> Learn how to add a new website to Maigret's database. Follow our guide to create a JSON entry and update Maigret's data for new site integrations.

- Repository: [Soxoj/maigret](https://github.com/soxoj/maigret)
- Tags: how-to-guide
- Published: 2026-04-30

---

**To add a new website to Maigret, create a JSON entry in [`maigret/resources/data.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/data.json) defining the site's URL patterns and detection logic, then run `python utils/update_site_data.py` to regenerate the documentation and metadata.**

Maigret is an open-source OSINT tool that automates username checks across hundreds of social networks and websites. The tool stores all supported sites in a centralized JSON database that maps username patterns to HTTP validation logic. Understanding how to extend this database allows you to customize Maigret for specific investigative targets or contribute new sites to the upstream project.

## Understanding Maigret's Site Database Architecture

Maigret's site definitions live in [`maigret/resources/data.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/data.json), a structured JSON file bundled with the package. Each top-level key represents a unique site identifier (e.g., `"Twitter"`, `"GitHub"`), with nested key-value pairs describing how the engine queries and validates accounts.

The core engine reads this JSON at runtime through [`maigret/sites.py`](https://github.com/soxoj/maigret/blob/main/maigret/sites.py). This module defines a `Site` class that deserializes each entry into an object with typed attributes mapping directly to JSON keys like `url`, `urlMain`, `checkType`, and `regexCheck`. When you add a new site, you are essentially extending the schema that this class expects, ensuring the engine can instantiate new `Site` objects for your target platform.

After modifying [`data.json`](https://github.com/soxoj/maigret/blob/main/data.json), you must run [`utils/update_site_data.py`](https://github.com/soxoj/maigret/blob/main/utils/update_site_data.py). This helper script regenerates the human-readable [`sites.md`](https://github.com/soxoj/maigret/blob/main/sites.md) file and updates [`maigret/resources/db_meta.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/db_meta.json) with derived statistics. These generated files power the `--list-sites` CLI option and project documentation, making this step mandatory for any database change.

## Step-by-Step Guide to Adding a New Site

### 1. Create the Site Entry in [`data.json`](https://github.com/soxoj/maigret/blob/main/data.json)

Copy an existing entry from [`maigret/resources/data.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/data.json) that uses similar detection logic, then modify it for your target site. At minimum, you must define how to construct the profile URL and how to verify if a username exists.

```json
"ExampleForum": {
    "tags": ["forum", "tech"],
    "urlMain": "https://example.com/",
    "url": "https://example.com/user/{username}",
    "checkType": "status_code",
    "usernameClaimed": "alice",
    "usernameUnclaimed": "nonexistent_user_99999",
    "regexCheck": "^[a-zA-Z0-9_]{3,20}$",
    "alexaRank": 5000
}

```

**Critical fields:**
- **`urlMain`**: The base domain for display purposes.
- **`url`**: The profile URL template using `{username}` as a placeholder.
- **`checkType`**: The detection method—`status_code`, `message`, or `response_url` are common options.
- **`usernameClaimed`** / **`usernameUnclaimed`**: Known existing and non-existing usernames for self-testing.

### 2. Regenerate the Documentation

Navigate to the repository root and execute the updater script to synchronize [`sites.md`](https://github.com/soxoj/maigret/blob/main/sites.md) and metadata:

```bash
python utils/update_site_data.py

```

This command parses your changes to [`data.json`](https://github.com/soxoj/maigret/blob/main/data.json) and rebuilds the markdown list of supported sites. If you skip this step, the CLI's `--list-sites` output will drift out of sync with the actual database.

### 3. Test Your New Site

Validate that your detection logic works correctly before submitting:

```bash

# Test a known existing account

maigret alice -t ExampleForum

# Test a known non-existing account  

maigret nonexistent_user_99999 -t ExampleForum

```

The first command should report the account as **Claimed**, while the second should report **Unclaimed**. If using `checkType: message`, verify that `presenseStrs` or `absenceStrs` correctly trigger based on the HTTP response body.

### 4. Submit Your Changes

Commit the following files to your fork:
- Modified [`maigret/resources/data.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/data.json)
- Regenerated [`sites.md`](https://github.com/soxoj/maigret/blob/main/sites.md)
- Updated [`maigret/resources/db_meta.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/db_meta.json) (auto-generated by the script)
- Optional: Add test cases in the test suite covering the new site's detection logic

Submit a pull request to the `soxoj/maigret` repository with a clear description of the site and your validation methodology.

## Essential Configuration Fields

When you add a new website to Maigret, these fields control the scanning behavior:

- **`checkType`**: Determines how Maigret interprets the HTTP response.
  - `status_code`: Username exists if the request returns a specific status (usually 200).
  - `message`: Checks for specific strings in the response body using `presenseStrs` or `absenceStrs`.
  - `response_url`: Validates based on URL redirects or final destination.

- **`regexCheck`**: A regular expression that validates username format before sending requests (e.g., `^[a-zA-Z0-9]+$` for alphanumeric-only sites).

- **`headers`**: Custom HTTP headers required by the target site, such as specific `User-Agent` strings or `Accept-Language` values.

- **`alexaRank`**: Integer representing the site's global traffic rank, used for prioritizing scans.

- **`tags`**: Array of category strings (e.g., `["social"]`, `["gaming"]`) enabling filtered scans via `maigret --tags gaming`.

## Summary

- Maigret stores site definitions in **[`maigret/resources/data.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/data.json)**, which the `Site` class in **[`maigret/sites.py`](https://github.com/soxoj/maigret/blob/main/maigret/sites.py)** parses at runtime.
- To add a new website, edit [`data.json`](https://github.com/soxoj/maigret/blob/main/data.json) with the appropriate `url`, `checkType`, and validation strings, then run **`python utils/update_site_data.py`** to regenerate documentation.
- Always test with known claimed and unclaimed usernames using the `-t` flag before submitting changes.
- The process requires no API keys and supports any site exposing public user profiles via predictable URL patterns.

## Frequently Asked Questions

### What file format does Maigret use for its site database?

Maigret uses a JSON file located at [`maigret/resources/data.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/data.json). Each site is a top-level object key containing nested configuration fields that the engine reads via the `Site` class in [`maigret/sites.py`](https://github.com/soxoj/maigret/blob/main/maigret/sites.py).

### Do I need to manually edit the sites.md file when adding a new site?

No. Running `python utils/update_site_data.py` automatically regenerates [`sites.md`](https://github.com/soxoj/maigret/blob/main/sites.md) and [`maigret/resources/db_meta.json`](https://github.com/soxoj/maigret/blob/main/maigret/resources/db_meta.json) based on the current state of [`data.json`](https://github.com/soxoj/maigret/blob/main/data.json). Manual edits to these files will be overwritten.

### What is the difference between `usernameClaimed` and `usernameUnclaimed`?

These fields provide sample usernames for automated testing. `usernameClaimed` should be a username known to exist on the platform, while `usernameUnclaimed` should be impossible or extremely unlikely to exist. Maigret uses these to self-verify that its detection logic remains accurate for the site.

### Can I add sites that require custom HTTP headers?

Yes. Include a `headers` object in your JSON entry with any required header key-value pairs, such as custom `User-Agent` strings or authentication tokens, and Maigret will include them in all requests to that site.