How Maigret's Recursive Search Feature Works: Auto-Discovering Usernames Across Platforms

Maigret's recursive search automatically extracts additional usernames and identifiers from search results and linked pages, adding them to the scan queue for continuous discovery until no new identifiers remain.

The open-source OSINT tool Maigret (soxoj/maigret) includes a powerful recursive search capability that transforms single-username investigations into comprehensive digital footprint mapping. This feature continuously expands search scope by parsing discovered profiles for关联 identifiers, then automatically scanning those new usernames across all configured sites.

Configuration and Command-Line Control

Maigret's recursive behavior is controlled through a boolean flag that can be modified via settings files or command-line arguments.

Settings File Configuration (maigret/settings.py)

The default behavior is defined in the settings dataclass at line 19 of maigret/settings.py:

recursive_search: bool

This attribute is loaded from the user's settings configuration file. When true, the recursive discovery engine activates automatically during scans.

Command-Line Interface (--no-recursion)

The CLI parser in maigret/maigret.py (lines 71-77) exposes a flag to disable this feature at runtime:

parser.add_argument(
    "--no-recursion",
    action="store_true",
    dest="disable_recursive_search",
    default=(not settings.recursive_search),
    help="Disable recursive search by additional data extracted from pages.",
)

At line 564, the effective boolean is computed by inverting the CLI argument:

recursive_search_enabled = not args.disable_recursive_search

The Main Execution Loop

During the asynchronous main loop in maigret/maigret.py (lines 22-27), Maigret processes one username at a time. After completing standard site checks, a conditional block triggers the recursive discovery:

if recursive_search_enabled:
    extracted_ids = extract_ids_from_results(results, db)
    query_notify.warning(f'Extracted IDs: {extracted_ids}')
    usernames.update(extracted_ids)

This implementation creates a depth-first exploration pattern: each newly discovered identifier is added to the usernames dictionary and processed before moving to the next original query.

Extracting Identifiers from Results

The extract_ids_from_results function at lines 90-106 of maigret/maigret.py serves as the primary extraction orchestrator:

def extract_ids_from_results(results, db):
    ids_results = {}
    for website_name in results:
        dictionary = results[website_name]
        if not dictionary:
            continue
        new_usernames = dictionary.get('ids_usernames')
        if new_usernames:
            for u, utype in new_usernames.items():
                ids_results[u] = utype
        for url in dictionary.get('ids_links', []):
            ids_results.update(db.extract_ids_from_url(url))
    return ids_results

This function performs two distinct extraction operations:

• Direct usernames: Harvests ids_usernames from site results (already parsed identifiers)
• Linked pages: Follows ids_links URLs to fetch and parse additional pages

Parsing Pages for Additional IDs

For external URLs, Maigret delegates to the MaigretDatabase.extract_ids_from_url method in maigret/sites.py (lines 577-586):

def extract_ids_from_url(self, url: str) -> dict:
    # … fetch the page, run socid_extractor, normalize results …

    return results

This method fetches the target page and passes the content to the socid_extractor library for parsing. The low-level page parsing logic resides in maigret/maigret.py (lines 50-86) within the extract_ids_from_page function:

def extract_ids_from_page(url, logger, timeout=5) -> dict:
    # fetch + parse → extract → normalise (username / usernames / supported IDs)

This helper normalizes the output into {username: type} pairs to ensure consistency across different platforms.

Depth-First Processing of New Usernames

The usernames variable is a mutable dictionary where keys represent identifiers and values specify the type (e.g., "username"). By calling usernames.update(extracted_ids), Maigret injects newly discovered identities directly into the active scan queue.

This architectural choice means recursion happens immediately: if scanning alice reveals alice_dev and alice_photos, those accounts are fully processed before any other original usernames. The loop continues until the queue empties or recursion is disabled.

Disabling Recursive Search

There are two methods to disable this feature:

• Command line: Append --no-recursion to skip the extraction block entirely
• Configuration: Set "recursive_search": false in your settings file

When disabled, recursive_search_enabled evaluates to False, bypassing the extraction logic and limiting results to the initially provided usernames.

Summary

• Maigret recursive search automatically expands investigations by extracting identifiers from result pages and linked URLs.
• The feature is controlled by the recursive_search boolean in maigret/settings.py and can be disabled via --no-recursion.
• The extract_ids_from_results function (lines 90-106) harvests both embedded usernames and external links.
• External URL parsing relies on extract_ids_from_url in maigret/sites.py and the socid_extractor library.
• New usernames are merged into the active queue via usernames.update(), creating depth-first exploration.
• Disabling the flag forces single-pass scanning without discovery expansion.

Frequently Asked Questions

What triggers maigret's recursive search during execution?

The recursive search triggers after each username completes its standard site checks, governed by the recursive_search_enabled flag at line 564 of maigret/maigret.py. When enabled, Maigret calls extract_ids_from_results to harvest ids_usernames and ids_links from the current results, then adds these discoveries to the active scan queue using usernames.update().

How do I disable recursive search in maigret?

You can disable this feature by adding the --no-recursion command-line flag, which sets disable_recursive_search to True and forces recursive_search_enabled to False. Alternatively, set recursive_search: false in your Maigret settings configuration file to change the default behavior permanently.

Which component handles URL-based username extraction in maigret?

The MaigretDatabase.extract_ids_from_url method in maigret/sites.py (lines 577-586) handles URL-based extraction. This method fetches the specified page and processes it through the socid_extractor library to identify additional usernames and identifiers, returning them as normalized key-value pairs.

Does maigret recursive search include all found usernames automatically?

Yes, when recursive search is enabled, all extracted identifiers from ids_usernames and ids_links are automatically added to the scan queue via the update() method on the usernames dictionary. This creates a continuous loop that processes newly discovered accounts depth-first until no new identifiers remain or the feature is disabled.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}