How Ghidra's Function ID (FID) Database Identifies Functions: A Technical Deep Dive

Ghidra's Function ID (FID) database identifies functions by generating cryptographic hashes of instruction streams using FNV-1a digests, storing them in packed *.fidb files, and matching unknown binaries against these hashes with a configurable scoring algorithm.

The NSA's Ghidra reverse engineering framework includes a powerful system for recognizing known library functions across compiled binaries. According to the source code in NationalSecurityAgency/ghidra, the FID system works by normalizing function bodies into deterministic hash values that remain stable across different compiler versions and optimization levels.

How FID Hashing Works

At the core of the Ghidra Function ID database is the hashing pipeline implemented in MessageDigestFidHasher.java. This class transforms a function's machine code into a FidHashQuad—a four-part structure containing the short-code-unit count, full hash, specific-hash additional size, and specific hash.

The hasher operates on a function extent, which is a sequential list of code units (instructions) within the function boundaries. As the hasher walks this extent in MessageDigestFidHasher.hash(), it feeds instruction bytes into two independent 64-bit FNV-1a digests:

1. Full hash: Incorporates all instructions with operands masked out
2. Specific hash: Preserves small scalar operands (≤255) and small registers for finer discrimination

Full, Short, and Specific Hashes

The FID system generates three distinct hash types to balance speed and precision:

• Short hash: Computed from the first N code units (default 4), used for rapid pre-filtering in the database index
• Full hash: Covers up to 24 code units with operands normalized, providing a compiler-agnostic function signature
• Specific hash: Extends the full hash with operand details, enabling distinction between similar functions like different printf variants

Handling Relocations and Operands

To ensure cross-binary compatibility, MessageDigestFidHasher applies deterministic normalization rules:

• Call instructions: Target addresses are replaced with a placeholder value (0xfeeddead) to prevent address-dependent hash changes
• Registers: Mixed in an order-independent way using FNV-1a
• Scalar values: Small constants (≤255) are included in the specific hash but excluded from the full hash

This normalization allows the Ghidra Function ID database to match functions even when they are loaded at different base addresses or compiled with minor toolchain variations.

Building the FID Database

Creating a Function ID database involves harvesting hashes from known-good library binaries and packing them into a *.fidb file. The entry point for this process is FidService.createNewLibraryFromPrograms(), which iterates through a list of Ghidra program files and extracts function records.

Database Population Process

When populating the database, the system:

1. Opens each source program and enumerates its functions
2. Computes the FidHashQuad for each function using FidHasher.hashFunction()
3. Stores the hash quad along with metadata (function name, address range, library ID) in the packed database
4. Deduplicates strings via StringsTable to reduce file size

Database Schema

The packed FID file (*.fidb) contains several tables managed through FidDB.java:

• LibrariesTable: Stores library metadata including family name, version, and variant strings
• FunctionsTable: Contains one row per function with the hash quad, name ID, library ID, code-unit count, and flags (auto-pass, auto-fail, force-specific, force-relation)
• StringsTable: De-duplicates all strings to conserve space, referenced by ID from other tables
• RelationsTable: Optional parent/child relationships between functions, used when the "force-relation" flag is set

The underlying storage uses PackedDBHandle, which provides memory-mapped access to the hash tables for fast lookups.

Searching and Matching Functions

When analyzing an unknown binary, Ghidra uses FidProgramSeeker to identify known functions against the FID database. This process reverses the hashing pipeline to find candidate matches and validate them with a scoring algorithm.

The Matching Algorithm

FidProgramSeeker implements the search logic in processProgram():

1. Hash Computation: For each function in the target program, compute its FidHashQuad using the same MessageDigestFidHasher used during database creation
2. Short-Hash Lookup: Query the database using the short hash (first 4 code units) to retrieve candidate function records from FunctionsTable
3. Full-Hash Validation: Compare the full hash (operands masked) to filter out false positives from the short-hash collision set
4. Specific-Hash Refinement: When multiple candidates remain, use the specific hash (with operands) to disambiguate similar functions
5. Score Calculation: Compute a match score based on the ratio of matching code units to the short-hash length, multiplied by 100

Score Thresholds and Configuration

The FID system uses configurable thresholds to control match quality:

• FidService.SCORE_THRESHOLD (default 14.6%): The minimum score required for a regular match to be considered valid
• FidService.MULTINAME_SCORE_THRESHOLD (default 30%): A higher threshold used when multiple different function names match the same hash, indicating potential ambiguity

These thresholds can be tuned when creating a FidProgramSeeker instance via FidService.getProgramSeeker(). The scoring mechanism ensures that only high-confidence matches are applied automatically, while borderline cases can be flagged for manual review.

Key Implementation Files

The Function ID system is implemented across several packages in Ghidra/Features/FunctionID/src/main/java/ghidra/feature/fid/:

File	Role
hash/FidHasher.java	Interface defining the hash operation contract
hash/MessageDigestFidHasher.java	Concrete FNV-1a implementation generating full, short, and specific hashes
hash/FidHashQuad.java	Container for the four hash values and metadata
service/FidService.java	High-level API for hashing, library creation, and searching
service/FidProgramSeeker.java	Implements per-program search logic and score computation
service/FidQueryService.java	Manages opening/closing of multiple FID databases for a language
service/FidSearchResult.java	Result object containing match details and scores
db/FidDB.java	Low-level wrapper around packed *.fidb files
db/FunctionsTable.java	Stores function hash quads and metadata
db/LibrariesTable.java	Manages library family/version information
util/InstructionSkipper.java	Processor-specific logic to skip NOPs and padding during hashing
service/FIDFixedSizeMRUCachingFactory.java	Caches computed hashes to accelerate repeated queries

Summary

• Ghidra's Function ID database identifies known library functions by comparing cryptographic hashes of instruction streams rather than byte patterns.
• Hash generation uses MessageDigestFidHasher to create three hash levels (short, full, specific) using FNV-1a digests with operand masking and relocation normalization.
• Database files (*.fidb) are packed binary stores containing FunctionsTable, LibrariesTable, and StringsTable, managed through FidDB.java.
• Matching algorithm employs FidProgramSeeker to compute hashes for unknown functions, lookup candidates via short-hash, validate with full/specific hashes, and score results against configurable thresholds (default 14.6%).
• Integration occurs through FidService, which provides high-level methods for library creation (createNewLibraryFromPrograms) and program analysis (processProgram).

Frequently Asked Questions

What hash algorithm does Ghidra's FID system use?

The Ghidra Function ID system uses the FNV-1a (Fowler-Noll-Vo) hash algorithm, implemented in MessageDigestFidHasher.java. It maintains two independent 64-bit FNV-1a digests during the hashing process—one for the full hash (operands masked) and one for the specific hash (small operands preserved). This provides deterministic, architecture-independent hashing of instruction streams.

How does Ghidra handle false positives in function matching?

Ghidra employs a multi-tier validation system to minimize false positives. First, the FidProgramSeeker uses a short hash (first 4 code units) for rapid candidate lookup. Candidates are then filtered by the full hash (operands masked) to eliminate collisions. Finally, the specific hash resolves ambiguities between similar functions. A configurable score threshold (default 14.6% in FidService.SCORE_THRESHOLD) ensures only high-confidence matches are returned.

Can I create my own FID databases for custom libraries?

Yes, Ghidra provides the FidService.createNewLibraryFromPrograms() API to build custom FID databases. You supply a list of Ghidra program files containing known library functions, specify metadata (family name, version, variant), and the service computes hash quads for all functions using MessageDigestFidHasher. The resulting data is packed into a *.fidb file containing FunctionsTable and LibrariesTable entries that can be distributed and loaded via FidQueryService.

What is the difference between full hash and specific hash in FID?

The full hash and specific hash serve different precision levels in the Ghidra Function ID system. The full hash (FidHashQuad.getFullHash()) masks out all operands, creating a compiler-agnostic signature that matches functions regardless of immediate values or addresses. The specific hash (FidHashQuad.getSpecificHash()) preserves small scalar operands (≤255) and small registers, allowing discrimination between functions like different printf variants that have identical instruction sequences but different format string references. Both are computed simultaneously by MessageDigestFidHasher using separate FNV-1a digests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}