How GeoIP Handles IP Address Overlaps When Merging Data from Different Sources

When merging IP data, the geoip tool uses a canonical string key system where the first occurrence of any CIDR block wins and subsequent duplicates are silently discarded, while overlapping but non-identical ranges are preserved as separate entries.

The loyalsoldier/geoip repository provides a command-line utility for merging IP and CIDR data from multiple sources. Understanding the internal logic for handling IP address overlaps is critical for users who need deterministic control over which data sources take precedence when the same IP ranges appear across different input files.

The Container Architecture: Storing IP Entries

At the core of the merge process is the Container struct defined in [lib/container.go](https://github.com/loyalsoldier/geoip/blob/master/lib/container.go). This structure maintains two internal maps that store unique network entries:

• c.ipv4 – map[string]*Entry for IPv4 networks
• c.ipv6 – map[string]*Entry for IPv6 networks

The maps use the canonical string representation of the CIDR block as the key (e.g., "192.168.0.0/16" or "2001:db8::/32"). This design choice ensures that identical representations of the same network always resolve to the same map key, regardless of formatting variations in the source files.

The Merge Logic: How Overlaps Are Resolved

Exact Duplicate Handling

When the Container.AddEntry method receives a new entry, it performs a simple existence check against the appropriate IP version map:

// Simplified logic from lib/container.go
func (c *Container) AddEntry(e *Entry) {
    key := e.String() // canonical "IP/CIDR" representation
    
    if e.IPVersion == IPv4 {
        if _, exists := c.ipv4[key]; !exists {
            c.ipv4[key] = e
        }
    } else {
        if _, exists := c.ipv6[key]; !exists {
            c.ipv6[key] = e
        }
    }
}

First-win policy: If the canonical key already exists in the map, the new entry is silently ignored. The first occurrence encountered during the merge process permanently occupies that key slot.

Partial Overlap and Nested CIDRs

The container treats nested or partially overlapping CIDR blocks as distinct entries because they generate different canonical string keys. For example:

• 10.0.0.0/8 (key: "10.0.0.0/8")
• 10.0.0.0/16 (key: "10.0.0.0/16")

Both entries are stored separately in the map. The Container.Finalize method (also in lib/container.go) converts these maps into sorted slices without attempting to merge or coalesce overlapping ranges. The output converters simply emit the unique CIDR strings in the order they were first added.

Deterministic Precedence Control

Because the merge logic preserves the first occurrence and ignores subsequent duplicates, you control precedence strictly through input ordering. When invoking the merge command via [merge.go](https://github.com/loyalsoldier/geoip/blob/master/merge.go), list your highest-priority source first:

# Source A takes precedence over B and C for any overlapping ranges

cat sourceA.txt sourceB.txt sourceC.txt | geoip merge > merged.txt

If sourceA.txt contains 192.168.0.0/16 and sourceB.txt contains the same CIDR, the final output contains only the entry from sourceA.txt.

Code Implementation Details

The Entry struct defined in [lib/entry.go](https://github.com/loyalsoldier/geoip/blob/master/lib/entry.go) provides the String() method that generates the canonical key used for deduplication. This normalization ensures that whitespace variations or different formatting styles do not create false duplicates.

The complete merge pipeline implemented in the CLI follows this flow:

1. Input conversion: special.Stdin, special.File, or remote URL readers parse raw text into Entry objects
2. Deduplication: Container.AddEntry filters exact duplicates using the canonical key map
3. Finalization: Container.Finalize sorts the unique entries by IP and prefix length
4. Output conversion: Writers emit the sorted list in the desired format (text, MaxMind DB, etc.)

Summary

• Canonical key deduplication: The container uses exact string matches of normalized CIDR notation to identify duplicates.
• First-win precedence: When identical CIDR blocks appear across sources, the first occurrence is retained and subsequent matches are discarded.
• No range coalescing: Overlapping but non-identical CIDR blocks (e.g., /8 and /16) remain separate entries in the final output.
• Deterministic control: Precedence is governed strictly by input order, allowing users to prioritize specific data sources by sequencing them accordingly.

Frequently Asked Questions

How does geoip handle exact duplicate IP ranges across different source files?

When the same CIDR block appears in multiple source files, geoip keeps the first occurrence and ignores all subsequent duplicates. The Container.AddEntry method in lib/container.go checks the internal map for the canonical string key; if the key already exists, the new entry is silently discarded without merging or modification.

Does geoip automatically merge overlapping CIDR blocks into larger ranges?

No, geoip does not coalesce overlapping ranges. If your sources contain 10.0.0.0/8 and 10.0.0.0/16, both entries are preserved as separate keys in the container's map. The Container.Finalize method sorts these entries but does not attempt to merge them into a single larger block or split them into smaller pieces.

Can I control which source file takes precedence when merging IP data?

Yes, precedence is determined entirely by input order. Because geoip implements a first-win policy, you should list your highest-priority source first in the input stream. For example, piping cat priority.txt secondary.txt | geoip merge ensures that any duplicates found in both files retain the values from priority.txt.

What data structure does geoip use to store IP entries during the merge process?

GeoIP uses a container struct with two string-keyed maps defined in lib/container.go: c.ipv4 and c.ipv6. Each map stores *Entry values using the canonical CIDR string (e.g., "192.168.0.0/24") as the key. This map-based approach enables O(1) duplicate detection and ensures that only unique network entries proceed to the final output stage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}