How Multiple Data Sources Are Integrated and Prioritized in loyalsoldier/geoip: MaxMind, IPIP.net, and china-operator-ip

The loyalsoldier/geoip toolchain processes IP data sources sequentially according to their order in config.json, where later sources override earlier entries for the same list name through union and removal operations in a centralized container.

The loyalsoldier/geoip repository aggregates IP geolocation data from disparate providers like MaxMind GeoLite2, IPIP.net, and community-curated lists such as china-operator-ip. Understanding how these potentially overlapping datasets are merged and prioritized requires examining the internal pipeline architecture that treats every source as an ordered input plugin.

Input Plugin Architecture and Configuration

Every data source is defined as an input plugin within config.json. When the program initializes, it parses this configuration into a slice of inputConvConfig objects, each specifying a converter type (e.g., maxmindGeoLite2CountryCSV, text, cutter) and its associated arguments.

The core data structure resides in lib/container.go. The application creates a single lib.Container instance via lib.NewContainer(), which serves as the central registry for all IP sets. This container maintains a map of list names (such as cn, cloudflare, or tor) to their corresponding lib.Entry objects.

Sequential Processing Determines Priority

Priority is determined strictly by declaration order. The main loop iterates over the input array in config.json sequentially:

// Configuration loading and container initialization
cfg, _ := lib.LoadConfig("config.json")
c := lib.NewContainer()

// Sequential application of inputs
for _, ic := range cfg.Input {
    conv, _ := ic.converter // Created by createInputConfig()
    for _, e := range conv.Entries() {
        _ = c.Add(e) // Merges IP sets
    }
}

Because inputs are processed in the order they appear, later specifications always win for entries sharing the same list name. When container.Add(entry) encounters an existing entry, it performs a union operation on the IPv4 and IPv6 prefix sets rather than replacing the object entirely.

Container Operations: Merging and Cutting

The lib.Container in lib/container.go provides two primary mutation methods:

• Add(entry): Merges the IP set of the new entry with any existing entry of the same name. This creates a union of IPv4 and IPv6 prefixes.
• Remove(entry, CaseRemoveEntry): Deletes IP prefixes from an existing entry. This is invoked internally by the cutter input type.

The cutter plugin (specified with type: "cutter" and action: "remove") enables subtraction operations. When processing a cutter, the converter calls container.Remove(), which eliminates the specified IP version or entire entry from the container. You can also use the ignoreIP parameter to restrict removal to specific IP versions.

Data Source Integration Flow

A typical integration workflow demonstrating priority override operates as follows:

1. MaxMind GeoLite2 (maxmindGeoLite2CountryCSV) adds a comprehensive cn entry containing all China IP ranges.
2. A cutter step removes that specific entry: {"type": "cutter", "action": "remove", "args": {"wantedList": ["cn"]}}.
3. china-operator-ip lists (loaded via type: "text" from custom URLs) add a new, refined cn entry containing only operator-specific ranges.

This sequential pattern allows community-curated data to override generic geolocation databases without modifying source code—simply by reordering the JSON array.

Performing Lookups on Merged Data

Once all inputs are processed, the final container represents the definitive IP set database. The lookup plugin (plugin/special/lookup.go) queries this container via the special.Lookup struct:

lookup := &special.Lookup{
    Search:     "1.2.3.4",           // IP or CIDR to query
    SearchList: []string{"cn", "tor"} // Optional whitelist filter
}
_ = lookup.Output(c) // Returns matching list names or "false"

The lookup performs no additional weighting or ranking; it simply reports which list names contain the queried IP or CIDR. You can narrow results using the SearchList field to restrict queries to specific entries.

Summary

• Order-based priority: Data sources in config.json are processed sequentially; later entries override earlier ones for identical list names.
• Union semantics: The container.Add() method merges IP prefixes rather than replacing entries entirely.
• Selective removal: The cutter input type invokes container.Remove() to subtract specific IP sets or versions.
• Centralized storage: lib.Container in lib/container.go manages all lib.Entry objects and provides the lookup interface.
• Configuration-driven: No code changes are required to reprioritize sources; simply reorder the input array in config.json.

Frequently Asked Questions

What happens when two sources define the same country code?

The second source processed will union its IP prefixes with the existing entry. If a cutter step appears between them, it may first remove the initial entry entirely before the second source adds its data. The final IP set always reflects the cumulative result of all operations in the declared order.

How do I completely exclude MaxMind data from the output?

Add a cutter entry immediately after the MaxMind input specification with "action": "remove" and "wantedList": ["cn"] (or whichever lists you want to eliminate). Then add your preferred alternative sources after the cutter. Alternatively, simply remove the MaxMind input object from config.json entirely.

Can I prioritize IPv4 over IPv6 from different sources?

Yes. Use the ignoreIP parameter within a cutter configuration to remove only specific IP versions from an entry. For example, you can retain MaxMind's IPv6 data while replacing only the IPv4 ranges with data from china-operator-ip.

Where is the lookup logic implemented?

The lookup functionality is implemented in plugin/special/lookup.go. It utilizes the Lookup method of lib.Container (defined in lib/container.go) to check membership across all stored entries. The method returns all matching list names, with optional filtering via the SearchList configuration field.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}