# How the loyalsoldier/geoip Repository Generates GeoIP Format Files Weekly: Internal Process and Dependencies

> Discover how the loyalsoldier github repository automatically generates GeoIP files weekly. Learn about its internal process, dependencies, and supported output formats like V2Ray DAT and MMDB.

- Repository: [Loyalsoldier/geoip](https://github.com/loyalsoldier/geoip)
- Tags: internals
- Published: 2026-03-06

---

**The loyalsoldier/geoip repository automates GeoIP file generation every Thursday at 00:00 UTC through a GitHub Actions workflow that downloads MaxMind GeoLite2 data, compiles a Go 1.25 binary, and executes a plugin-based conversion pipeline to produce multiple output formats including V2Ray DAT and MaxMind MMDB.**

The `loyalsoldier/geoip` project maintains an open-source pipeline for building customizable GeoIP databases used by proxy tools like V2Ray, Clash, and sing-box. Understanding the internal process for generating GeoIP format files weekly reveals how the repository transforms raw MaxMind CSV data into optimized distribution files without manual intervention.

## Weekly Automation Workflow

The entire generation process is orchestrated by [`.github/workflows/build.yml`](https://github.com/loyalsoldier/geoip/blob/main/.github/workflows/build.yml), which triggers automatically via `cron: "0 0 * * 4"`—midnight UTC every Thursday.

### Pipeline Stages

The workflow executes eleven distinct stages to ensure deterministic, reproducible builds:

- **Repository Checkout** — Uses `actions/checkout@v6` to pull the full git history required for release management.

- **Go Environment Setup** — Installs Go 1.25 (as declared in `go.mod`) using `actions/setup-go@v6`.

- **Timestamp Generation** — Creates a timestamp variable used for release tags and asset versioning.

- **MaxMind Data Acquisition** — Downloads the latest *GeoLite2-Country* and *GeoLite2-ASN* archives using the secret `MAXMIND_GEOLITE2_LICENSE` via authenticated `curl` requests.

- **File Preparation** — Unpacks tar.gz and zip archives, copying `.mmdb` files and CSV datasets into `./output/` and `./geolite2/` directories using standard Bash utilities.

- **Binary Compilation and Conversion** — Executes `go build ./` to compile the CLI, then runs `./geoip convert -c ./config.json` to process all configured inputs and outputs.

- **MMDB Integrity Verification** — Installs `github.com/maxmind/mmdbverify` and validates each generated `.mmdb` file against MaxMind specifications.

- **Checksum Generation** — Computes SHA-256 hashes for all `.dat` and `.mmdb` files, writing them to `*.sha256sum` files.

- **Release Branch Deployment** — Commits generated assets to a lightweight `release` branch used exclusively for GitHub Release attachments.

- **CDN Cache Purging** — Sends HTTP PURGE requests to jsDelivr for every new asset, ensuring global mirrors refresh immediately.

- **Artifact Publication** — Uses `svenstaro/upload-release-action@v2` to publish final assets as a timestamped GitHub Release, removing intermediate CSV files and raw archives beforehand to keep the distribution clean.

## Core Conversion Architecture

The conversion logic centers on a command-line interface defined in [`main.go`](https://github.com/loyalsoldier/geoip/blob/main/main.go) and [`convert.go`](https://github.com/loyalsoldier/geoip/blob/main/convert.go), implementing a modular plugin system that decouples data ingestion from format output.

### CLI Entry Points

In [`main.go`](https://github.com/loyalsoldier/geoip/blob/main/main.go), the application registers the `convert` sub-command. The [`convert.go`](https://github.com/loyalsoldier/geoip/blob/main/convert.go) file implements this command, parsing the `-c` or `--config` flag to load [`config.json`](https://github.com/loyalsoldier/geoip/blob/main/config.json) and orchestrating the transformation pipeline.

### Plugin-Based Data Flow

The architecture implements two primary interfaces:

- **Input Plugins** — Types like `maxmindGeoLite2CountryCSV` (in [`plugin/maxmind/maxmind_country_csv_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/maxmind/maxmind_country_csv_in.go)), `text` ([`plugin/plaintext/text_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/plaintext/text_in.go)), `json` ([`plugin/special/json_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/special/json_in.go)), and `private` ([`plugin/special/private.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/special/private.go)) implement the `InputConverter` interface. Each plugin calls `container.Add` or `container.Remove` methods to populate a central data structure defined in [`lib/container.go`](https://github.com/loyalsoldier/geoip/blob/main/lib/container.go) and [`lib/entry.go`](https://github.com/loyalsoldier/geoip/blob/main/lib/entry.go).

- **Output Plugins** — Types like `v2rayGeoIPDat` ([`plugin/v2ray/dat_out.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/v2ray/dat_out.go)), `maxmindMMDB` ([`plugin/maxmind/maxmind_country_mmdb_out.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/maxmind/maxmind_country_mmdb_out.go)), `singboxSRS`, and `clashRuleSet` implement the `OutputConverter` interface, serializing the container contents into target formats using format-specific writers.

The container maintains CIDR lists and supports add/remove operations, enabling complex transformations like merging MaxMind data with custom IP lists while excluding specific ranges.

## Critical Go Dependencies

All third-party modules are declared in `go.mod`. The generation pipeline relies on specialized libraries for compression, IP handling, and MaxMind compatibility:

- **`github.com/klauspost/compress`** — Handles ZIP/ZIP-64 operations for archive extraction and V2Ray DAT output compression.

- **`github.com/maxmind/mmdbwriter`** — Writes MaxMind MMDB files via the `maxmindMMDB` output plugin, including the `mmdbtype` subpackage for data types.

- **`github.com/oschwald/geoip2-golang/v2`** — Reads original MaxMind `.mmdb` files when the pipeline requires additional metadata from source databases.

- **`github.com/spf13/cobra`** — Provides the CLI framework powering the `geoip convert` command structure.

- **`github.com/tailscale/hujson`** — Parses JSON configuration files while preserving comments, used by [`lib/config.go`](https://github.com/loyalsoldier/geoip/blob/main/lib/config.go) for human-readable configuration.

- **`github.com/tidwall/gjson`** — Enables fast extraction of IP ranges from remote JSON sources like Google and Fastly endpoint lists.

- **`go4.org/netipx`** — Supplies advanced IP address utilities for CIDR merging and prefix manipulation.

- **`google.golang.org/protobuf`** — Serializes V2Ray `GeoIP` protobuf messages in [`dat_out.go`](https://github.com/loyalsoldier/geoip/blob/main/dat_out.go).

- **`gopkg.in/yaml.v2`** — Parses YAML fragments embedded in certain plugin configurations.

## Configuration-Driven Generation

The [`config.json`](https://github.com/loyalsoldier/geoip/blob/main/config.json) file acts as the declarative blueprint, separating data sources from output targets without requiring code changes.

### Input Configuration

The `input` array defines data sources using `type`, `action` (add/remove), and `args` fields:

- **MaxMind CSV** — `maxmindGeoLite2CountryCSV` type reads `GeoLite2-Country-Locations-en.csv`, `GeoLite2-Country-Blocks-IPv4.csv`, and `GeoLite2-Country-Blocks-IPv6.csv`.

- **Remote Text Lists** — `text` type fetches plain-text IP lists from URIs like `https://raw.githubusercontent.com/17mon/china_ip_list/master/china_ip_list.txt`.

- **JSON Endpoints** — `json` type extracts CIDR ranges from remote JSON APIs using `gjson` selectors.

- **Private Ranges** — `private` type injects standard private IP ranges (RFC 1918) into the container.

### Output Configuration

The `output` array specifies destination formats using the same `type`/`action`/`args` pattern:

- **V2Ray DAT** — `v2rayGeoIPDat` type generates `geoip.dat` protobuf files compatible with V2Ray and Xray cores.

- **MaxMind MMDB** — `maxmindMMDB` type produces `Country.mmdb` files, supporting `sourceMMDBURI` for template inheritance and `overwriteList` for selective updates.

- **Proxy Tool Formats** — Additional types generate Clash Rule Sets, Surge rule files, and sing-box SRS databases.

## Running the Pipeline Locally

Developers can replicate the weekly generation process on local machines for testing or custom builds.

### Building and Executing

```bash

# Compile the binary

go build -o geoip .

# Run conversion with default configuration

./geoip convert -c ./config.json

```

### Adding Custom Data Sources

To include a new IP list in the next generation cycle, add an entry to the `input` array in [`config.json`](https://github.com/loyalsoldier/geoip/blob/main/config.json):

```json
{
  "type": "text",
  "action": "add",
  "args": {
    "name": "custom",
    "uri": "https://example.com/custom-ip-list.txt",
    "onlyIPType": "ipv4"
  }
}

```

### Defining Additional Outputs

To generate a specialized V2Ray database containing only specific categories, extend the `output` array:

```json
{
  "type": "v2rayGeoIPDat",
  "action": "output",
  "args": {
    "outputDir": "./output",
    "outputName": "geoip-custom.dat",
    "wantedList": ["cn", "tor", "custom"]
  }
}

```

## Summary

- The loyalsoldier/geoip repository triggers automated builds every Thursday via `cron: "0 0 * * 4"` in [`.github/workflows/build.yml`](https://github.com/loyalsoldier/geoip/blob/main/.github/workflows/build.yml).
- A Go 1.25 binary processes data through a plugin architecture implementing `InputConverter` and `OutputConverter` interfaces.
- The pipeline downloads fresh MaxMind GeoLite2-Country and GeoLite2-ASN data using authenticated API requests.
- Core data structures in [`lib/container.go`](https://github.com/loyalsoldier/geoip/blob/main/lib/container.go) manage CIDR additions and removals across all input sources.
- Output formats include V2Ray DAT, MaxMind MMDB, Clash rules, and sing-box SRS via specialized dependencies like `mmdbwriter` and `protobuf`.
- The entire workflow is configurable through [`config.json`](https://github.com/loyalsoldier/geoip/blob/main/config.json), requiring no code changes to modify inputs or outputs.

## Frequently Asked Questions

### How does the repository schedule its weekly GeoIP generation?

The repository uses a GitHub Actions workflow defined in [`.github/workflows/build.yml`](https://github.com/loyalsoldier/geoip/blob/main/.github/workflows/build.yml) that triggers automatically via the cron expression `0 0 * * 4`, which executes at 00:00 UTC every Thursday without requiring manual intervention.

### What validates the integrity of generated MaxMind MMDB files?

After conversion, the workflow installs the `github.com/maxmind/mmdbverify` tool and validates each generated `.mmdb` file against MaxMind's official specification before including it in the GitHub Release, ensuring database compatibility with standard readers.

### Which Go version and dependencies power the conversion engine?

The pipeline requires Go 1.25 as specified in `go.mod`, with critical dependencies including `github.com/maxmind/mmdbwriter` for MMDB creation, `github.com/klauspost/compress` for archive handling, and `google.golang.org/protobuf` for V2Ray DAT serialization.

### How can I customize the weekly output formats without modifying Go code?

Edit the [`config.json`](https://github.com/loyalsoldier/geoip/blob/main/config.json) file at the repository root to add new entries to the `input` array (for data sources) or `output` array (for target formats), then commit the changes; the next scheduled run will automatically pick up the new configuration and generate the additional files.