# How to Implement Automated Testing for Custom GeoIP Data Transformations and Format Conversions

> Automate GeoIP data transformation and format conversion testing with Go. Learn to test custom converters and output structures for reliable GeoIP data processing in the loyalsoldier/geoip repo.

- Repository: [Loyalsoldier/geoip](https://github.com/loyalsoldier/geoip)
- Tags: how-to-guide
- Published: 2026-03-06

---

**To implement automated testing for custom GeoIP data transformations in the loyalsoldier/geoip repository, create Go test files that instantiate a `lib.Container`, register input and output converters via the library's registration functions, feed deterministic test fixtures through the conversion pipeline, and assert the output matches expected CIDR lists or format structures.**

The **loyalsoldier/geoip** repository is a Go-based command-line tool that ingests GeoIP data from multiple sources, processes it through a pipeline of transformations, and exports it to various output formats like V2Ray, MaxMind, and Clash rule sets. Because the project uses a plugin-based architecture where each input and output format is implemented as a registered converter, automated testing ensures that custom transformations correctly handle CIDR merging, filtering, and format-specific serialization without regressions.

## Understanding the GeoIP Plugin Architecture

Before writing tests, you must understand how the conversion pipeline operates. The core data structure is the **`lib.Container`**, defined in [`lib/container.go`](https://github.com/loyalsoldier/geoip/blob/main/lib/container.go), which holds all IP entries grouped by list name (e.g., `geoip:cn`). Input converters read raw data and populate this container, while output converters serialize the container to a specific format.

Each plugin registers itself during initialization using **`lib.RegisterInputConfigCreator`** or **`lib.RegisterOutputConfigCreator`**, typically inside an `init()` function. For example, [`plugin/v2ray/dat_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/v2ray/dat_in.go) registers the `v2rayGeoIPDatIn` input type, while [`plugin/v2ray/dat_out.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/v2ray/dat_out.go) registers the corresponding output converter. Your tests must invoke these same registration mechanisms to ensure you are testing the actual code paths used by the CLI.

## Setting Up Your Test Environment

Create test files adjacent to the plugin code you are validating, following Go conventions. Name the file `*_test.go` so the `go test` runner recognizes it automatically.

For example, to test V2Ray format conversions, add:

```go
package v2ray

import (
    "os"
    "testing"

    "github.com/Loyalsoldier/geoip/lib"
)

```

Store deterministic test fixtures in a **`testdata/`** subdirectory within your plugin folder. This directory is treated specially by Go tooling and is preserved when running tests. Place minimal `.dat`, `.mmdb`, or JSON files here that contain known CIDRs (such as `1.2.3.0/24` and `2.2.2.2/32`) to serve as controlled inputs.

## Building a Test Pipeline

A robust test exercises the full pipeline: input conversion, container processing, and output conversion. You need helper functions to instantiate the container and load registered converters.

### Initializing the Container

Always start with a fresh container to avoid state pollution between tests:

```go
func newTestContainer(t *testing.T) lib.Container {
    c, err := lib.NewContainer()
    if err != nil {
        t.Fatalf("failed to create container: %v", err)
    }
    return c
}

```

The `lib.NewContainer()` function returns an empty container with default configuration, ready to receive entries from any input converter.

### Loading Input Converters

Use the library's registration lookup to instantiate the exact converter the CLI would use:

```go
func loadInput(t *testing.T, typ string, action lib.Action, data []byte) lib.InputConverter {
    creator, ok := lib.GetInputConfigCreator(typ)
    if !ok {
        t.Fatalf("input type %q not registered", typ)
    }
    conv, err := creator(action, data)
    if err != nil {
        t.Fatalf("failed to create input converter: %v", err)
    }
    return conv
}

```

This pattern ensures you are testing the actual initialization logic found in files like [`plugin/maxmind/maxmind_country_csv_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/maxmind/maxmind_country_csv_in.go) or [`plugin/plaintext/json_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/plaintext/json_in.go), not just mock implementations.

### Loading Output Converters

Similarly, obtain output converters using **`lib.GetOutputConfigCreator`**:

```go
func loadOutput(t *testing.T, typ string, action lib.Action, data []byte) lib.OutputConverter {
    creator, ok := lib.GetOutputConfigCreator(typ)
    if !ok {
        t.Fatalf("output type %q not registered", typ)
    }
    conv, err := creator(action, data)
    if err != nil {
        t.Fatalf("failed to create output converter: %v", err)
    }
    return conv
}

```

## Writing an End-to-End Conversion Test

Combine these helpers to test a complete transformation. The following example validates that a custom V2Ray `.dat` file correctly converts to plaintext CIDR lists:

```go
func TestV2RayDatConversion(t *testing.T) {
    // 1. Prepare container
    container := newTestContainer(t)

    // 2. Load input referencing a fixture in testdata/
    input := loadInput(t, "v2rayGeoIPDatIn", lib.ActionAdd, []byte(`{"source":"testdata/custom.dat"}`))

    // 3. Feed the input to the container
    var err error
    container, err = input.Input(container)
    if err != nil {
        t.Fatalf("input conversion failed: %v", err)
    }

    // 4. Configure output to write to a temporary file
    tmpFile := t.TempDir() + "/out.txt"
    outConfig := []byte(`{"path":"` + tmpFile + `"}`)
    
    output := loadOutput(t, "textOut", lib.ActionAdd, outConfig)

    // 5. Run the output conversion
    if err = output.Output(container); err != nil {
        t.Fatalf("output conversion failed: %v", err)
    }

    // 6. Verify the resulting CIDRs
    data, err := os.ReadFile(tmpFile)
    if err != nil {
        t.Fatalf("cannot read output file: %v", err)
    }
    expected := "1.2.3.0/24\n2.2.2.2/32\n"
    if string(data) != expected {
        t.Fatalf("unexpected output.\nGot:\n%s\nWant:\n%s", string(data), expected)
    }
}

```

Use `t.TempDir()` to ensure the test writes to an isolated, auto-cleaned directory. The expected output string should reflect the exact CIDRs embedded in your `testdata/custom.dat` fixture.

## Managing Test Fixtures

Generate minimal fixtures to keep tests fast and deterministic. For binary formats like V2Ray's `.dat` or MaxMind's `.mmdb`, you can create small reference files using the CLI itself:

```bash
go run ./ convert -c testdata/config.json

```

Where [`config.json`](https://github.com/loyalsoldier/geoip/blob/main/config.json) references a tiny text list containing only the CIDRs you need to test. Commit these generated files to `plugin/<name>/testdata/` so the test suite remains self-contained and works offline.

The reference implementation in [`plugin/special/test.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/special/test.go) demonstrates how to inject a known CIDR (`127.0.0.0/8`) programmatically without external files, which you can adapt for unit-style tests that don't require full file I/O.

## Integrating with CI/CD

The repository's existing continuous integration pipeline, defined in [`.github/workflows/build.yml`](https://github.com/loyalsoldier/geoip/blob/main/.github/workflows/build.yml), executes `go test ./...` across all packages. By placing your `*_test.go` files alongside the plugin code and including fixtures in `testdata/`, your automated tests for custom GeoIP data transformations automatically integrate with the CI workflow without additional configuration.

Run tests locally during development to validate your changes:

```bash
go test ./...

```

Or target a specific package for faster iteration:

```bash
go test ./plugin/v2ray/...

```

## Troubleshooting Common Testing Issues

When implementing automated testing for format conversions, you may encounter these specific failures:

- **Nil container errors**: Always initialize the container with `lib.NewContainer()` before passing it to `Input()`. Attempting to use a nil container causes panics during entry insertion.

- **Missing registration errors**: If `lib.GetInputConfigCreator` returns `ok = false`, verify that the plugin's `init()` function actually calls the registration function. Check files like [`plugin/v2ray/dat_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/v2ray/dat_in.go) for the correct registration pattern.

- **Path resolution failures**: Output plugins like `textOut` often require an absolute or relative path in their JSON configuration. Use `t.TempDir()` to generate a writable temporary path and inject it into the configuration bytes, as shown in the test example above.

- **Fixture bloat**: Large MaxMind or V2Ray databases slow down tests and consume memory. Keep fixtures under 10 KB, containing only IPv4 and IPv6 edge cases (e.g., `/32`, `/128`, overlapping ranges) rather than full commercial databases.

## Extending Tests to Other Formats

The same testing pattern applies across all supported formats. Simply change the type strings and configuration JSON to target different converters:

- **MaxMind CSV input**: Use `maxmindGeoLite2CountryCSVIn` from [`plugin/maxmind/maxmind_country_csv_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/maxmind/maxmind_country_csv_in.go)
- **JSON plaintext input**: Use `jsonIn` from [`plugin/plaintext/json_in.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/plaintext/json_in.go)
- **MaxMind MMDB output**: Use `maxmindMMDB` from [`plugin/maxmind/maxmind_country_mmdb_out.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/maxmind/maxmind_country_mmdb_out.go)
- **Clash rule set output**: Use `clashRuleSet` from [`plugin/plaintext/clash_out.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/plaintext/clash_out.go)

Each converter expects specific configuration fields documented in [`configuration.md`](https://github.com/loyalsoldier/geoip/blob/main/configuration.md). Pass these as JSON bytes to the creator functions in your tests.

## Summary

- Create `*_test.go` files next to the plugin code you are validating to ensure the Go test runner discovers them automatically.
- Initialize a fresh `lib.Container` using `lib.NewContainer()` for every test to prevent cross-test contamination.
- Load converters via `lib.GetInputConfigCreator` and `lib.GetOutputConfigCreator` to exercise the actual registration logic used by the CLI.
- Store minimal binary and text fixtures in a `testdata/` subdirectory, generating them with the CLI if necessary.
- Capture output to temporary directories using `t.TempDir()` and assert against expected CIDR strings or format-specific structures.
- Execute `go test ./...` to validate your automated tests locally; the existing [`.github/workflows/build.yml`](https://github.com/loyalsoldier/geoip/blob/main/.github/workflows/build.yml) CI pipeline will automatically run these tests on every pull request.

## Frequently Asked Questions

### How do I test a custom input plugin that isn't registered in the main library?

Ensure your plugin file includes an `init()` function that calls `lib.RegisterInputConfigCreator` with a unique type string. Without this registration step, `lib.GetInputConfigCreator` cannot locate your converter, and the test will fail with a "not registered" error. See [`plugin/special/test.go`](https://github.com/loyalsoldier/geoip/blob/main/plugin/special/test.go) for a minimal working example of registration.

### What is the best way to handle binary format fixtures like .mmdb or .dat files?

Generate small, deterministic fixtures using the CLI tool itself (e.g., `go run ./ convert` with a minimal config), then commit these files to your `testdata/` directory. Keep them under 10 KB by including only a handful of CIDRs that exercise IPv4, IPv6, and edge cases like single IPs (`/32`) and network ranges.

### Can I run tests in parallel for faster CI execution?

Yes, but use caution when writing to files. Either avoid `t.Parallel()` for tests that write to the filesystem, or ensure each parallel test uses a unique subdirectory within `t.TempDir()`. The `lib.Container` itself is safe for concurrent use, but output converters that write to fixed paths may cause race conditions.

### How do I verify that my transformation correctly filters or merges CIDRs?

Populate the input fixture with overlapping or adjacent CIDRs (e.g., `192.168.0.0/24` and `192.168.1.0/24`), run them through the container, and inspect the output. The `lib.Container` automatically merges entries when possible. Assert that the output contains the expected merged ranges or that excluded ranges are absent, depending on your filter logic.