How the GeoIP Instance Interface Manages InputConverter and OutputConverter for Data Transformation

The Instance interface in lib/instance.go orchestrates data transformation by maintaining ordered slices of InputConverter and OutputConverter implementations, executing them sequentially through RunInput and RunOutput methods while sharing a mutable Container state across the pipeline.

The Instance interface serves as the core orchestration layer in the loyalsoldier/geoip repository, abstracting the complexity of GeoIP data transformation pipelines. It manages the lifecycle of input and output converters, allowing dynamic composition and deterministic execution of data processing workflows through a declarative or programmatic API.

Core Architecture of the Instance Interface

The Instance interface is defined in lib/instance.go (lines 12‑22) and provides a contract for pipeline orchestration that decouples converter management from execution logic.

Interface Definition

The interface exposes methods for configuration loading, dynamic composition, and pipeline execution:

type Instance interface {
    InitConfig(configFile string) error
    InitConfigFromBytes(content []byte) error
    AddInput(InputConverter)
    AddOutput(OutputConverter)
    ResetInput()
    ResetOutput()
    RunInput(Container) error
    RunOutput(Container) error
    Run() error
}

Source: lib/instance.go lines 12‑22

Concrete Implementation

The private instance struct implements this interface by maintaining ordered collections of converters:

type instance struct {
    input  []InputConverter
    output []OutputConverter
}

Source: lib/instance.go lines 24‑27

These slices preserve insertion order, ensuring deterministic execution during pipeline runs. The design intentionally uses ordered collections rather than maps to guarantee that converters execute in the exact sequence they were registered.

Managing Converter Collections

The Instance interface provides dynamic composition capabilities through explicit add and reset operations, enabling both programmatic and configuration-driven pipeline assembly.

Dynamic Composition

You can append converters to the pipeline using AddInput and AddOutput. According to the source code (lines 73‑78), these methods append to the respective slices using the built-in append function:

func (i *instance) AddInput(ic InputConverter) {
    i.input = append(i.input, ic)
}

func (i *instance) AddOutput(oc OutputConverter) {
    i.output = append(i.output, oc)
}

This design allows runtime assembly of complex transformation chains, supporting scenarios where pipeline stages must be determined dynamically based on runtime conditions.

Resetting State

The ResetInput and ResetOutput methods (lines 81‑87) recreate empty slices, effectively clearing the pipeline for reuse:

func (i *instance) ResetInput() {
    i.input = []InputConverter{}
}

func (i *instance) ResetOutput() {
    i.output = []OutputConverter{}
}

This capability supports reuse of Instance objects across multiple distinct transformation workflows without creating new instances, optimizing resource utilization in long-running applications.

Execution Flow and Data Transformation

The Instance interface orchestrates data transformation by executing converters in sequence while maintaining shared state through a Container object that carries the data payload.

The Container Pattern

The Container serves as a mutable data payload that carries state through the transformation pipeline. When Run() is invoked, it creates a fresh Container via NewContainer() and passes it sequentially through each processing stage. This shared state pattern eliminates complex message passing while maintaining type safety through interface contracts defined in lib/lib.go.

Input Processing

The RunInput method (lines 91‑96) iterates over the input slice, handing the Container from one converter to the next:

func (i *instance) RunInput(container Container) error {
    var err error
    for _, ic := range i.input {
        container, err = ic.Input(container)
        if err != nil {
            return err
        }
    }
    return nil
}

Each InputConverter transforms the container (such as parsing JSON or reading MaxMind databases) and returns the modified container for the next stage. If any stage returns an error, the pipeline halts immediately to prevent corrupted data from propagating.

Output Processing

The RunOutput method (lines 101‑106) processes the final container through all OutputConverter implementations:

func (i *instance) RunOutput(container Container) error {
    for _, oc := range i.output {
        if err := oc.Output(container); err != nil {
            return err
        }
    }
    return nil
}

Unlike inputs, output converters do not return a modified container; they persist data to files, databases, or other destinations. The method implements fail-fast error handling, aborting on the first failure to ensure data integrity.

Full Pipeline Execution

The Run method (lines 111‑126) ties the entire workflow together:

func (i *instance) Run() error {
    if len(i.input) == 0 || len(i.output) == 0 {
        return errors.New("input type and output type must be specified")
    }
    
    container := NewContainer()
    
    if err := i.RunInput(container); err != nil {
        return err
    }
    
    if err := i.RunOutput(container); err != nil {
        return err
    }
    
    return nil
}

This method validates that both input and output converters exist, creates a fresh container, executes the input chain, then executes the output chain. The deterministic sequence ensures that all input processing completes before any output generation begins.

Configuration-Driven Converter Management

The Instance interface supports declarative pipeline construction through JSON or YAML configuration files, decoupling converter instantiation from orchestration logic.

JSON and YAML Configuration

The InitConfig and InitConfigFromBytes methods parse configuration files to automatically populate converter slices. According to lib/instance.go (lines 62‑68), these methods unmarshal the configuration and instantiate converters through the registry system.

A typical configuration file defines input and output arrays:

{
  "input": [
    { "type": "json", "action": "add", "args": {} }
  ],
  "output": [
    { "type": "maxmind-mmdb", "action": "output", "args": { "path": "./GeoIP.mmdb" } }
  ]
}

Registry Pattern for Converter Creation

The registry pattern, defined in lib/config.go (lines 19‑53), decouples converter instantiation from the Instance orchestration. The system maintains maps of creator functions keyed by type strings:

var inputConfigCreators = map[string]InputConfigCreator{}
var outputConfigCreators = map[string]OutputConfigCreator{}

func RegisterInputConfigCreator(typeName string, creator InputConfigCreator) {
    inputConfigCreators[typeName] = creator
}

When InitConfigFromBytes processes a configuration entry, it looks up the appropriate creator function, instantiates the concrete converter, and appends it to the internal slices via AddInput or AddOutput. This allows users to define entire transformation pipelines declaratively without writing Go code, while the Instance interface remains agnostic to specific converter implementations.

Converter Interface Contracts

Both InputConverter and OutputConverter embed three small interfaces (Typer, Actioner, Descriptioner) plus a single transformation method, as defined in lib/lib.go (lines 39‑50).

InputConverter Contract

type InputConverter interface {
    Typer
    Actioner
    Descriptioner
    Input(Container) (Container, error)
}

The Input method receives a Container, performs transformation (such as parsing JSON or reading MaxMind databases), and returns the modified container for the next stage.

OutputConverter Contract

type OutputConverter interface {
    Typer
    Actioner
    Descriptioner
    Output(Container) error
}

The Output method receives the final container and persists data to the target format (such as MaxMind MMDB or plaintext lists). It returns only an error, as the transformation is terminal.

Concrete implementations reside in the plugin/ directory, such as plugin/plaintext/json_in.go for JSON input and plugin/maxmind/maxmind_country_mmdb_out.go for MaxMind MMDB output.

Practical Implementation Examples

Manual Converter Registration

The following example demonstrates programmatic assembly of a transformation pipeline:

package main

import (
	"log"

	"github.com/loyalsoldier/geoip/lib"
	"github.com/loyalsoldier/geoip/plugin/plaintext"
	"github.com/loyalsoldier/geoip/plugin/maxmind"
)

func main() {
	// Create a new instance
	inst, err := lib.NewInstance()
	if err != nil {
		log.Fatalf("cannot create instance: %v", err)
	}

	// Add JSON input converter
	jsonIn := plaintext.NewJSONInput()
	inst.AddInput(jsonIn)

	// Add MaxMind MMDB output converter
	mmdbOut := maxmind.NewMMDBOutput("./GeoIP.mmdb")
	inst.AddOutput(mmdbOut)

	// Execute the pipeline
	if err = inst.Run(); err != nil {
		log.Fatalf("pipeline failed: %v", err)
	}
}

Configuration-Driven Pipeline

Alternatively, define the pipeline declaratively in JSON:

{
  "input": [
    { "type": "json", "action": "add", "args": {} }
  ],
  "output": [
    { "type": "maxmind-mmdb", "action": "output", "args": { "path": "./GeoIP.mmdb" } }
  ]
}

Load and execute via:

inst, _ := lib.NewInstance()
if err := inst.InitConfig("pipeline.json"); err != nil {
    log.Fatalf("config error: %v", err)
}
inst.Run()

Summary

The Instance interface in the loyalsoldier/geoip repository provides a robust orchestration layer for data transformation pipelines. Key architectural characteristics include:

• Ordered Collections: The private instance struct maintains []InputConverter and []OutputConverter slices to preserve execution sequence.
• Dynamic Composition: Methods like AddInput, AddOutput, ResetInput, and ResetOutput enable runtime pipeline assembly and reuse.
• Shared State: A mutable Container object carries data through the pipeline, passed sequentially through each InputConverter and finally to OutputConverter implementations.
• Declarative Configuration: The InitConfig methods support JSON/YAML-driven pipeline construction via a registry pattern that maps type strings to concrete converter implementations.
• Fail-Fast Execution: The Run, RunInput, and RunOutput methods implement immediate error propagation, halting the pipeline when any stage fails.

Frequently Asked Questions

How does the Instance interface maintain execution order across multiple converters?

The Instance interface maintains execution order by storing converters in ordered slices ([]InputConverter and []OutputConverter) within the private instance struct. When AddInput or AddOutput is called, converters are appended to these slices using the built-in append function. During execution, RunInput and RunOutput iterate over these slices with for _, ic := range i.input, ensuring first-in-first-out processing. This slice-based approach guarantees that converters execute in the exact sequence they were registered, whether added programmatically or loaded from configuration files.

What is the difference between InputConverter and OutputConverter in the GeoIP pipeline?

InputConverter and OutputConverter serve distinct roles in the data transformation pipeline, as defined in lib/lib.go (lines 39‑50). InputConverter implementations must provide an Input(Container) (Container, error) method that receives a container, performs transformation (such as parsing JSON files or reading MaxMind databases), and returns the modified container for the next stage. OutputConverter implementations provide an Output(Container) error method that receives the final container and persists data to the target format (such as MaxMind MMDB or plaintext lists), returning only an error since the transformation is terminal. This architectural separation ensures that input stages can chain mutations while output stages handle final persistence.

How does the registry pattern enable extensibility in the GeoIP Instance interface?

The registry pattern, implemented in lib/config.go (lines 19‑53), decouples converter instantiation from the Instance orchestration logic. The system maintains global maps (inputConfigCreators and outputConfigCreators) that associate type strings (such as "json" or "maxmind-mmdb") with factory functions. Plugin packages register their converters during initialization using RegisterInputConfigCreator or RegisterOutputConfigCreator. When InitConfigFromBytes processes a configuration file, it looks up the appropriate creator by type string, instantiates the converter, and appends it to the internal slices. This design allows third-party developers to add new converter types without modifying the core Instance implementation, supporting the open-source extensibility goals of the loyalsoldier/geoip project.

What happens when a converter returns an error during pipeline execution?

The Instance interface implements fail-fast error handling that immediately halts pipeline execution when any converter encounters an error. In RunInput (lines 91‑96), if any InputConverter returns a non-nil error from its Input method, the function immediately returns that error without processing remaining inputs. Similarly, RunOutput (lines 101‑106) checks for errors from each OutputConverter and aborts on the first failure. The main Run method (lines 111‑126) propagates these errors upward after validating that both input and output slices contain at least one converter. This strict error handling ensures data integrity by preventing partial or corrupted data from reaching output stages when upstream processing fails.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}