How to Use a Custom Tokenizer to Split Comma-Separated Property Values in Owner

Use the @TokenizerClass annotation to specify a custom Tokenizer implementation at either the method or interface level, overriding the default SplitAndTrimTokenizer that splits on commas.

The Owner library simplifies Java configuration management by mapping properties to type-safe interfaces. When converting a single property value into an array or collection, Owner must split the string. While the default tokenizer handles comma-separated values, you can inject custom logic by implementing the Tokenizer interface and declaring it via annotations.

Understanding the Tokenizer Interface and Default Behavior

Owner delegates string splitting to implementations of the Tokenizer interface, which defines a single method:

public interface Tokenizer {
    String[] tokens(String values);
}

By default, Owner uses SplitAndTrimTokenizer (located in org.aeonbits.owner.SplitAndTrimTokenizer.java), which splits input on commas and trims whitespace from each token. This satisfies most use cases where properties follow the pattern value1, value2, value3.

Creating a Custom Tokenizer

To implement custom splitting logic—such as preserving empty tokens, splitting on multiple delimiters, or handling escaped characters—create a class that implements Tokenizer.

The following example from the Owner test suite demonstrates a tokenizer that splits on commas while preserving trailing empty strings:

package org.aeonbits.owner.typeconversion.arrays;

import org.aeonbits.owner.Tokenizer;

public class CustomCommaTokenizer implements Tokenizer {
    @Override
    public String[] tokens(String values) {
        // The -1 limit preserves trailing empty tokens (e.g., "a,b," becomes ["a", "b", ""])
        return values.split(",", -1);
    }
}

Source: owner/src/test/java/org/aeonbits/owner/typeconversion/arrays/CustomCommaTokenizer.java

Applying Custom Tokenizers with @TokenizerClass

Owner provides the @TokenizerClass annotation to specify which tokenizer to use. You can apply this annotation at two scopes, with method-level declarations taking precedence over interface-level declarations.

Method-Level Tokenizer Configuration

Annotate individual methods to use a specific tokenizer for that property only. This overrides any class-level tokenizer or @Separator annotation.

import org.aeonbits.owner.Config;
import org.aeonbits.owner.Config.TokenizerClass;
import org.aeonbits.owner.ConfigFactory;

public interface ServerConfig extends Config {

    @TokenizerClass(CustomCommaTokenizer.class)
    @DefaultValue("8080,8081,8082")
    int[] ports();
}

// Usage
ServerConfig cfg = ConfigFactory.create(ServerConfig.class);
int[] ports = cfg.ports();  // Returns [8080, 8081, 8082]

Class-Level Tokenizer Configuration

Apply @TokenizerClass to the interface declaration to set a default tokenizer for all methods that do not specify their own.

@TokenizerClass(CustomCommaTokenizer.class)
public interface AppConfig extends Config {

    @DefaultValue("one,two,three")
    String[] names();  // Uses CustomCommaTokenizer

    @Separator(";")   // Overrides class-level tokenizer for this method
    @DefaultValue("a; b; c")
    String[] semicolons();  // Uses built-in SplitAndTrimTokenizer with semicolon
}

Handling Conflicts with @Separator

Owner enforces mutual exclusivity between @TokenizerClass and @Separator. If both annotations appear on the same method or class, the library throws an UnsupportedOperationException during configuration creation.

This validation occurs in Config.java at lines 320-343, where the annotation processor checks for the presence of both annotations and raises the exception if they conflict.

How Tokenizer Resolution Works

Owner resolves the appropriate tokenizer at runtime through TokenizerResolver.resolveTokenizer(Method) (located in org.aeonbits.owner.TokenizerResolver.java). The resolution follows this precedence:

1. Method-level @TokenizerClass — Highest priority
2. Method-level @Separator — Uses built-in SplitAndTrimTokenizer with custom delimiter
3. Class-level @TokenizerClass — Default for all methods in the interface
4. Default SplitAndTrimTokenizer — Fallback when no annotations are present

Complete Working Example

The following example demonstrates class-level tokenizer configuration with method-level overrides, based on the test case TokenizerAnnotationOnClassLevelTest.java:

// Custom tokenizer that splits on dashes
public class CustomDashTokenizer implements Tokenizer {
    @Override
    public String[] tokens(String values) {
        return values.split("-");
    }
}

// Interface configuration
@TokenizerClass(CustomDashTokenizer.class)
public interface MixedConfig extends Config {

    @TokenizerClass(CustomCommaTokenizer.class)  // Override: use commas
    @DefaultValue("10,20,30")
    int[] commaSeparated();

    @Separator(";")  // Override: use semicolons with default tokenizer
    @DefaultValue("1;2;3")
    int[] semicolonSeparated();

    @DefaultValue("7-8-9")  // Uses class-level dash tokenizer
    int[] dashSeparated();
}

// Usage
MixedConfig cfg = ConfigFactory.create(MixedConfig.class);
System.out.println(Arrays.toString(cfg.commaSeparated()));      // [10, 20, 30]
System.out.println(Arrays.toString(cfg.semicolonSeparated())); // [1, 2, 3]
System.out.println(Arrays.toString(cfg.dashSeparated()));      // [7, 8, 9]

Summary

• Implement the Tokenizer interface to define custom splitting logic for property values.
• Apply custom tokenizers using the @TokenizerClass annotation at either the method or interface level.
• Method-level @TokenizerClass declarations override class-level settings.
• Do not combine @TokenizerClass with @Separator on the same element; this triggers an UnsupportedOperationException in Config.java.
• The default SplitAndTrimTokenizer handles comma-separated values when no custom tokenizer is specified.

Frequently Asked Questions

How do I split property values on a delimiter other than commas?

Create a class implementing the Tokenizer interface and override the tokens(String values) method to use your preferred delimiter. For example, split on semicolons using values.split(";"). Then annotate your config method or interface with @TokenizerClass(YourTokenizer.class).

Can I use @TokenizerClass and @Separator together?

No. Owner explicitly forbids combining these annotations on the same method or interface. If both are present, the library throws an UnsupportedOperationException during configuration initialization, as validated in Config.java lines 320-343.

What happens if I don't specify a custom tokenizer?

Owner defaults to SplitAndTrimTokenizer, which splits input strings on commas and trims whitespace from each resulting token. This behavior applies to all array or collection return types when no @TokenizerClass or @Separator annotation is present.

How does Owner choose which tokenizer to use?

The resolution order implemented in TokenizerResolver.resolveTokenizer(Method) is: method-level @TokenizerClass → method-level @Separator → class-level @TokenizerClass → default SplitAndTrimTokenizer. Method-level annotations always take precedence over interface-level settings.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}