# How to Configure iii Engine with Custom Workers and Modules in config.yaml

> Learn how to configure iii engine with custom workers and modules in config.yaml. Extend iii's functionality by registering your own implementations easily.

- Repository: [iii/iii](https://github.com/iii-hq/iii)
- Tags: how-to-guide
- Published: 2026-05-28

---

**Configure the iii engine by defining `modules` and `workers` arrays in [`engine/config.yaml`](https://github.com/iii-hq/iii/blob/main/engine/config.yaml), then register custom implementations via `EngineBuilder::register_worker()` before calling `build()`.**

The **iii** engine (from the `iii-hq/iii` repository) boots by deserializing a single YAML configuration file that declares both built-in workers and custom Rust modules. Understanding how to structure this file and wire it into the `EngineBuilder` API is essential for extending the engine’s processing capabilities.

## Understanding the config.yaml Structure

The engine expects a top-level configuration object with two arrays: `modules` and `workers`. According to the source in [`src/workers/config.rs`](https://github.com/iii-hq/iii/blob/main/src/workers/config.rs)【L28-L35】, these arrays deserialize into the `EngineConfig` struct:

```rust
pub struct EngineConfig {
    pub modules: Vec<WorkerEntry>,
    pub workers: Vec<WorkerEntry>,
}

```

Each entry follows the `WorkerEntry` structure【L70-L77】, which supports:

- **name**: The worker identifier (e.g., `my::CustomWorker` or `iii-http`)
- **image**: Optional external binary image (for non-built-in workers)
- **config**: Optional JSON/YAML configuration object passed to the worker’s `create` method

The loader expands `${ENV}` variables during deserialization【L78-L101】 and merges both arrays into a single internal list before instantiation.

## Creating a Custom Worker Module

To register a custom module, you must implement the `Worker` trait and register it with the `EngineBuilder` before loading the configuration file.

### Implement the Worker Trait

Define your module in Rust by implementing the async `Worker` trait from [`src/workers/traits.rs`](https://github.com/iii-hq/iii/blob/main/src/workers/traits.rs):

```rust
use async_trait::async_trait;
use serde_json::Value;
use std::sync::Arc;
use iii::engine::Engine;
use iii::workers::traits::Worker;

#[derive(Debug)]
struct AnalyticsProcessor;

#[async_trait]
impl Worker for AnalyticsProcessor {
    fn name(&self) -> &'static str {
        "my::AnalyticsProcessor"
    }

    async fn create(
        _engine: Arc<Engine>,
        config: Option<Value>
    ) -> anyhow::Result<Box<dyn Worker>> {
        // Config block from config.yaml arrives here as JSON
        if let Some(cfg) = config {
            println!("Initializing with config: {}", cfg);
        }
        Ok(Box::new(AnalyticsProcessor))
    }

    async fn initialize(&self) -> anyhow::Result<()> {
        Ok(())
    }
}

```

### Register and Build

Use `EngineBuilder::register_worker()` to map the name to your implementation, then load the YAML file:

```rust
use iii::engine::EngineBuilder;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    EngineBuilder::new()
        .register_worker::<AnalyticsProcessor>("my::AnalyticsProcessor")
        .config_file("engine/config.yaml")?
        .build()
        .await?
        .serve()
        .await
}

```

The `WorkerRegistry` (populated via the `inventory` macro for built-ins and manual registration for customs) invokes `create_worker`【L86-L108】 to instantiate your module when it encounters the matching name in [`config.yaml`](https://github.com/iii-hq/iii/blob/main/config.yaml).

## Configuring Workers in config.yaml

The YAML file distinguishes between **modules** (Rust/TS/Python implementations) and **workers** (built-in or external processes).

### Basic Configuration Example

```yaml
modules:
  - name: "my::AnalyticsProcessor"
    config:
      batch_size: 1000
      flush_interval_sec: 30

workers:
  - name: iii-http
    config:
      port: 8080
      host: "0.0.0.0"

  - name: iii-queue
    config:
      concurrency: 4

```

**Key rules:**
- Built-in workers (like `iii-http`) **must not** specify an `image` field; the engine errors if they do.
- Custom modules listed under `modules` resolve to types registered via `register_worker()`.
- The `config` block is passed verbatim to the worker’s `create` method as a `serde_json::Value`.

### External Worker Binaries

To override a built-in worker or add an external binary, provide a unique name and an `image` field:

```yaml
workers:
  - name: "custom::QueueWorker"
    image: "docker.io/tenant/custom-queue:1.2.3"
    config:
      redis_url: "redis://localhost:6379"

```

The engine launches external images via `iii-worker start` as implemented in `WorkerRegistry::create_worker`.

## Programmatic Configuration Without YAML

If you prefer compile-time configuration, bypass `config_file()` and use `add_worker()`:

```rust
EngineBuilder::new()
    .register_worker::<AnalyticsProcessor>("my::AnalyticsProcessor")
    .add_worker(
        "my::AnalyticsProcessor",
        Some(serde_json::json!({
            "batch_size": 500,
            "flush_interval_sec": 60
        }))
    )
    .default_config()  // Loads built-in defaults for unspecified workers
    .build()
    .await?
    .serve()
    .await

```

This approach is equivalent to a YAML entry but hardcoded in the binary, useful for testing or locked-down deployments.

## Summary

- The **iii** engine reads `modules` and `workers` arrays from [`engine/config.yaml`](https://github.com/iii-hq/iii/blob/main/engine/config.yaml) into `EngineConfig` defined in [`src/workers/config.rs`](https://github.com/iii-hq/iii/blob/main/src/workers/config.rs).
- Each entry is a `WorkerEntry` containing a name, optional image, and configuration object.
- Register custom Rust modules via `EngineBuilder::register_worker()` before calling `config_file()` or `add_worker()`.
- Built-in workers resolve through `WorkerRegistry` using the `inventory` macro; custom workers require explicit registration.
- External binaries require an `image` field and a unique name distinct from built-in worker IDs.

## Frequently Asked Questions

### What is the difference between the `modules` and `workers` arrays in config.yaml?

Both arrays deserialize into `Vec<WorkerEntry>` and are merged during engine startup. By convention, `modules` holds custom Rust/TS/Python implementations you register via `EngineBuilder::register_worker()`, while `workers` holds built-in engine workers (like `iii-http`) or external binary images. Functionally, they behave identically after the merge step in [`src/workers/config.rs`](https://github.com/iii-hq/iii/blob/main/src/workers/config.rs)【L81-L86】.

### How do I use environment variables in config.yaml?

The engine automatically expands `${ENV_VAR}` syntax during YAML deserialization. For example, `port: ${PORT}` resolves to the value of the `PORT` environment variable before the configuration is validated. This expansion logic is implemented in `EngineConfig::config_file`【L78-L101】.

### Can I register a custom worker written in TypeScript or Python?

Yes, provided the engine binary includes the appropriate runtime bindings. You must register the foreign worker type through `EngineBuilder::register_worker()` using a Rust wrapper that implements the `Worker` trait and delegates to the TS/Python runtime. The [`config.yaml`](https://github.com/iii-hq/iii/blob/main/config.yaml) entry resembles a native Rust module with the name matching your registered identifier.

### Why does the engine fail when I specify an `image` for a built-in worker?

Built-in workers (such as `iii-http` or `iii-queue`) are compiled into the engine binary and instantiated via factory functions in `WorkerRegistry`. Specifying an `image` field triggers the external binary boot logic, which is incompatible with built-in types. Use the `image` field only for custom external workers with unique names.