How to Apply Directional Steering to Modify Model Behavior in ds4

Directional steering in ds4 allows you to bias transformer hidden states toward custom direction vectors on a per-layer basis by loading a binary steering file and setting scaling factors for attention and feed-forward outputs.

The ds4 inference engine implements directional steering as a lightweight, inference-time intervention that modifies latent representations without retraining the model. By projecting hidden states onto pre-computed direction vectors and adding scaled projections back into the activation tensors, you can influence style, tone, or content generation programmatically.

Loading Steering Vectors from Binary Files

To activate directional steering, you must supply a binary file containing 32-bit floating-point direction vectors. In ds4_cli.c, the CLI parses the --dir-steering-file argument and loads the data via read_f32_binary_file.

The binary file must contain exactly L × D floats, where L is the number of transformer layers and D is the hidden dimension size. The engine stores the resulting pointer in ctx->steering_dirs within the ds4_context structure defined in ds4.h.

float *steering_dirs = NULL;
uint64_t ndirs = (uint64_t)ctx->model.layers * ctx->model.hidden_dim;
if (!read_f32_binary_file(path_to_dir_bin, steering_dirs, ndirs))
    ds4_die("failed to load steering vectors");
ctx->steering_dirs = steering_dirs;
ctx->steering_ffn_scale   = 1.0f;   // defaults if not overridden
ctx->steering_attn_scale = 0.0f;

Configuring Steering Scales for Attention and FFN

You control where steering is applied using two independent scaling flags parsed in ds4_cli.c and ds4_agent.c:

• --dir-steering-ffn – Scale factor applied after the feed-forward network (default 1.0 if a steering file is provided but this flag is omitted)
• --dir-steering-attn – Scale factor applied after the attention block (default 0.0)

Setting a scale to 0.0 disables steering for that component, while values between 0.0 and 1.0 attenuate the effect. The parsed values are stored in ctx->steering_ffn_scale and ctx->steering_attn_scale.

The Projection Algorithm

The core steering logic resides in ds4.c. The function cpu_directional_steering_enabled guards execution to ensure a non-null direction pointer and non-zero scale are present. When active, cpu_directional_steering_project_rows computes the dot product between each row of the activation tensor and the layer-specific direction vector, then adds the scaled projection back into the tensor.

/* Returns true when steering is active for a given layer */
static bool cpu_directional_steering_enabled(const float *dirs, float scale) {
    return dirs != NULL && scale != 0.0f;
}

/* Projects each row of `x` onto the direction vector `dirs[il]`
   and adds the scaled projection back into `x`. */
static void cpu_directional_steering_project_rows(
        float *x, const float *dirs, uint32_t il,
        uint32_t rows, float scale)
{
    if (!cpu_directional_steering_enabled(dirs, scale)) return;

    const float *dir = dirs + il;               // one direction per layer
    for (uint32_t r = 0; r < rows; ++r) {
        float dot = 0.0f;
        /* (x + r*dim) points to the start of the row */
        for (uint32_t i = 0; i < dim; ++i)
            dot += x[r*dim + i] * dir[i];
        /* add the scaled projection */
        for (uint32_t i = 0; i < dim; ++i)
            x[r*dim + i] += scale * dot * dir[i];
    }
}

When running multi-threaded inference via ds4_thread_pool, the engine distributes rows across worker threads, with each thread invoking cpu_directional_steering_project_rows on its slice to maintain linear scaling with core count.

Integrating Steering into the Inference Loop

During the forward pass in ds4.c, the engine calls the projection helper immediately after the attention and feed-forward blocks. For each layer index il, the code executes:

After attention:

cpu_directional_steering_project_rows(
    attn_out,                      // output tensor
    ctx->steering_dirs,            // per‑layer direction vectors
    layer_index,                   // current layer id
    n_tokens,                      // number of rows (tokens)
    ctx->steering_attn_scale);     // scale from CLI flag

After feed-forward:

cpu_directional_steering_project_rows(
    ffn_out,
    ctx->steering_dirs,
    layer_index,
    n_tokens,
    ctx->steering_ffn_scale);

The same integration pattern appears in ds4_gpu.c and ds4_cuda.c for GPU-accelerated paths, ensuring steering behavior remains consistent across CPU and CUDA backends.

Command-Line Usage Examples

To steer generation toward a specific behavioral direction, invoke ds4 with the steering file and desired scales:

# Generate text while steering the model toward a custom direction

./ds4 -p "Write tersely" \
      --dir-steering-file   dir.bin \
      --dir-steering-ffn    0.8 \
      --dir-steering-attn   0.0

This example applies steering at 80% strength after the feed-forward network while leaving attention outputs unmodified. The file dir.bin must contain the properly dimensioned float array (layers × hidden_dim) in 32-bit IEEE 754 format.

Summary

• Directional steering in ds4 modifies hidden states by projecting them onto custom direction vectors loaded from binary files.
• The CLI flags --dir-steering-file, --dir-steering-ffn, and --dir-steering-attn control the data source and scaling factors, parsed in ds4_cli.c and mirrored in ds4_agent.c and ds4_server.c.
• The projection function cpu_directional_steering_project_rows in ds4.c applies the transformation row-wise after attention and/or feed-forward blocks when cpu_directional_steering_enabled returns true.
• Thread safety is maintained by distributing row-wise projection work across the ds4_thread_pool worker threads.
• Steering vectors must match the model architecture: one float per hidden dimension per layer.

Frequently Asked Questions

What file format does ds4 expect for steering vectors?

ds4 expects a raw binary file containing 32-bit IEEE 754 floating-point numbers. The file size must equal layers * hidden_dim * sizeof(float). The read_f32_binary_file function in ds4_cli.c loads this data into memory and assigns it to ctx->steering_dirs.

Where exactly does directional steering modify the model computation?

According to the source code in ds4.c, steering is applied immediately after the attention block and/or the feed-forward network (FFN) within each transformer layer. The function cpu_directional_steering_project_rows is called on the output tensors attn_out and ffn_out respectively, allowing you to target specific components of the computation graph.

Can I apply different scaling factors to attention and FFN layers?

Yes. ds4 exposes independent scales via --dir-steering-attn and --dir-steering-ffn. You can disable one by setting its scale to 0.0 while applying full or partial steering to the other. If you provide a steering file but omit the FFN scale, it defaults to 1.0, while the attention scale defaults to 0.0.

Does directional steering work with GPU acceleration in ds4?

Yes. While the reference implementation resides in ds4.c for CPU execution, the same steering logic and data structures are utilized by the GPU backends in ds4_gpu.c and ds4_cuda.c. The ds4_context structure holds the steering vectors and scales, making them available to both CPU and CUDA code paths during inference.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}