# ds4_dist_session_sync vs ds4_dist_session_eval: Understanding Distributed Session Functions

> Understand the key differences between ds4_dist_session_sync and ds4_dist_session_eval. Learn how ds4_dist_session_sync modifies state while ds4_dist_session_eval offers read-only evaluation.

- Repository: [Salvatore Sanfilippo/ds4](https://github.com/antirez/ds4)
- Tags: deep-dive
- Published: 2026-08-04

---

**`ds4_dist_session_sync` modifies KV-state during distributed inference while `ds4_dist_session_eval` performs read-only evaluation without cache mutation.**

Both functions form the core of the **distributed inference runtime** in the **antirez/ds4** repository. They share the same transport layer but serve fundamentally different purposes: one for stateful generation workflows, the other for stateless scoring and testing. This guide explores their implementation differences, use cases, and when to choose each approach.

## Core Differences Between ds4_dist_session_sync and ds4_dist_session_eval

| Feature | `ds4_dist_session_sync` | `ds4_dist_session_eval` |
|---------|-------------------------|--------------------------|
| **Primary purpose** | Executes prefill or generation with **KV-state synchronization** | Performs **read-only evaluation** without modifying KV-state |
| **Cache behavior** | Updates hidden-state, token hashes, and KV snapshots | No hidden-state stored, no token hash updates |
| **Typical use case** | Normal generation workflows (`ds4_cli`, `ds4_server`) | Scoring prompts, model testing, pure evaluation |
| **Return value** | `0` on success, session state mutated | `0` on success, session state unchanged |

## What ds4_dist_session_sync Does

In [`ds4_distributed.c`](https://github.com/antirez/ds4/blob/main/ds4_distributed.c), `ds4_dist_session_sync` serves as the **primary distributed execution path** for both prefill and incremental generation steps.

The function calls the **distributed prefill pipeline** via `dist_coordinator_prefill_prompt` when the request contains a prefill flag. It handles activation payload transmission, receives worker responses, updates KV snapshots, and persists hidden-state back to the session.

This is the function that **normal front-ends invoke** including:

- `ds4_cli` — the command-line interface
- `ds4_server` — the inference server
- `ds4_agent` — automated agents

The public API layer in [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c) delegates `ds4_session_sync()` to this distributed implementation when running in distributed mode.

### ds4_dist_session_sync Implementation Details

```c
/* Defined at ds4_distributed.c#L5517 */
int ds4_dist_session_sync(ds4_dist_session *s,
                          const ds4_tokens *prompt,
                          char *err, size_t errlen);

```

Key pipeline steps:

1. Detects prefill vs. decode phase from request flags
2. Invokes `dist_coordinator_prefill_prompt` for new prompts
3. Transmits activations to workers and aggregates results
4. Updates KV cache snapshots and token hash tables
5. Returns logits for the next token position

## What ds4_dist_session_eval Does

`ds4_dist_session_eval` provides a **stateless alternative** for scenarios where you need logits without side effects. It bypasses the prefill pipeline entirely and invokes `dist_coordinator_eval_prompt` directly.

The function **never writes hidden-state** to the session and **skips KV-snapshot handling**. This makes it ideal for:

- Prompt scoring and ranking
- Model correctness testing
- Stateless batch evaluation
- A/B testing without cache pollution

### ds4_dist_session_eval Implementation Details

```c
/* Defined at ds4_distributed.c#L5545 */
int ds4_dist_session_eval(ds4_dist_session *s,
                          const ds4_tokens *prompt,
                          char *err, size_t errlen);

```

The evaluation path shares the same distributed transport but omits:

- KV cache updates
- Token hash persistence
- Hidden-state storage
- Snapshot synchronization

## Code Examples: Using ds4_dist_session_sync and ds4_dist_session_eval

### Stateful Generation with ds4_session_sync

Front-end applications use the public wrapper which delegates to `ds4_dist_session_sync`:

```c
ds4_session *session = ds4_session_new(engine, options);
ds4_tokens prompt = ds4_tokenize("Hello, world!", engine->tokenizer);
char err[256];

/* Calls ds4_dist_session_sync internally in distributed mode */
int rc = ds4_session_sync(session, &prompt, err, sizeof(err));
if (rc != 0) {
    fprintf(stderr, "Sync error: %s\n", err);
}

```

**Call flow:** `ds4_session_sync` → `ds4_dist_session_sync` → prefill/decode pipeline → KV state updated

### Stateless Evaluation with ds4_session_eval

Test utilities and scoring systems use the eval path:

```c
ds4_session *session = ds4_session_new(engine, options);
ds4_tokens prompt = ds4_tokenize("What is the capital of France?", 
                                 engine->tokenizer);
char err[256];

/* Calls ds4_dist_session_eval internally — no KV update */
int rc = ds4_session_eval(session, &prompt, err, sizeof(err));
if (rc != 0) {
    fprintf(stderr, "Eval error: %s\n", err);
}

```

**Call flow:** `ds4_session_eval` → `ds4_dist_session_eval` → evaluation pipeline → session state unchanged

### Direct Low-Level Invocation

For fine-grained control over distributed behavior:

```c
/* Direct sync — updates KV cache */
int rc_sync = ds4_dist_session_sync(dist_session,
                                    &prompt,
                                    err,
                                    sizeof(err));

/* Direct eval — read-only operation */
int rc_eval = ds4_dist_session_eval(dist_session,
                                    &prompt,
                                    err,
                                    sizeof(err));

```

## Source File Reference Map

| File | Role in ds4_dist_session Functions |
|------|-------------------------------------|
| [[`ds4_distributed.c`](https://github.com/antirez/ds4/blob/main/ds4_distributed.c)](https://github.com/antirez/ds4/blob/main/ds4_distributed.c) | Core implementation at lines 5517 (`sync`) and 5545 (`eval`) |
| [[`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c)](https://github.com/antirez/ds4/blob/main/ds4.c) | Public API delegation layer |
| [[`ds4_cli.c`](https://github.com/antirez/ds4/blob/main/ds4_cli.c)](https://github.com/antirez/ds4/blob/main/ds4_cli.c) | CLI front-end using sync path |
| [[`ds4_server.c`](https://github.com/antirez/ds4/blob/main/ds4_server.c)](https://github.com/antirez/ds4/blob/main/ds4_server.c) | Server using both sync and eval paths |
| [`tests/`](https://github.com/antirez/ds4/tree/main/tests) | Test suite exercising both code paths |

## When to Use Each Function

**Choose `ds4_dist_session_sync` when you need:**

- Interactive generation with context preservation
- Incremental decoding with KV cache reuse
- Production inference serving
- Multi-turn conversation handling

**Choose `ds4_dist_session_eval` when you need:**

- Prompt scoring without affecting session state
- Parallel evaluation of multiple hypotheses
- Deterministic testing with clean cache state
- Stateless batch processing

## Performance and State Implications

Both functions incur similar network overhead for distributed coordination. However, `ds4_dist_session_sync` adds:

- KV snapshot serialization/deserialization
- Hidden-state persistence latency
- Token hash table updates

For read-only workloads, `ds4_dist_session_eval` eliminates these costs while maintaining distributed computation benefits.

## Summary

- **`ds4_dist_session_sync`** = distributed prefill/generation with full KV-state mutation via `dist_coordinator_prefill_prompt`
- **`ds4_dist_session_eval`** = distributed read-only evaluation via `dist_coordinator_eval_prompt` with no cache side effects
- Both functions live in [`ds4_distributed.c`](https://github.com/antirez/ds4/blob/main/ds4_distributed.c) (lines 5517 and 5545) and share transport infrastructure
- Front-ends access them through `ds4_session_sync()` and `ds4_session_eval()` wrappers in [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c)
- Use sync for stateful generation, eval for stateless scoring and testing

## Frequently Asked Questions

### Can I use ds4_dist_session_eval for production inference?

No. `ds4_dist_session_eval` is designed for **read-only scenarios** like scoring and testing. It does not update the KV cache, so subsequent calls would lack context. Use `ds4_dist_session_sync` for production generation workflows.

### Do both functions work in single-node mode?

The distributed implementations are specifically for multi-node configurations. Single-node inference uses separate code paths in [`ds4.c`](https://github.com/antirez/ds4/blob/main/ds4.c) that do not invoke these distributed functions. Check your engine configuration to determine which path is active.

### What happens if I call ds4_dist_session_sync twice with the same prompt?

Each call **advances the session state**. The second call treats the previous output as context, performing incremental generation rather than reprocessing the original prompt. This is the intended behavior for autoregressive models.

### Is there a performance difference beyond KV operations?

Network traffic is comparable since both use the same coordinator-worker protocol. The measurable difference comes from **snapshot handling and state persistence** in `ds4_dist_session_sync`. For latency-sensitive read-only tasks, prefer `ds4_dist_session_eval`.