ds4_dist_session_sync vs ds4_dist_session_eval: Understanding Distributed Session Functions

ds4_dist_session_sync modifies KV-state during distributed inference while ds4_dist_session_eval performs read-only evaluation without cache mutation.

Both functions form the core of the distributed inference runtime in the antirez/ds4 repository. They share the same transport layer but serve fundamentally different purposes: one for stateful generation workflows, the other for stateless scoring and testing. This guide explores their implementation differences, use cases, and when to choose each approach.

Core Differences Between ds4_dist_session_sync and ds4_dist_session_eval

Feature	ds4_dist_session_sync	ds4_dist_session_eval
Primary purpose	Executes prefill or generation with KV-state synchronization	Performs read-only evaluation without modifying KV-state
Cache behavior	Updates hidden-state, token hashes, and KV snapshots	No hidden-state stored, no token hash updates
Typical use case	Normal generation workflows (ds4_cli, ds4_server)	Scoring prompts, model testing, pure evaluation
Return value	0 on success, session state mutated	0 on success, session state unchanged

What ds4_dist_session_sync Does

In ds4_distributed.c, ds4_dist_session_sync serves as the primary distributed execution path for both prefill and incremental generation steps.

The function calls the distributed prefill pipeline via dist_coordinator_prefill_prompt when the request contains a prefill flag. It handles activation payload transmission, receives worker responses, updates KV snapshots, and persists hidden-state back to the session.

This is the function that normal front-ends invoke including:

• ds4_cli — the command-line interface
• ds4_server — the inference server
• ds4_agent — automated agents

The public API layer in ds4.c delegates ds4_session_sync() to this distributed implementation when running in distributed mode.

ds4_dist_session_sync Implementation Details

/* Defined at ds4_distributed.c#L5517 */
int ds4_dist_session_sync(ds4_dist_session *s,
                          const ds4_tokens *prompt,
                          char *err, size_t errlen);

Key pipeline steps:

1. Detects prefill vs. decode phase from request flags
2. Invokes dist_coordinator_prefill_prompt for new prompts
3. Transmits activations to workers and aggregates results
4. Updates KV cache snapshots and token hash tables
5. Returns logits for the next token position

What ds4_dist_session_eval Does

ds4_dist_session_eval provides a stateless alternative for scenarios where you need logits without side effects. It bypasses the prefill pipeline entirely and invokes dist_coordinator_eval_prompt directly.

The function never writes hidden-state to the session and skips KV-snapshot handling. This makes it ideal for:

• Prompt scoring and ranking
• Model correctness testing
• Stateless batch evaluation
• A/B testing without cache pollution

ds4_dist_session_eval Implementation Details

/* Defined at ds4_distributed.c#L5545 */
int ds4_dist_session_eval(ds4_dist_session *s,
                          const ds4_tokens *prompt,
                          char *err, size_t errlen);

The evaluation path shares the same distributed transport but omits:

• KV cache updates
• Token hash persistence
• Hidden-state storage
• Snapshot synchronization

Code Examples: Using ds4_dist_session_sync and ds4_dist_session_eval

Stateful Generation with ds4_session_sync

Front-end applications use the public wrapper which delegates to ds4_dist_session_sync:

ds4_session *session = ds4_session_new(engine, options);
ds4_tokens prompt = ds4_tokenize("Hello, world!", engine->tokenizer);
char err[256];

/* Calls ds4_dist_session_sync internally in distributed mode */
int rc = ds4_session_sync(session, &prompt, err, sizeof(err));
if (rc != 0) {
    fprintf(stderr, "Sync error: %s\n", err);
}

Call flow: ds4_session_sync → ds4_dist_session_sync → prefill/decode pipeline → KV state updated

Stateless Evaluation with ds4_session_eval

Test utilities and scoring systems use the eval path:

ds4_session *session = ds4_session_new(engine, options);
ds4_tokens prompt = ds4_tokenize("What is the capital of France?", 
                                 engine->tokenizer);
char err[256];

/* Calls ds4_dist_session_eval internally — no KV update */
int rc = ds4_session_eval(session, &prompt, err, sizeof(err));
if (rc != 0) {
    fprintf(stderr, "Eval error: %s\n", err);
}

Call flow: ds4_session_eval → ds4_dist_session_eval → evaluation pipeline → session state unchanged

Direct Low-Level Invocation

For fine-grained control over distributed behavior:

/* Direct sync — updates KV cache */
int rc_sync = ds4_dist_session_sync(dist_session,
                                    &prompt,
                                    err,
                                    sizeof(err));

/* Direct eval — read-only operation */
int rc_eval = ds4_dist_session_eval(dist_session,
                                    &prompt,
                                    err,
                                    sizeof(err));

Source File Reference Map

File	Role in ds4_dist_session Functions
[ds4_distributed.c](https://github.com/antirez/ds4/blob/main/ds4_distributed.c)	Core implementation at lines 5517 (sync) and 5545 (eval)
[ds4.c](https://github.com/antirez/ds4/blob/main/ds4.c)	Public API delegation layer
[ds4_cli.c](https://github.com/antirez/ds4/blob/main/ds4_cli.c)	CLI front-end using sync path
[ds4_server.c](https://github.com/antirez/ds4/blob/main/ds4_server.c)	Server using both sync and eval paths
tests/	Test suite exercising both code paths

When to Use Each Function

Choose ds4_dist_session_sync when you need:

• Interactive generation with context preservation
• Incremental decoding with KV cache reuse
• Production inference serving
• Multi-turn conversation handling

Choose ds4_dist_session_eval when you need:

• Prompt scoring without affecting session state
• Parallel evaluation of multiple hypotheses
• Deterministic testing with clean cache state
• Stateless batch processing

Performance and State Implications

Both functions incur similar network overhead for distributed coordination. However, ds4_dist_session_sync adds:

• KV snapshot serialization/deserialization
• Hidden-state persistence latency
• Token hash table updates

For read-only workloads, ds4_dist_session_eval eliminates these costs while maintaining distributed computation benefits.

Summary

• ds4_dist_session_sync = distributed prefill/generation with full KV-state mutation via dist_coordinator_prefill_prompt
• ds4_dist_session_eval = distributed read-only evaluation via dist_coordinator_eval_prompt with no cache side effects
• Both functions live in ds4_distributed.c (lines 5517 and 5545) and share transport infrastructure
• Front-ends access them through ds4_session_sync() and ds4_session_eval() wrappers in ds4.c
• Use sync for stateful generation, eval for stateless scoring and testing

Frequently Asked Questions

Can I use ds4_dist_session_eval for production inference?

No. ds4_dist_session_eval is designed for read-only scenarios like scoring and testing. It does not update the KV cache, so subsequent calls would lack context. Use ds4_dist_session_sync for production generation workflows.

Do both functions work in single-node mode?

The distributed implementations are specifically for multi-node configurations. Single-node inference uses separate code paths in ds4.c that do not invoke these distributed functions. Check your engine configuration to determine which path is active.

What happens if I call ds4_dist_session_sync twice with the same prompt?

Each call advances the session state. The second call treats the previous output as context, performing incremental generation rather than reprocessing the original prompt. This is the intended behavior for autoregressive models.

Is there a performance difference beyond KV operations?

Network traffic is comparable since both use the same coordinator-worker protocol. The measurable difference comes from snapshot handling and state persistence in ds4_dist_session_sync. For latency-sensitive read-only tasks, prefer ds4_dist_session_eval.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}