What Is Deterministic Replay in Duroxide and Why pg_durable Orchestrations Require Deterministic Code

Deterministic replay in duroxide is the crash-recovery process that re-executes persisted activities from duro_history to resume pg_durable orchestrations after a crash, requiring purely deterministic orchestration code because any nondeterminism would cause replayed executions to diverge from the original history and break exactly-once guarantees.

pg_durable is the open-source Microsoft extension that brings durable task orchestration directly into PostgreSQL. Its runtime is built on duroxide, a Rust-based durable-task framework that relies on deterministic replay to resume orchestrations after a crash or restart. Every line of code inside an orchestration must be purely deterministic, because replay re-executes the same sequence of logic and expects bit-for-bit identical results.

How Deterministic Replay Works in Duroxide

When an orchestration runs inside pg_durable, each unit of work is executed as a duroxide activity. After an activity finishes, its result is persisted to the duro_history table. If the PostgreSQL server or the background worker crashes, the runtime can replay the workflow by re-executing the activity sequence from the last persisted checkpoint.

In src/worker.rs, the duroxide_worker_main function starts the runtime and initiates this replay logic. Because the framework reconstructs execution history rather than snapshotting full memory state, every replayed step must match the original run exactly.

Why Deterministic Code Is Required for pg_durable Orchestrations

Because replay re-executes the same source code, any nondeterminism would cause the new execution to diverge from the original history. Sources such as Utc::now(), random number generators, or unregistered external I/O would yield different values on restart, breaking correctness and potentially violating ordering constraints.

The guarantees that depend on strict determinism include:

• Crash-recovery – The workflow resumes from any checkpoint without losing work. Nondeterministic steps would produce different results on replay, corrupting the persisted state model.
• Exactly-once execution – Each activity runs once and its result is cached in duro_history; replay reuses that cached result. If an activity could return a different value on a second execution, the cache would be meaningless.
• Deterministic ordering – The DAG of futures is traversed in a fixed order. In src/types.rs, DUROXIDE_SCHEMA uses a BTreeMap for deterministic iteration, ensuring activity IDs are generated in a repeatable sequence.
• Testability – pg_durable’s regression tests rely on deterministic output files. Flaky behavior would make the *_out verification files useless.

Deterministic vs Nondeterministic Orchestration Patterns

The source code in src/orchestrations/mod.rs contains explicit comments and assertions that forbid nondeterministic sources inside orchestration logic. Below are concrete patterns that illustrate the boundary between valid and invalid code.

Valid Deterministic Orchestration

The following SQL DSL workflow is safe because every side-effect is expressed as a registered duroxide activity:

SELECT durable.start(
    durable.sql('SELECT count(*) AS total FROM users') => 'users'
    ~> durable.sql('SELECT count(*) AS total FROM orders') => 'orders'
    ~> durable.func('store_stats', '{"users": $users.rows[0].total, "orders": $orders.rows[0].total}')
);

All three steps—durable.sql and durable.func—are activities recorded by the runtime. The orchestration itself never calls now(), random(), or any other nondeterministic primitive, so replay after a crash is safe.

Invalid Nondeterministic Orchestration

This workflow is invalid because it embeds nondeterministic SQL directly in the orchestration:

SELECT durable.start(
    durable.sql('SELECT now()') => 'now'
    ~> durable.sql('SELECT random()') => 'rand'
    ~> durable.func('store', '{"ts": $now.rows[0].now, "r": $rand.rows[0].rand}')
);

If the worker restarts, now() and random() would return different values, causing the replayed execution to diverge from the history stored in duro_history.

Wrapping Nondeterministic Work in an Activity

The correct way to capture the current time is to route the call through a registered activity so the result is persisted:

SELECT durable.start(
    durable.func('fetch_current_time', '{}') => 'now'
    ~> durable.func('store', '{"ts": $now.rows[0].now}')
);

Under the hood, fetch_current_time is implemented as a duroxide activity. Its timestamp is saved to duro_history, so on replay the cached value is reused instead of recomputed.

Key Source Files That Enforce the Deterministic Contract

The deterministic guarantee is not merely a coding convention; it is embedded in the architecture and enforced by specific modules:

• src/worker.rs – Contains duroxide_worker_main, the background worker entry point that starts the duroxide runtime and upholds the deterministic contract at startup.
• src/types.rs – Defines DUROXIDE_SCHEMA and documents deterministic serialization order through the use of BTreeMap.
• src/orchestrations/mod.rs – Carries a top-level comment that explicitly prohibits random numbers, current time, and other nondeterministic sources inside orchestrations.
• src/orchestrations/execute_function_graph.rs – Sorts keys for deterministic logging and iterates the DAG with a deterministic clock.
• docs/pg_durable_spec.md – States that the runtime is powered by duroxide and that orchestrations are deterministic and survive crashes via replay.
• docs/ARCHITECTURE.md – Provides a visual diagram of the durable runtime and notes the deterministic ordering of DAG nodes.

Summary

• Deterministic replay is the duroxide mechanism that rebuilds workflow state by replaying persisted activities from duro_history after a crash.
• Deterministic code is mandatory because replay re-executes the same orchestration logic; any deviation would invalidate cached results and break exactly-once semantics.
• pg_durable enforces this contract at compile-time through module comments in src/orchestrations/ and at runtime through duroxide_worker_main in src/worker.rs.
• All side-effects—timers, SQL, HTTP calls—must be expressed as duroxide activities so their outputs are recorded and replayed identically.

Frequently Asked Questions

What is deterministic replay in duroxide?

Deterministic replay is the crash-recovery strategy used by the duroxide runtime to resume pg_durable orchestrations. After each activity completes, its result is written to the duro_history table; if PostgreSQL restarts, duroxide replays the activity sequence from the last checkpoint to reconstruct the workflow state exactly as it was.

Why is nondeterministic code forbidden in pg_durable orchestrations?

Nondeterministic code is forbidden because duroxide replays orchestration logic during recovery. If an instruction such as now() or random() produced a different value on replay, the execution path would diverge from the persisted history, breaking correctness and exactly-once guarantees. All side-effects must be routed through activities whose results are cached.

How does the runtime enforce deterministic execution?

The runtime enforces determinism architecturally and procedurally. The source in src/orchestrations/mod.rs documents the prohibition on nondeterministic sources, src/types.rs mandates deterministic iteration order with BTreeMap, and src/worker.rs starts the duroxide engine through duroxide_worker_main only after validating that the execution environment meets the deterministic contract, including a check that max_duroxide_connections is configured above the required minimum.

Are all SQL queries inside an orchestration automatically deterministic?

No. Only SQL queries that are routed through registered duroxide activities—such as durable.sql or durable.func—are safe, because their results are persisted. Raw or dynamic SQL that invokes nondeterministic functions like now() or random() inside the orchestration logic itself violates the deterministic requirement and will break replay.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}