How pg_durable Detects Extension Drop/Recreate Events and Restarts Its Background Worker

pg_durable detects extension drop and recreate events by combining polling of the pg_extension catalog with a UUID epoch sentinel stored in df._worker_epoch, then gracefully shuts down the Duroxide runtime and returns to a wait loop so the background worker restarts automatically when the extension is recreated.

Microsoft's pg_durable runs a Rust-based background worker inside PostgreSQL that must survive extension drops and recreates without requiring a server restart. The worker implements a state machine in src/worker.rs that continuously polls for extension existence, validates schema ownership, and monitors an epoch sentinel to detect rapid drop-and-recreate cycles that a simple catalog query would miss.

Waiting for the Extension to Appear

When the background worker starts, it enters the wait_for_extension_creation() function and polls the pg_extension catalog every five seconds until a row with extname = 'pg_durable' appears. If PostgreSQL sets the ShutdownRequestPending flag before the extension exists, the worker exits cleanly.

// src/worker.rs
async fn wait_for_extension_creation(poll_pool: &sqlx::PgPool, poll_interval: Duration) -> bool {
    loop {
        if is_shutdown_requested() { return false; }
        if check_extension_exists(poll_pool).await {
            log!("pg_durable: extension detected, proceeding with initialization");
            return true;
        }
        tokio::time::sleep(poll_interval).await;
    }
}

Verifying Schema Ownership

Before launching the Duroxide runtime, the worker confirms that the duroxide schema is owned by the extension through pg_depend. This check, implemented in check_duroxide_schema_owned(), prevents the worker from running against a manually created schema that is not managed by the extension.

// src/worker.rs
if !check_duroxide_schema_owned(mgmt_pool).await {
    log!("pg_durable: duroxide schema missing or not extension-owned … will retry");
    tokio::time::sleep(retry_interval).await;
    continue;
}

If the ownership check fails, the state machine sleeps and retries instead of proceeding.

Writing the Epoch Sentinel

After the runtime initializes, the worker calls write_epoch_sentinel() to generate a UUID and insert it into df._worker_epoch. This row is deleted and replaced on every startup, ensuring that any subsequent worker instance can distinguish its own epoch from a previous one.

// src/worker.rs
async fn write_epoch_sentinel(pool: &sqlx::PgPool) -> Result<String, sqlx::Error> {
    let epoch_id = uuid::Uuid::new_v4().to_string();
    sqlx::query("DELETE FROM df._worker_epoch")
        .execute(pool).await?;
    sqlx::query("INSERT INTO df._worker_epoch (epoch_id, started_at, last_seen_at) \
                 VALUES ($1::uuid, now(), now())")
        .bind(&epoch_id)
        .execute(pool).await?;
    Ok(epoch_id)
}

The sentinel design is documented in docs/extension_lifecycle.md under the "Epoch sentinel: detecting drop+recreate" section.

Detecting Drop and Recreate Events

While the worker is running, run_until_extension_dropped_or_shutdown() interleaves two asynchronous checks:

• Shutdown check – reads PostgreSQL's ShutdownRequestPending flag every second.
• Extension check – every five seconds, validates the epoch sentinel if one was written, or falls back to a plain pg_extension lookup.

// src/worker.rs
let still_valid = match epoch_id {
    Some(eid) => check_epoch_sentinel(poll_pool, eid).await,
    None => check_extension_exists(poll_pool).await,
};
if !still_valid {
    log!("pg_durable: epoch sentinel gone — extension dropped or recreated");
    break;
}

Because DROP EXTENSION … CASCADE removes all objects the extension owns, the _worker_epoch table disappears along with its sentinel row. If the extension is recreated faster than the polling interval, a new sentinel UUID will be present, which does not match the worker's stored epoch_id, so the worker still detects the recreate event.

Graceful Shutdown and Automatic Restart

When the sentinel disappears or the extension entry vanishes, the worker breaks out of the monitoring loop and initiates a graceful shutdown of the Duroxide runtime with a ten-second timeout.

// src/worker.rs
log!("pg_durable: initiating duroxide runtime shutdown...");
duroxide_runtime.shutdown(Some(10_000)).await;
log!("pg_durable: duroxide runtime shutdown complete");

After shutdown completes, the outer run_duroxide_runtime() loop returns to its starting state and invokes wait_for_extension_creation() again. This yields automatic restart behavior without restarting the PostgreSQL server.

Practical Examples

Starting PostgreSQL with pg_durable Preloaded

Add the library to postgresql.conf and start the server:

# Add to postgresql.conf

shared_preload_libraries = 'pg_durable'

# Start PostgreSQL

pg_ctl start -D /path/to/data

The background worker starts immediately but logs a wait message because the extension has not yet been created.

Creating the Extension

CREATE EXTENSION pg_durable;

The worker detects the new pg_extension row, validates schema ownership, writes a fresh epoch sentinel, and begins processing.

Dropping the Extension

DROP EXTENSION pg_durable CASCADE;

The next sentinel check fails, producing logs that the epoch is gone, initiating shutdown, and returning the worker to its wait loop for the next CREATE EXTENSION.

Summary

• wait_for_extension_creation() polls pg_extension every five seconds until the extension appears.
• check_duroxide_schema_owned() verifies that the duroxide schema belongs to the extension before starting the runtime.
• write_epoch_sentinel() inserts a UUID into df._worker_epoch so the worker can detect drop and rapid drop-recreate cycles.
• run_until_extension_dropped_or_shutdown() interleaves shutdown checks with sentinel or catalog polls every five seconds.
• When a drop is detected, the worker calls duroxide_runtime.shutdown(Some(10_000)).await and the outer state machine restarts the wait loop automatically.

Frequently Asked Questions

Why does pg_durable use an epoch sentinel instead of only checking pg_extension?

A pure pg_extension poll can miss a rapid DROP EXTENSION followed by CREATE EXTENSION if both events occur between two five-second polls. The epoch sentinel is an owned database object that vanishes during DROP EXTENSION … CASCADE and is replaced with a new UUID on recreate, giving the worker a reliable signal that the extension lifecycle reset.

How often does the background worker poll for the extension or sentinel?

The worker polls for the extension existence every five seconds when waiting for creation and every five seconds when checking for drops while running. It checks PostgreSQL's ShutdownRequestPending flag every one second during active runtime monitoring.

What happens if the duroxide schema exists but is not owned by the extension?

The check_duroxide_schema_owned() function in src/worker.rs returns false, and the state machine logs a retry message instead of launching the Duroxide runtime. This prevents the worker from attaching to a schema that was created manually or by another process.

Does dropping pg_durable require a PostgreSQL server restart to bring the worker back?

No. The background worker is designed to survive extension drops without a server restart. After DROP EXTENSION pg_durable CASCADE, the worker shuts down the runtime gracefully and returns to wait_for_extension_creation(). Once CREATE EXTENSION pg_durable runs again, the worker detects it and restarts automatically.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}