# How Scheduled Background Jobs Work with Leader Election in Instatic

> Learn how Instatic uses leader election and database advisory locks with PostgreSQL or SQLite to ensure scheduled background jobs run successfully on only one instance.

- Repository: [CoreBunch/Instatic](https://github.com/CoreBunch/Instatic)
- Tags: internals
- Published: 2026-07-30

---

**Instatic ensures only one instance processes scheduled background jobs at any given moment by using database advisory locks, with PostgreSQL providing distributed coordination and SQLite using sentinel tokens for single-process deployments.**

In the CoreBunch/Instatic repository, every recurring background task—from the publish scheduler to the plugin scheduler—runs on every host instance, but only the elected leader performs the actual work. This leader election mechanism relies on a shared advisory lock implementation that transparently supports both high-availability PostgreSQL clusters and single-process SQLite environments.

## The Advisory Lock Foundation

The core of Instatic's leader election resides in **[`server/db/advisoryLock.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/db/advisoryLock.ts)**, which provides a unified interface over PostgreSQL's native advisory locks and SQLite's single-process mode.

The implementation differs by database backend:

- **PostgreSQL** – Uses the native `pg_try_advisory_lock` and `pg_advisory_unlock` functions. Each lock is identified by a 31-bit integer (the *advisory lock key*). Only the instance that successfully acquires the lock becomes the leader for that tick; others receive `null` and skip the work.
- **SQLite** – Runs in single-process mode, so no cross-process coordination is required. The lock helper catches the "function not found" error and returns a sentinel token (`'sqlite-leader'`), allowing the caller to behave as if it were the leader without any database coordination.

The public API exported from [`advisoryLock.ts`](https://github.com/CoreBunch/Instatic/blob/main/advisoryLock.ts) consists of three key functions:

| Function | Purpose |
|----------|---------|
| `tryAcquireLeader(db, lockKey)` | Attempts to claim the lock; returns `'pg-advisory'`, `'sqlite-leader'`, or `null`. |
| `releaseLeader(db, token, lockKey, logPrefix)` | Releases a PostgreSQL advisory lock (no-op for SQLite). |
| `withSchedulerLeaderLock(db, lockKey, logPrefix, fn)` | Convenience wrapper that runs `fn` only if the instance wins the election, automatically releasing the lock afterwards. |

## Publish Scheduler Implementation

The publish scheduler in **[`server/publish/publishScheduler.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/publish/publishScheduler.ts)** handles the automatic publishing of content rows where `status = 'scheduled'` and `scheduled_publish_at` is in the past.

The scheduler uses a `setInterval` timer (defaulting to 10 seconds) to drive a polling loop. Before each tick, it calls `withSchedulerLeaderLock` with the advisory lock key `982410937`:

```typescript
import { withSchedulerLeaderLock } from '@/server/db/advisoryLock';
import { listDuePublishSchedules, cancelScheduledPublish } from '@/server/repositories/data/rows';
import { publishDataRow, emitContentEntryUpdated } from '@/server/publish/publishRow';

const ADVISORY_LOCK_KEY = 982410937;

async function tickPublishScheduler(db: DbClient) {
  await withSchedulerLeaderLock(db, ADVISORY_LOCK_KEY, '[publish-scheduler]', async () => {
    const due = await listDuePublishSchedules(db, new Date().toISOString(), 25);
    for (const entry of due) {
      try {
        await publishDataRow(db, entry.rowId, null);
        await emitContentEntryUpdated(db, entry.rowId, ['status'], { kind: 'system' });
      } catch (err) {
        console.error(`[publish-scheduler] failed for ${entry.rowId}:`, err);
        await cancelScheduledPublish(db, entry.rowId, null);
      }
    }
  });
}

```

If another host instance holds the lock, the entire tick is skipped. This guarantees that **only one instance ever publishes the same row**, even when multiple servers are running behind a load balancer.

## Plugin Scheduler Implementation

The plugin scheduler in **[`server/plugins/scheduler.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/plugins/scheduler.ts)** drives the `api.cms.schedule.*` API used by installed plugins. It follows the same leader election pattern but uses a distinct advisory lock key to avoid contention.

The flow mirrors the publish scheduler but adds an additional safety layer:

1. A `setInterval` timer (also 10 seconds) triggers `tickPluginScheduler`.
2. The tick wraps its work inside `withSchedulerLeaderLock` using lock key `712830541`.
3. It fetches due schedules via `selectDueSchedules` and attempts an **atomic row-level claim** via `tryClaimSchedule`, preventing two ticks from firing the same schedule even if the advisory lock were somehow shared.
4. Each claimed schedule executes in a worker (`runScheduleInWorker`), results are recorded, and the advisory lock is released.

Because the advisory lock is scoped by a unique key, the publish scheduler and plugin scheduler never contend with each other, even when both run in the same process.

## Failure Handling and High Availability

Instatic's leader election handles failure scenarios differently depending on the database backend:

- **PostgreSQL** – If the leader crashes before releasing the lock, PostgreSQL automatically frees the advisory lock when the connection is lost. This allows the next tick on another host to acquire the lock and become the new leader.
- **SQLite** – No crash hand-off is required because there is only one process; the sentinel token is always returned.
- **Task-level failures** – Both schedulers treat individual task failures (e.g., a publish error) as ordinary errors, logging them and either reverting state via `cancelScheduledPublish` or pausing the schedule after a configurable failure cap (in the plugin scheduler).

## Code Examples

### Running a One-Off Scheduled Publish

To implement a custom scheduler tick that respects leader election:

```typescript
import { withSchedulerLeaderLock } from '@/server/db/advisoryLock';
import { listDuePublishSchedules, cancelScheduledPublish } from '@/server/repositories/data/rows';
import { publishDataRow, emitContentEntryUpdated } from '@/server/publish/publishRow';

const ADVISORY_LOCK_KEY = 982410937;

async function tickPublishScheduler(db: DbClient) {
  await withSchedulerLeaderLock(db, ADVISORY_LOCK_KEY, '[publish-scheduler]', async () => {
    const due = await listDuePublishSchedules(db, new Date().toISOString(), 25);
    for (const entry of due) {
      try {
        await publishDataRow(db, entry.rowId, null);
        await emitContentEntryUpdated(db, entry.rowId, ['status'], { kind: 'system' });
      } catch (err) {
        console.error(`[publish-scheduler] failed for ${entry.rowId}:`, err);
        await cancelScheduledPublish(db, entry.rowId, null);
      }
    }
  });
}

```

### Using the Advisory Lock Helper Directly

For custom background jobs that need leader election:

```typescript
import { tryAcquireLeader, releaseLeader } from '@/server/db/advisoryLock';

const LOCK_KEY = 123456789;
const LOG_PREFIX = '[my-job]';

async function runIfLeader(db: DbClient) {
  const token = await tryAcquireLeader(db, LOCK_KEY);
  if (!token) {
    // Another instance is the leader – skip work.
    return;
  }
  try {
    // …perform exclusive work here…
  } finally {
    await releaseLeader(db, token, LOCK_KEY, LOG_PREFIX);
  }
}

```

### Starting the Plugin Scheduler at Boot

Initialize the plugin scheduler idempotently at application startup:

```typescript
import { startScheduler } from '@/server/plugins/scheduler';
import { db } from '@/server/db/client';

startScheduler(db); // Safe to call on every boot

```

## Summary

- **[`server/db/advisoryLock.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/db/advisoryLock.ts)** provides the central leader election mechanism using PostgreSQL advisory locks or SQLite sentinel tokens.
- **Publish scheduler** ([`server/publish/publishScheduler.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/publish/publishScheduler.ts)) uses lock key `982410937` to ensure only one instance publishes scheduled content rows.
- **Plugin scheduler** ([`server/plugins/scheduler.ts`](https://github.com/CoreBunch/Instatic/blob/main/server/plugins/scheduler.ts)) uses lock key `712830541` with an additional atomic row-level claim to prevent duplicate executions.
- **PostgreSQL** automatically releases locks on connection loss, enabling automatic failover to healthy instances.
- **SQLite** deployments skip coordination entirely by returning `'sqlite-leader'` for all lock attempts.

## Frequently Asked Questions

### What happens if the leader instance crashes?

PostgreSQL automatically frees the advisory lock when the database connection is lost, allowing another instance to acquire the lock on the next tick. SQLite deployments do not require crash handling because they operate as a single process.

### Can the publish and plugin schedulers run concurrently?

Yes. Each scheduler uses a distinct advisory lock key (`982410937` for publish, `712830541` for plugins), ensuring they never block each other while maintaining single-leader semantics within their respective domains.

### How does leader election work with SQLite?

SQLite runs in single-process mode, so no distributed coordination is needed. The advisory lock helper catches the missing function error and returns a sentinel token (`'sqlite-leader'`), causing the instance to always behave as the leader without database overhead.

### What is the default polling interval for scheduled jobs?

Both the publish scheduler and plugin scheduler poll every 10 seconds by default, using `setInterval` to trigger their respective tick functions. This interval determines how quickly scheduled tasks execute after their target time is reached.