# Inference Provisioning in OpenWork: Scaling Model Providers and Controlling Usage

> OpenWork scales model providers and controls usage with a stateless proxy, Den DB schema, organization quotas, tiered limits, and rolling window policies for efficient inference provisioning.

- Repository: [Different AI/openwork](https://github.com/different-ai/openwork)
- Tags: internals
- Published: 2026-08-04

---

**OpenWork manages inference provisioning through a stateless proxy service backed by a Den DB schema that enforces organization-specific quotas, supports horizontal scaling, and provides granular usage controls via tiered limits and rolling window policies.**

OpenWork's inference provisioning system allows organizations to enable and scale access to large language models from providers like OpenRouter, Moonshot, and DeepSeek while maintaining strict control over usage and costs. The architecture separates the inference proxy from the management API, storing all quota state in a shared database to ensure consistent enforcement across horizontally scaled workers. This guide examines the database schema, configuration options, and API endpoints that govern how OpenWork provisions and scales inference capabilities.

## Core Data Model in Den DB

All inference-related state persists in the **Den DB** schema defined in [`ee/packages/den-db/src/schema/inference.ts`](https://github.com/different-ai/openwork/blob/main/ee/packages/den-db/src/schema/inference.ts). The system uses six primary tables to track keys, policies, usage buckets, and audit trails:

- **`inference_keys`** – Stores per-organization inference keys used by the proxy, including `organization_id`, `org_membership_id`, `status`, and `key_hash`.

- **`inference_org_limit_policies`** – Defines quota limits for each organization tier with columns for `window_type`, `limit_amount`, and `reset_strategy`.

- **`inference_org_usage_buckets`** – Holds rolling window counters that enforce limits, tracking `window_start_at`, `window_end_at`, and `used_amount`.

- **`inference_org_upstream_provider_keys`** – Stores upstream provider credentials such as OpenRouter API keys in `external_key_hash`.

- **`inference_usage_ledger_entries`** – Audits every inference request for reporting and billing, linking `inference_key_id` to `usage_amount` and `external_event_id`.

- **`inference_usage_ledger_bucket_charges`** – Links ledger entries to usage buckets for fast quota reconciliation.

## Model Provider Configuration

The available models are declared in [`packages/types/src/den/inference.ts`](https://github.com/different-ai/openwork/blob/main/packages/types/src/den/inference.ts). Each entry in the inference model aliases configuration specifies how OpenWork maps internal names to upstream providers:

```typescript
{
  upstreamModel: "provider/model-id",   // e.g., "z-ai/glm-5.2"
  displayName: "OpenWork: GLM‑5.2",
  enabled: true,
  usageFactor: 1                     // Multiplier applied to raw token count
}

```

The **`usageFactor`** field allows OpenWork to weight expensive models differently when charging against an organization's quota, ensuring that premium models consume proportionally more of the allocated limit.

## Enabling and Configuring Inference

Administrators control inference provisioning via the **Org Inference API** (`/v1/inference`) implemented in [`ee/apps/den-api/src/routes/org/inference.ts`](https://github.com/different-ai/openwork/blob/main/ee/apps/den-api/src/routes/org/inference.ts). The endpoint supports two primary operations:

**Retrieving current status:**

```bash
curl -H "Authorization: Bearer $TOKEN" \
     https://api.openwork.dev/v1/inference

```

**Updating configuration:**

```typescript
// PATCH payload to enable inference and upgrade tier
{
  "enabled": true,
  "tier": "tier2"
}

```

When processing a PATCH request, the server performs several validation steps:

1. Calls `setInferenceEnabled` to write the new flag and tier to the `inference_keys` table.
2. Verifies the organization owns an active subscription via `organizationHasActiveInferenceSubscription`.
3. Confirms the presence of the `OPENROUTER_MANAGEMENT_API_KEY` environment variable, returning a clear error if the upstream provider key is missing.

## Scaling the Inference Proxy

The inference proxy runs as a **stateless service** located in `ee/apps/inference` behind the Den API. This design enables **horizontal scalability** through replication behind a load balancer (Kubernetes Deployment, Docker Swarm, etc.).

Because all quota state resides in the shared Den DB, every proxy replica reads identical usage buckets and enforces consistent limits. When a request would exceed the organization's `limit_amount`, the proxy immediately returns a **429 Too Many Requests** error, preventing runaway usage regardless of how many pods handle traffic.

## Usage Control: Limits and Reset Strategies

OpenWork supports flexible quota management through window types and reset strategies defined in [`packages/types/src/den/inference.ts`](https://github.com/different-ai/openwork/blob/main/packages/types/src/den/inference.ts):

```typescript
export const INFERENCE_WINDOW_TYPES = ["five_hour","weekly","monthly"] as const;
export const INFERENCE_RESET_STRATEGIES = ["anchored","activity_based"] as const;

```

**Window Types:**
- **`five_hour`** – Short-term burst control
- **`weekly`** – Standard organizational planning cycles  
- **`monthly`** – Long-term budget management

**Reset Strategies:**
- **Anchored** – Windows start at fixed epochs (e.g., every Monday for weekly windows).
- **Activity-based** – The window slides forward from the timestamp of the first request triggering the bucket.

The `INFERENCE_TIER_LIMITS` map assigns concrete quotas to each tier. For example, `tier1` receives 100 million tokens per five-hour window while `tier2` receives 150 million.

When processing requests, the proxy:

1. Looks up the organization's current `InferenceOrgLimitPolicy`.
2. Selects the appropriate usage bucket by `window_type`.
3. Validates that `used_amount + request_usage ≤ limit_amount`.
4. Increments `used_amount` and records a ledger entry if the check passes.

## Auditing and Reporting

Every successful inference request generates an **`InferenceUsageLedgerEntry`** and corresponding **`inference_usage_ledger_bucket_charges`** record. The reporting layer in [`ee/apps/inference/src/inference-reporting.ts`](https://github.com/different-ai/openwork/blob/main/ee/apps/inference/src/inference-reporting.ts) aggregates these immutable entries to produce:

- Per-organization usage dashboards displayed in the Den web UI.
- Billing events for the OpenWork Marketplace subscription.

Because the ledger is append-only, administrators can back-fill or audit historic usage at any time without risk of data mutation.

## Summary

- OpenWork's inference provisioning relies on a Den DB schema with six specialized tables tracking keys, policies, usage buckets, and audit trails.
- Model configuration in [`packages/types/src/den/inference.ts`](https://github.com/different-ai/openwork/blob/main/packages/types/src/den/inference.ts) supports usage factors for differential pricing of expensive models.
- The Org Inference API at `/v1/inference` enables administrators to toggle services and upgrade tiers while enforcing subscription validation.
- The stateless proxy architecture supports horizontal scaling while maintaining consistent quota enforcement through shared database state.
- Rolling window limits with anchored or activity-based reset strategies provide flexible usage control across five-hour, weekly, and monthly periods.

## Frequently Asked Questions

### How does OpenWork enforce usage limits when scaling the inference proxy horizontally?

OpenWork stores all quota state in the shared Den DB rather than in-memory. When the horizontally scaled proxy processes a request, each replica queries the same `inference_org_usage_buckets` table to check `used_amount` against `limit_amount`. If the request would exceed the quota, the proxy returns a 429 Too Many Requests error immediately, ensuring consistent enforcement regardless of which pod handles the traffic.

### What is the difference between anchored and activity-based reset strategies?

**Anchored** reset strategies start windows at fixed calendar epochs, such as every Monday for weekly limits or the first of the month for monthly quotas. **Activity-based** strategies slide the window forward from the timestamp of the first request that triggered the bucket, creating a rolling window that moves with the organization's actual usage patterns rather than calendar dates.

### How can administrators enable inference provisioning for an organization?

Administrators send a PATCH request to `/v1/inference` with an `enabled: true` flag and optional `tier` specification. The system validates admin permissions, checks for an active subscription via `organizationHasActiveInferenceSubscription`, and verifies the presence of upstream provider keys before activating the service. The implementation resides in [`ee/apps/den-api/src/routes/org/inference.ts`](https://github.com/different-ai/openwork/blob/main/ee/apps/den-api/src/routes/org/inference.ts).

### What happens when an organization exceeds its inference quota?

When the proxy detects that `used_amount + request_usage` would exceed the `limit_amount` defined in the organization's `InferenceOrgLimitPolicy`, it rejects the request with a 429 Too Many Requests status code. This hard stop prevents any additional usage until the current window expires and the bucket resets, or until the administrator upgrades to a higher tier with increased limits.