Inference Provisioning in OpenWork: Scaling Model Providers and Controlling Usage

OpenWork manages inference provisioning through a stateless proxy service backed by a Den DB schema that enforces organization-specific quotas, supports horizontal scaling, and provides granular usage controls via tiered limits and rolling window policies.

OpenWork's inference provisioning system allows organizations to enable and scale access to large language models from providers like OpenRouter, Moonshot, and DeepSeek while maintaining strict control over usage and costs. The architecture separates the inference proxy from the management API, storing all quota state in a shared database to ensure consistent enforcement across horizontally scaled workers. This guide examines the database schema, configuration options, and API endpoints that govern how OpenWork provisions and scales inference capabilities.

Core Data Model in Den DB

All inference-related state persists in the Den DB schema defined in ee/packages/den-db/src/schema/inference.ts. The system uses six primary tables to track keys, policies, usage buckets, and audit trails:

• inference_keys – Stores per-organization inference keys used by the proxy, including organization_id, org_membership_id, status, and key_hash.

• inference_org_limit_policies – Defines quota limits for each organization tier with columns for window_type, limit_amount, and reset_strategy.

• inference_org_usage_buckets – Holds rolling window counters that enforce limits, tracking window_start_at, window_end_at, and used_amount.

• inference_org_upstream_provider_keys – Stores upstream provider credentials such as OpenRouter API keys in external_key_hash.

• inference_usage_ledger_entries – Audits every inference request for reporting and billing, linking inference_key_id to usage_amount and external_event_id.

• inference_usage_ledger_bucket_charges – Links ledger entries to usage buckets for fast quota reconciliation.

Model Provider Configuration

The available models are declared in packages/types/src/den/inference.ts. Each entry in the inference model aliases configuration specifies how OpenWork maps internal names to upstream providers:

{
  upstreamModel: "provider/model-id",   // e.g., "z-ai/glm-5.2"
  displayName: "OpenWork: GLM‑5.2",
  enabled: true,
  usageFactor: 1                     // Multiplier applied to raw token count
}

The usageFactor field allows OpenWork to weight expensive models differently when charging against an organization's quota, ensuring that premium models consume proportionally more of the allocated limit.

Enabling and Configuring Inference

Administrators control inference provisioning via the Org Inference API (/v1/inference) implemented in ee/apps/den-api/src/routes/org/inference.ts. The endpoint supports two primary operations:

Retrieving current status:

curl -H "Authorization: Bearer $TOKEN" \
     https://api.openwork.dev/v1/inference

Updating configuration:

// PATCH payload to enable inference and upgrade tier
{
  "enabled": true,
  "tier": "tier2"
}

When processing a PATCH request, the server performs several validation steps:

1. Calls setInferenceEnabled to write the new flag and tier to the inference_keys table.
2. Verifies the organization owns an active subscription via organizationHasActiveInferenceSubscription.
3. Confirms the presence of the OPENROUTER_MANAGEMENT_API_KEY environment variable, returning a clear error if the upstream provider key is missing.

Scaling the Inference Proxy

The inference proxy runs as a stateless service located in ee/apps/inference behind the Den API. This design enables horizontal scalability through replication behind a load balancer (Kubernetes Deployment, Docker Swarm, etc.).

Because all quota state resides in the shared Den DB, every proxy replica reads identical usage buckets and enforces consistent limits. When a request would exceed the organization's limit_amount, the proxy immediately returns a 429 Too Many Requests error, preventing runaway usage regardless of how many pods handle traffic.

Usage Control: Limits and Reset Strategies

OpenWork supports flexible quota management through window types and reset strategies defined in packages/types/src/den/inference.ts:

export const INFERENCE_WINDOW_TYPES = ["five_hour","weekly","monthly"] as const;
export const INFERENCE_RESET_STRATEGIES = ["anchored","activity_based"] as const;

Window Types:

• five_hour – Short-term burst control
• weekly – Standard organizational planning cycles
• monthly – Long-term budget management

Reset Strategies:

• Anchored – Windows start at fixed epochs (e.g., every Monday for weekly windows).
• Activity-based – The window slides forward from the timestamp of the first request triggering the bucket.

The INFERENCE_TIER_LIMITS map assigns concrete quotas to each tier. For example, tier1 receives 100 million tokens per five-hour window while tier2 receives 150 million.

When processing requests, the proxy:

1. Looks up the organization's current InferenceOrgLimitPolicy.
2. Selects the appropriate usage bucket by window_type.
3. Validates that used_amount + request_usage ≤ limit_amount.
4. Increments used_amount and records a ledger entry if the check passes.

Auditing and Reporting

Every successful inference request generates an InferenceUsageLedgerEntry and corresponding inference_usage_ledger_bucket_charges record. The reporting layer in ee/apps/inference/src/inference-reporting.ts aggregates these immutable entries to produce:

• Per-organization usage dashboards displayed in the Den web UI.
• Billing events for the OpenWork Marketplace subscription.

Because the ledger is append-only, administrators can back-fill or audit historic usage at any time without risk of data mutation.

Summary

• OpenWork's inference provisioning relies on a Den DB schema with six specialized tables tracking keys, policies, usage buckets, and audit trails.
• Model configuration in packages/types/src/den/inference.ts supports usage factors for differential pricing of expensive models.
• The Org Inference API at /v1/inference enables administrators to toggle services and upgrade tiers while enforcing subscription validation.
• The stateless proxy architecture supports horizontal scaling while maintaining consistent quota enforcement through shared database state.
• Rolling window limits with anchored or activity-based reset strategies provide flexible usage control across five-hour, weekly, and monthly periods.

Frequently Asked Questions

How does OpenWork enforce usage limits when scaling the inference proxy horizontally?

OpenWork stores all quota state in the shared Den DB rather than in-memory. When the horizontally scaled proxy processes a request, each replica queries the same inference_org_usage_buckets table to check used_amount against limit_amount. If the request would exceed the quota, the proxy returns a 429 Too Many Requests error immediately, ensuring consistent enforcement regardless of which pod handles the traffic.

What is the difference between anchored and activity-based reset strategies?

Anchored reset strategies start windows at fixed calendar epochs, such as every Monday for weekly limits or the first of the month for monthly quotas. Activity-based strategies slide the window forward from the timestamp of the first request that triggered the bucket, creating a rolling window that moves with the organization's actual usage patterns rather than calendar dates.

How can administrators enable inference provisioning for an organization?

Administrators send a PATCH request to /v1/inference with an enabled: true flag and optional tier specification. The system validates admin permissions, checks for an active subscription via organizationHasActiveInferenceSubscription, and verifies the presence of upstream provider keys before activating the service. The implementation resides in ee/apps/den-api/src/routes/org/inference.ts.

What happens when an organization exceeds its inference quota?

When the proxy detects that used_amount + request_usage would exceed the limit_amount defined in the organization's InferenceOrgLimitPolicy, it rejects the request with a 429 Too Many Requests status code. This hard stop prevents any additional usage until the current window expires and the bucket resets, or until the administrator upgrades to a higher tier with increased limits.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}