How the Grok Gateway Performs Dynamic Model Discovery Per Account

The Grok gateway dynamically discovers models on a per-account basis by generating "discovered" model routes when an account first requests a model not present in the catalog, using a partial unique index to ensure idempotency.

The chenyme/grok2api repository implements a sophisticated gateway that automatically detects available AI models for each connected account. This dynamic model discovery per account mechanism ensures that users see only the models accessible through their specific provider credentials, without requiring manual configuration. The system persists discovered routes in a relational database while maintaining strict isolation between different accounts and providers.

The Model Storage Schema

The foundation of dynamic discovery lies in the model_routes table defined in backend/internal/infra/persistence/relational/schema.go. The schema declares an origin column that accepts three values: catalog, discovered, or manual.

A partial unique index on (public_id, capability) restricts uniqueness to rows where the origin is either catalog or discovered. This constraint guarantees that discovery is idempotent per public-ID/capability combination, preventing duplicate entries when multiple sync operations occur simultaneously.

// schema.go lines 68-71 - Partial unique index definition
// CREATE UNIQUE INDEX idx_model_routes_unique_active 
// ON model_routes (public_id, capability) 
// WHERE origin IN ('catalog', 'discovered');

How Discovery Is Triggered

Discovery initiates through the Sync endpoint registered in backend/internal/transport/http/model/handler.go. When a client sends a POST request to /v1/models/sync, the handler invokes the gateway service to refresh model information across all accounts.

# Trigger a full sync to create discovered routes for missing models

curl -X POST https://api.example.com/v1/models/sync \
  -H "Authorization: Bearer $TOKEN"

Because the service iterates over each account's capability set during synchronization, any missing model for an account materializes as a discovered route automatically.

The Discovery Logic

When the system encounters a model request without an existing route, the discoveredRouteDefaults helper function in backend/internal/infra/persistence/relational/model_repository.go (lines 728-735) executes. This function creates a deterministic local ID and assigns the origin field the value discovered.

The source code explicitly notes that "Concurrent discovery is guarded by the managed-route partial unique index," ensuring thread-safe operations when multiple requests target the same model simultaneously.

// model_repository.go - Discovery defaults implementation
func discoveredRouteDefaults(provider account.Provider, upstreamModel string) (string, model.Capability) {
    // Compute deterministic local ID (e.g., "Build/grok-4.5")
    localID := fmt.Sprintf("%s/%s", provider.String(), upstreamModel)
    capability := model.CapabilityResponses // Default capability
    return localID, capability
}

Per-Account Isolation

The gateway maintains strict per-account isolation by computing the public_id using the account's provider (e.g., Console, Build, or Web) combined with the upstream model name. This architecture ensures that the same upstream model can exist as separate discovered routes under different provider prefixes.

The OriginDiscovered constant defined in backend/internal/domain/model/model.go (line 41) marks these routes distinctly from manually configured or catalog entries.

// Example discovered route returned by the API
{
  "id": 12345,
  "public_id": "Build/grok-4.5",
  "provider": "grok_build",
  "upstream_model": "grok-4.5",
  "capability": "responses",
  "origin": "discovered",
  "enabled": true
}

Rediscovery and Cleanup

The system supports rediscovery after deletion. When a discovered route is removed via batch delete operations, the repository allows it to be rediscovered on the next sync cycle. The partial unique index logic ensures that deletion does not permanently hide a model from an account.

Tests in backend/internal/application/model/service_test.go verify this behavior through TestBatchDeleteModelRoutesAllowsRediscovery, confirming that the discovery mechanism remains robust across account lifecycle events.

Summary

• The Grok gateway uses a partial unique index on (public_id, capability) to ensure idempotent model discovery per account.
• Discovery triggers via the POST /v1/models/sync endpoint, which iterates over each account's capabilities to materialize missing routes.
• The discoveredRouteDefaults function in model_repository.go creates deterministic local IDs with origin set to discovered.
• Per-account isolation is maintained by prefixing the public_id with the account provider (Console, Build, or Web).
• Deleted discovered routes can be rediscovered on subsequent sync operations, ensuring model catalogs remain current.

Frequently Asked Questions

How does the gateway prevent duplicate discovered models for the same account?

The database schema implements a partial unique index in schema.go that enforces uniqueness on the combination of public_id and capability only for rows where the origin is catalog or discovered. This constraint prevents the creation of duplicate entries when concurrent sync operations occur, effectively making the discovery process idempotent.

What happens when a discovered model is deleted?

When a discovered route is removed through batch delete operations, the repository allows it to be rediscovered during the next sync cycle. Tests such as TestBatchDeleteModelRoutesAllowsRediscovery verify that the deletion does not permanently prevent the model from appearing in the account's catalog, as the unique index constraint is released upon deletion.

How does the system distinguish between manually added and discovered models?

The origin column in the model_routes table tracks the provenance of each entry. The constant OriginDiscovered defined in model.go marks automatically detected routes, while manual indicates user-configured entries and catalog represents system-defined models. This distinction enables different handling logic for discovered versus manually managed routes.

Can the same upstream model exist for different accounts?

Yes, the gateway maintains per-account isolation by computing the public_id using the account's provider prefix (such as Build/, Console/, or Web/) combined with the upstream model name. This ensures that the same upstream model (e.g., grok-4.5) can be discovered as separate entries for different accounts or provider types without collision.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}