How to Set Up and Configure Egress Quality Guard for Active Node Monitoring in Grok2API

Egress Quality Guard is a built-in sidecar that continuously probes egress nodes to verify latency, throughput, and output quality, automatically quarantining underperforming nodes without manual intervention.

The Egress Quality Guard system in the chenyme/grok2api repository provides autonomous health monitoring for your egress node fleet. By streaming test completions through each node and analyzing metrics like tokens-per-second and error rates, it ensures only healthy nodes serve production traffic. This guide covers the complete setup process, from initial configuration to runtime management, based on the actual implementation in the source code.

Understanding the Quality Guard Architecture

Egress Quality Guard operates through several coordinated components that handle authentication, probing, and state management.

Component	Role	Source File
Bootstrap Generator	Creates a sidecar bootstrap file containing a deterministic internal token derived from the HTTP server JWT secret. This token authenticates probe requests.	backend/internal/infra/qualityguard/bootstrap.go
HTTP Handler	Exposes /egress-quality-guard/config for runtime updates and /egress-nodes/:id/quality-test for executing probes.	backend/internal/transport/http/egress/handler.go
Probe Service	Builds streaming LLM requests, extracts timing metrics, and returns structured QualityProbeResult with firstTokenMs, durationMs, and outputTokensPerSecond.	backend/internal/application/gateway/quality_probe.go
Client-Key Identity	Manages a reserved internal client key prefixed with quality-guard-internal to isolate probe traffic from production API keys.	backend/internal/application/clientkey/service.go
Configuration Model	Defines the QualityGuardConfig struct with fields like Mode, ActiveIntervalSeconds, SoftTPS, and HardTPS.	backend/internal/infra/config/config.go
State Persistence	Maintains a JSON state.json file tracking node health statistics, quarantine status, and recent events.	qualityGuardState struct in handler.go

Enabling Quality Guard in Server Configuration

Begin by enabling the guard in your server configuration file. The system reads these values during initialization in backend/internal/infra/config/config.go.

# config.yaml

qualityGuard:
  enabled: true
  model: "grok-build-v1"
  activeInterval: "5m"
  passivePollInterval: "30s"
  softTPS: 150
  hardTPS: 300
  consecutiveSoft: 3
  consecutiveErrors: 5
  quarantineSeconds: 600
  minHealthyNodes: 2

The QualityGuardConfig struct validates these parameters at startup. Ensure minHealthyNodes is set lower than your total node count to prevent service interruption during quarantine events.

Bootstrapping the Sidecar

During server startup, the application calls qualityguard.Prepare to generate authentication credentials for the sidecar. This function writes a bootstrap file and returns an internal token.

// Typically implemented in backend/internal/app/application.go or main.go
bootstrapPath := "/var/lib/grok2api/quality-guard/bootstrap.json"
token, err := qualityguard.Prepare(bootstrapPath, cfg.QualityGuard, jwtSecret)
if err != nil {
    log.Fatalf("failed to prepare quality guard: %v", err)
}
os.Setenv("QUALITY_GUARD_TOKEN", token) // Pass to sidecar

The Prepare function in backend/internal/infra/qualityguard/bootstrap.go derives the token deterministically from your JWT secret, ensuring the sidecar can authenticate without database access.

Deploying the Sidecar and Executing Probes

The sidecar reads the bootstrap file to extract the InternalToken, then uses it in the Authorization: Bearer <token> header when calling probe endpoints.

To manually test a node probe from the sidecar context:

#!/usr/bin/env bash
TOKEN=$(cat /var/lib/grok2api/quality-guard/bootstrap.json | jq -r .internal_token)
NODE_ID=42

curl -s -X POST "http://localhost:8080/egress-nodes/${NODE_ID}/quality-test" \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
          "clientKeyId":"0",
          "model":"grok-build-v1",
          "prompt":"Write exactly 16 numbered lines about reliable distributed systems. Each line must be one complete English sentence, with no markdown heading. The final line must end with the exact marker QUALITY_OK.",
          "expected":"QUALITY_OK",
          "maxOutputTokens":1024
        }'

The testQualityGuardNode handler (line 61 in backend/internal/transport/http/egress/handler.go) validates the request and forwards it to h.service.ProbeQuality(). The probe service in backend/internal/application/gateway/quality_probe.go streams the completion and calculates performance metrics.

Configuring Runtime Guard Parameters

Adjust guard thresholds without restarting the server using the configuration endpoint. The handler validates updates against the schema defined in qualityGuardConfigRequest.validate (lines 69-92 of handler.go).

curl -X PUT http://localhost:8080/egress-quality-guard/config \
     -H "Authorization: Bearer $ADMIN_TOKEN" \
     -H "Content-Type: application/json" \
     -d '{
          "mode":"active",
          "activeIntervalSeconds":300,
          "passivePollSeconds":30,
          "softTPS":200,
          "hardTPS":400,
          "consecutiveSoft":2,
          "consecutiveErrors":4,
          "quarantineSeconds":900,
          "minHealthyNodes":3
        }'

Available modes include active for continuous probing and passive configurations for event-driven checks.

Monitoring Guard State and Node Health

The current system state is exposed via GET /egress-quality-guard. The handler reads state.json, verifies it is under 8 MiB, and unmarshals it into the qualityGuardState struct.

curl -s http://localhost:8080/egress-quality-guard \
     -H "Authorization: Bearer $ADMIN_TOKEN" | jq .

Key fields to monitor include:

• nodes[nodeID].DisabledByGuard — Boolean indicating quarantine status
• nodes[nodeID].QuarantinedUntil — Unix timestamp for automatic restoration
• statistics.active.soft — Count of soft threshold violations
• recentEvents — Chronological list of quarantine/restoration actions

When a node exceeds consecutiveSoft threshold violations or consecutiveErrors failures, the guard sets DisabledByGuard: true and schedules restoration based on quarantineSeconds.

Summary

• Egress Quality Guard runs as an authenticated sidecar that probes nodes via backend/internal/transport/http/egress/handler.go
• Bootstrap the system using qualityguard.Prepare() to generate the internal token required for probe authentication
• Configure thresholds like softTPS, hardTPS, and consecutiveErrors in config.yaml or via the /egress-quality-guard/config HTTP endpoint
• The probe service in backend/internal/application/gateway/quality_probe.go measures firstTokenMs and outputTokensPerSecond to determine node health
• Failed nodes are automatically quarantined with DisabledByGuard: true and excluded from the egress pool until the quarantine period expires

Frequently Asked Questions

How does the Quality Guard authenticate its probe requests?

The sidecar uses a deterministic internal token derived from the server's JWT secret. During startup, qualityguard.Prepare() in backend/internal/infra/qualityguard/bootstrap.go generates this token and writes it to a bootstrap JSON file. The sidecar reads this file and includes the token in the Authorization: Bearer header for all probe requests to the /egress-nodes/:id/quality-test endpoint.

What metrics determine if a node gets quarantined?

The system evaluates tokens-per-second (TPS) and error rates. If a node's outputTokensPerSecond falls below softTPS for consecutiveSoft attempts, or below hardTPS immediately, it receives a strike. Accumulating consecutiveErrors failures also triggers quarantine. The QualityProbeResult struct returned by backend/internal/application/gateway/quality_probe.go contains these metrics for every probe.

Can I adjust guard thresholds without restarting the server?

Yes. Send a PUT request to /egress-quality-guard/config with an admin token. The handler in backend/internal/transport/http/egress/handler.go validates the payload against the QualityGuardConfig schema and applies changes immediately. This allows dynamic tuning of activeIntervalSeconds, TPS thresholds, and quarantine durations based on observed traffic patterns.

Where does the Quality Guard store its state?

The sidecar writes to a local state.json file containing the qualityGuardState struct, which tracks per-node status, statistics, and recent events. The HTTP handler reads this file (with a size limit of 8 MiB) to serve status requests at GET /egress-quality-guard. This file-based approach ensures the guard persists state across sidecar restarts without requiring database connectivity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}