Designing Observability with SLOs and Alerting: Automated Framework Guide

The Observability Designer skill in alirezarezvani/claude-skills automatically generates Service-Level Indicators (SLIs), Service-Level Objectives (SLOs), error-budget policies, and multi-window burn-rate alerts from JSON service definitions, outputting Prometheus-ready rules and Grafana dashboards without external dependencies.

The alirezarezvani/claude-skills repository provides a pure-Python framework for designing observability with SLOs and alerting through code. It transforms high-level service metadata—such as criticality, type, and user-facing status—into complete monitoring artifacts including golden signal coverage, burn-rate alerts, and optimized runbooks.

Architecture of the Observability Designer

The framework operates through seven distinct layers, each implemented as specific methods in the core scripts. This modular design allows teams to generate or optimize individual components without adopting the entire pipeline.

Service Definition and SLI Generation

The workflow begins in engineering/observability-designer/scripts/slo_designer.py, where SLODesigner.create_service_definition() accepts a JSON document specifying service type (api, web, etc.), criticality (critical, high, medium, low), user-facing status, and dependencies. The method SLODesigner.generate_slis() then selects appropriate indicators—availability, latency (P95/P99), error rate, and throughput—automatically adding user-facing-specific metrics when the flag is enabled.

SLO Derivation and Error Budgets

For each generated SLI, SLODesigner.generate_slos() and the helper _create_slo_from_sli() map criticality levels to concrete targets. A critical service receives a 99.99% availability target, while high criticality services typically receive 99.9%. The framework calculates allowable error budgets per time window and invokes _generate_burn_rate_alerts() to create multi-window alert configurations (5m/1h, 30m/6h) that fire when burn rates exceed configured multipliers. Severity is determined by _determine_alert_severity() based on the service's criticality and the speed of budget exhaustion.

SLA Recommendations and Dashboards

For user-facing services, generate_sla_recommendations() produces contract-ready SLAs that apply a 0.1% buffer stricter than the internal SLO. The dashboard_generator.py script (documented in docs/skills/engineering/observability-designer.md) consumes the SLO framework to emit Grafana-compatible JSON with panels for each SLI, burn-rate status, and service-level summaries.

Alert Optimization Layer

The engineering/observability-designer/scripts/alert_optimizer.py module provides AlertOptimizer.analyze_configuration(), which examines existing Prometheus alert sets to detect noisy patterns (defined in NOISY_PATTERNS), identify missing golden-signal coverage, and flag exact or semantic duplicates. It produces a risk assessment highlighting critical gaps and assigning fatigue ratings when high-severity alert density exceeds thresholds.

The SLO Design Workflow

Implementing observability with this framework follows a four-step pipeline:

1. Describe the service – Create a JSON file defining the service type, criticality tier, user-facing boolean, team ownership, and dependencies.
2. Generate the framework – Execute slo_designer.py to produce SLIs, SLO targets, error budgets, and burn-rate alert rules.
3. Export artifacts – Write the complete framework to framework.json for consumption by Terraform, Helm, or the Dashboard Generator.
4. Deploy to production – Load recording rules and burn-rate alerts into Prometheus, import dashboard JSON into Grafana, and store generated runbooks alongside on-call documentation.

Mitigating Alert Fatigue

The Alert Optimizer analyzes existing configurations to reduce operational noise. It flags high-frequency, low-threshold alerts as noisy, identifies gaps in the four golden signals (availability, latency, errors, saturation), and removes duplicate rules. By feeding the optimizer's output back into the SLO Designer, teams create a refinement loop: optimized SLO targets produce tighter, more meaningful alerts with reduced false-positive rates.

Implementation Examples

Generating a Complete SLO Framework

Define your service in a JSON file:

{
  "name": "orders_api",
  "type": "api",
  "criticality": "high",
  "user_facing": true,
  "description": "High-traffic orders API",
  "dependencies": ["postgres", "redis"],
  "team": "platform",
  "environment": "production"
}

Run the designer via CLI:

python slo_designer.py \
  --input service_definition.json \
  --output orders_api_slo_framework.json

The output contains availability and latency SLIs, 99.9% availability SLOs, multi-window burn-rate alerts (5m/1h and 30m/6h windows), and an SLA recommendation with a 0.1% buffer. This functionality is implemented in SLODesigner.generate_framework() within slo_designer.py.

Using the Python API Directly

Import the designer for programmatic control:

from slo_designer import SLODesigner

designer = SLODesigner()
svc = designer.create_service_definition(
    service_type="web",
    criticality="critical",
    user_facing=True,
    name="checkout_ui"
)

framework = designer.generate_framework(svc)
print(framework["slos"])               # view generated SLO objects

print(framework["error_budgets"])      # see burn-rate alert definitions

Optimizing Existing Alert Configurations

Analyze and refine current Prometheus rules:

python alert_optimizer.py \
  --input alerts.json \
  --output alerts_optimized.json \
  --report alert_report.html \
  --format html

The alerts_optimized.json file removes duplicates and annotates noisy thresholds, while alert_report.html provides a human-readable risk assessment. This leverages AlertOptimizer.analyze_configuration() in alert_optimizer.py.

Creating Grafana Dashboards

Transform the SLO framework into visualization:

python dashboard_generator.py \
  --input orders_api_slo_framework.json \
  --output orders_api_dashboard.json

The resulting JSON is immediately importable into Grafana and includes burn-rate alert status panels and service-level summary views as specified in the repository's SKILL.md documentation.

Summary

• The framework generates Service-Level Indicators and Service-Level Objectives automatically from JSON service definitions using slo_designer.py.
• Multi-window burn-rate alerts (5m/1h, 30m/6h) are calculated via _generate_burn_rate_alerts() to detect fast budget exhaustion without noise.
• Alert fatigue is reduced through alert_optimizer.py, which detects noisy patterns, coverage gaps, and duplicate rules in existing Prometheus configurations.
• All components are pure-Python with zero external service dependencies, outputting standards-compliant JSON for Prometheus Alertmanager and Grafana.
• The pipeline supports SLA recommendations with automatic 0.1% buffer calculations for user-facing services.

Frequently Asked Questions

What distinguishes an SLI from an SLO in this framework?

Service-Level Indicators (SLIs) are the measurable metrics—such as availability percentage or latency milliseconds—that SLODesigner.generate_slis() selects based on service type. Service-Level Objectives (SLOs) are the target values (e.g., 99.9% availability) derived from criticality tiers via generate_slos() and _create_slo_from_sli(). The SLI represents what you measure; the SLO represents the threshold that determines whether the service is healthy.

How are burn-rate alert windows calculated?

The _generate_burn_rate_alerts() method in slo_designer.py creates multi-window alert pairs—typically 5 minutes over 1 hour and 30 minutes over 6 hours—based on the error budget derived from the SLO target. These windows detect both rapid exhaustion (fast burn) and gradual degradation (slow burn), with severity determined by _determine_alert_severity() according to the service's criticality level and the speed of budget consumption.

Can the framework analyze existing Prometheus alert rules?

Yes. The AlertOptimizer.analyze_configuration() method in alert_optimizer.py ingests existing alert configurations, flags noisy patterns defined in NOISY_PATTERNS, identifies missing golden-signal coverage, and removes duplicate rules. It outputs an optimized JSON configuration and an HTML risk report, allowing teams to refactor existing setups without manual audit.

What file formats does the dashboard generator produce?

The dashboard_generator.py script emits Grafana-compatible JSON dashboard specifications. These files include pre-configured panels for each SLI, burn-rate alert status indicators, and service-level summary views, ready for direct import into Grafana via its JSON import feature or deployment through infrastructure-as-code tools like Terraform.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}