How Webhooks Notify External Systems of Events in Macro: A Complete Technical Guide

Macro notifies external systems through an asynchronous pipeline that normalizes domain events into HTTP POST requests, queues them via SQS, and delivers them via a worker that signs each payload with HMAC-SHA256.

Macro's webhook notification system decouples internal domain events from external HTTP callbacks using a trait-based pipeline architecture. When events occur—such as document creation in the Document Storage Service—they transform into signed, queued messages that workers deliver to registered endpoints. This implementation, found in the macro-inc/macro repository's webhook crate, ensures reliable, retry-aware delivery to any external system.

The Webhook Notification Pipeline

Macro uses a six-stage pipeline to turn internal events into HTTP calls that external systems consume:

1. Event Normalisation – Domain events become NormalizedWebhookEvent instances capturing event metadata and broker payloads.
2. Queue Message Creation – The system wraps normalized events into WebhookEventQueueMessage structs containing the target webhook ID.
3. Enqueue – A WebhookEventEnqueuer implementation persists messages to an asynchronous queue (typically SQS).
4. Worker Consumption – A WebhookEventWorker pulls messages, resolves webhook configuration, and delegates to a delivery client.
5. HTTP Delivery – The WebhookDeliveryClient builds signed POST requests with Macro-specific headers.
6. Result Recording – Delivery outcomes are persisted via WebhookDeliveryRepository, enabling retry logic and auditability.

All components are defined by traits in the webhook crate, allowing pluggable implementations for different queue backends, HTTP clients, and storage engines.

Event Normalization and Enqueueing

When a domain event fires—such as a new document stored in services/document_storage_service—the ingestion layer creates a NormalizedWebhookEvent that captures the event ID, name, entity type, and original broker payload. This structure is defined in crates/webhook/src/domain/models.rs.

The ingestion service then constructs a WebhookEventQueueMessage::new(webhook_id, event), which versions the payload for queue compatibility. This message combines the normalized event with the specific webhook ID stored in the database.

The WebhookEventEnqueuer trait handles persistence. The SQS implementation in crates/webhook/src/outbound/sqs_queue.rs provides SqsWebhookQueue::enqueue(message), which stores the message for asynchronous processing:

// Normalise a broker event
let normalized = NormalizedWebhookEvent {
    event_id: "evt_123".into(),
    schema_version: 1,
    event_name: "document.created".into(),
    entity_type: "document".into(),
    entity_id: doc.id.clone(),
    ordering_key: "doc_123".into(),
    occurred_at: chrono::Utc::now(),
    broker_envelope: serde_json::json!({ "doc_id": doc.id, "title": doc.title }),
};

// Wrap for the queue
let queue_msg = WebhookEventQueueMessage::new("wh_abc".into(), normalized);

// Enqueue via SQS
let enqueuer = SqsWebhookQueue::new(sqs_client, queue_url);
enqueuer.enqueue(queue_msg).await?;

Asynchronous Delivery Architecture

Once events are queued, the delivery process becomes the responsibility of the worker and delivery client components.

The WebhookEventWorker Implementation

The WebhookEventWorker defined in crates/webhook/src/inbound/worker.rs continuously polls the queue via process_message. For each WebhookEventQueueMessage, it:

• Deserializes the queue payload
• Looks up the current webhook configuration (endpoint URL, custom headers, signing secret)
• Calls WebhookDeliveryClient::deliver(&webhook, &event)

The worker runs as a background task in services like the Document Storage Service, instantiated in services/document_storage_service/src/main.rs.

HTTP Request Construction and Signing

The actual HTTP delivery occurs in crates/webhook/src/outbound/http_delivery.rs within the delivery client implementation. The client builds requests through two key functions:

• delivery_headers – Assembles required Macro headers (x-macro-event, x-macro-event-id, x-macro-timestamp), merges custom user-defined headers, and calculates the signature header.
• delivery_request – Creates the reqwest::Request with the JSON-encoded broker envelope.

The signature_header function generates an HMAC-SHA256 signature using the webhook's signing secret, timestamp, and request body. This signature appears in the x-macro-signature header, allowing external systems to verify payload authenticity.

The request is sent via reqwest::Client, after which classify_http_status categorizes the response into a WebhookHttpOutcome variant: Success, RetryableFailure, or PermanentFailure.

Delivery Outcomes and Retry Logic

After attempting delivery, the system persists the result through the WebhookDeliveryRepository trait. The PostgreSQL implementation in crates/webhook/src/outbound/pg_delivery_repository.rs records the outcome, updates delivery status, and schedules retries for transient failures.

This repository pattern allows the worker to classify responses and implement backoff strategies without blocking the main application flow. External systems can rely on eventual delivery even during temporary network partitions or endpoint downtime.

Implementation Example

Below is a complete illustration showing how a service enqueues events and how the worker processes them:

// Service side: Enqueue a document creation event
let normalized = NormalizedWebhookEvent {
    event_id: "evt_123".into(),
    schema_version: 1,
    event_name: "document.created".into(),
    entity_type: "document".into(),
    entity_id: doc.id.clone(),
    ordering_key: "doc_123".into(),
    occurred_at: chrono::Utc::now(),
    broker_envelope: serde_json::json!({ "doc_id": doc.id, "title": doc.title }),
};

let queue_msg = WebhookEventQueueMessage::new("wh_abc".into(), normalized);
let enqueuer = SqsWebhookQueue::new(sqs_client, queue_url);
enqueuer.enqueue(queue_msg).await?;

// Worker side: Background delivery task
let queue = SqsWebhookQueue::new(sqs_client, queue_url);
let delivery = ReqwestWebhookDeliveryClient::new();
let worker = WebhookEventWorker::new(queue, delivery);
worker.run().await; // Continuously processes messages

The WebhookEventWorker pulls messages, resolves configuration via the repository layer, and invokes delivery_client.deliver(), which internally handles request construction, signing, and transmission.

Key Source Files and Data Models

Understanding the notification flow requires familiarity with these specific files:

• crates/webhook/src/domain/models.rs – Defines NormalizedWebhookEvent, WebhookEventQueueMessage, and delivery status enums.
• crates/webhook/src/domain/ports.rs – Trait definitions for WebhookEventEnqueuer, queue access, and delivery clients.
• crates/webhook/src/outbound/sqs_queue.rs – SQS implementation of the queue traits.
• crates/webhook/src/outbound/http_delivery.rs – HTTP delivery logic, header preparation, HMAC-SHA256 signing, and response classification.
• crates/webhook/src/inbound/worker.rs – The asynchronous worker that orchestrates message consumption and delivery.
• services/document_storage_service/src/main.rs – Example service wiring the ingestion pipeline.
• services/email_service/src/api/gmail/webhook.rs – Reference for inbound webhook endpoints that external systems call.

Summary

Macro's webhook notification system provides reliable, asynchronous delivery of domain events to external systems through these key mechanisms:

• Event normalization converts broker messages into standardized NormalizedWebhookEvent structures with versioning support.
• Queue-based architecture decouples event generation from delivery using WebhookEventQueueMessage and pluggable enqueuers like SQS.
• Signed delivery ensures security via HMAC-SHA256 signatures in the x-macro-signature header, generated in http_delivery.rs.
• Worker-driven processing handles HTTP transmission, response classification, and outcome persistence through the WebhookEventWorker.
• Retry logic at the repository layer allows recovery from transient failures without data loss.

Frequently Asked Questions

What triggers a webhook notification in Macro?

Domain events within Macro services—such as document creation in the Document Storage Service—trigger the webhook pipeline. When these events occur, the ingestion layer creates a NormalizedWebhookEvent that flows through the queue to registered webhook endpoints. The specific event types depend on which services instantiate the webhook ingestion infrastructure in their main.rs files.

How does Macro secure webhook deliveries to external systems?

Macro secures deliveries through HMAC-SHA256 request signing. The signature_header function in crates/webhook/src/outbound/http_delivery.rs generates a signature using the webhook's signing secret, timestamp, and request body. This signature appears in the x-macro-signature header, while x-macro-timestamp prevents replay attacks. External systems can verify these signatures to authenticate that payloads originated from Macro.

What happens when a webhook delivery fails?

The classify_http_status function categorizes HTTP responses into Success, RetryableFailure, or PermanentFailure variants of WebhookHttpOutcome. Retryable failures (such as 5xx errors or timeouts) are recorded in the WebhookDeliveryRepository for later retry, while permanent failures (such as 4xx client errors) are logged without retry attempts. This logic ensures external systems receive eventual delivery during transient outages without wasting resources on permanently broken endpoints.

Where is the webhook worker implementation located?

The core worker logic resides in crates/webhook/src/inbound/worker.rs, which defines the WebhookEventWorker responsible for polling queues and orchestrating delivery. The SQS queue implementation is in crates/webhook/src/outbound/sqs_queue.rs, while the HTTP delivery client is in crates/webhook/src/outbound/http_delivery.rs. Services instantiate these components in their respective main.rs files, such as services/document_storage_service/src/main.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}