Key Files for Understanding Macro Development in the Macro Repository

Understanding macro development requires navigating three architectural layers: the workspace configuration in Cargo.toml, shared infrastructure crates like macro_config and worker-rs-otel, and microservice implementations such as connection_gateway that demonstrate Axum-based patterns for real-time communication.

The Macro repository is a production-grade Rust monorepo implementing a microservices platform for document handling, real-time messaging, and AI-enabled processing. Grasping macro development involves studying how the Cargo workspace organizes over 80 crates, how shared libraries manage cross-cutting concerns like configuration and observability, and how individual services structure their business logic behind Axum routers.

Workspace and Build System Configuration

The foundation of macro development rests on the workspace manifest and custom build tooling that coordinate builds across the entire repository.

The Root Cargo.toml Workspace Manifest

The [Cargo.toml](https://github.com/macro-inc/macro/blob/main/Cargo.toml) file at the repository root defines the workspace structure, enumerating every service and library across three categories: core libraries (crates/*), tooling (tooling/*), and services (services/*). This manifest centralizes dependency management through [workspace.dependencies], ensuring consistent versions of axum, sqlx, and aws-sdk-* crates across all components.

Custom Build Tooling with xtask

The [tooling/xtask/src/main.rs](https://github.com/macro-inc/macro/blob/main/tooling/xtask/src/main.rs) file serves as the entry point for all custom build commands, enabling workflows like just build and just prepare_db. This pattern follows the Rust xtask convention for automating repository-specific tasks without external build scripts.

Core Infrastructure Crates

Macro development relies on shared libraries that abstract common concerns, allowing services to focus on business logic while maintaining consistent operational standards.

Configuration Management with macro_config

The [crates/macro_config/src/lib.rs](https://github.com/macro-inc/macro/blob/main/crates/macro_config/src/lib.rs) file provides the ConfigLoader pattern that eliminates ad-hoc environment variable access. Services use the ConfigLoader::load::<MyConfig>() method to deserialize configuration from APP_SECRETS_JSON or environment variables into strongly-typed structs deriving MacroConfig.

Observability via worker-rs-otel

Distributed tracing is standardized through the [crates/worker-rs-otel/src/lib.rs](https://github.com/macro-inc/macro/blob/main/crates/worker-rs-otel/src/lib.rs) crate, which exports an OpenTelemetry-compatible layer. Services initialize this via tracing_subscriber::registry().with(worker_rs_otel::layer::OtelLayer::new()) to emit unified logs, traces, and metrics across the platform.

Webhook Security in webhook_signature

The [crates/webhook_signature/src/lib.rs](https://github.com/macro-inc/macro/blob/main/crates/webhook_signature/src/lib.rs) crate provides utilities for validating incoming webhooks, implementing security patterns used across multiple services for external integrations.

Service Implementation Patterns

Individual microservices follow a consistent architectural pattern that separates routing, configuration, and business logic into distinct modules.

Anatomy of a Microservice

Every service in the services/ directory follows a standardized layout:

• src/main.rs – Boots the Axum router and HTTP server
• src/config.rs – Declares a #[derive(MacroConfig)] struct for service-specific settings
• src/service/ – Contains core business logic, database clients, and integration adapters

This deliberate splitting enables testing core logic without running the HTTP server, as noted in the service documentation.

Real-Time Communication in connection_gateway

The [services/connection_gateway/src/lib.rs](https://github.com/macro-inc/macro/blob/main/services/connection_gateway/src/lib.rs) file exemplifies complex service architecture, exporting core types like Tracker, Sender, Connection, Message, Stream, and MessageHandler. This WebSocket gateway demonstrates integration patterns for DynamoDB, Redis, and SQS, showing how external state stores are abstracted behind simple traits.

The corresponding [services/connection_gateway/src/main.rs](https://github.com/macro-inc/macro/blob/main/services/connection_gateway/src/main.rs) illustrates server bootstrap, assembling the Axum router with OpenAPI documentation and mounting the service endpoints.

HTTP API Services

For traditional request-response patterns, [services/email_service/src/main.rs](https://github.com/macro-inc/macro/blob/main/services/email_service/src/main.rs) demonstrates AWS SES integration with proper error mapping, while [services/document_storage_service/src/main.rs](https://github.com/macro-inc/macro/blob/main/services/document_storage_service/src/main.rs) shows database access via sqlx, multipart uploads to S3, and OpenAPI generation. The [services/worker_trigger/src/main.rs](https://github.com/macro-inc/macro/blob/main/services/worker_trigger/src/main.rs) file illustrates background worker patterns for triggering AWS ECS tasks.

Practical Implementation Examples

The following patterns demonstrate how macro development combines these architectural elements in practice.

Loading Service Configuration

Services initialize using the declarative config pattern:

use macro_config::ConfigLoader;
use macro_config_derive::MacroConfig;

#[derive(Debug, MacroConfig)]
struct MyServiceConfig {
    #[from_ref]
    database_url: String,
    api_key: Option<String>,
    #[serde(rename = "PORT")]
    port: u16,
}

fn main() -> Result<(), macro_config::MacroConfigError> {
    let cfg: MyServiceConfig = ConfigLoader::load()?;
    println!("Running on port {}", cfg.port);
    Ok(())
}

This approach, implemented in crates/macro_config/src/lib.rs, centralizes secret handling and type safety.

Bootstrapping an Axum Service

The connection gateway demonstrates server initialization:

use axum::{routing::get, Router};
use macro_config::load;
use service::Tracker;

#[tokio::main]
async fn main() {
    let cfg: connection_gateway::Config = load().expect("config missing");
    
    let router = Router::new()
        .route("/", get(root_handler))
        .merge(connection_gateway::api::router());
    
    axum::Server::bind(&format!("0.0.0.0:{}", cfg.port).parse().unwrap())
        .serve(router.into_make_service())
        .await
        .unwrap();
}

Adding OpenTelemetry Tracing

Observability initialization follows this pattern from worker-rs-otel:

use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

fn init_tracing() {
    let otel_layer = worker_rs_otel::layer::OtelLayer::new();
    tracing_subscriber::registry()
        .with(otel_layer)
        .with(tracing_subscriber::fmt::layer())
        .init();
}

Summary

• Workspace Configuration: The root Cargo.toml and tooling/xtask/src/main.rs define the build system and task automation for macro development.
• Shared Infrastructure: Crates like macro_config, worker-rs-otel, and webhook_signature provide cross-cutting concerns for configuration, tracing, and security.
• Service Structure: Microservices follow a three-layer pattern with main.rs for routing, config.rs for typed settings, and service/ modules for business logic.
• Integration Patterns: The connection_gateway service demonstrates how to integrate Axum with WebSocket connections, DynamoDB, Redis, and SQS.
• Observability: All services implement OpenTelemetry tracing through the standardized worker-rs-otel layer for unified monitoring.

Frequently Asked Questions

What is the entry point for understanding macro development?

Start with the [Cargo.toml](https://github.com/macro-inc/macro/blob/main/Cargo.toml) workspace manifest to understand the crate structure, then examine [tooling/xtask/src/main.rs](https://github.com/macro-inc/macro/blob/main/tooling/xtask/src/main.rs) for build workflows. These files reveal how the repository coordinates over 80 crates and manages custom build commands.

How does configuration management work in macro development?

Configuration uses the macro_config crate, where services define structs deriving MacroConfig and load them via ConfigLoader::load(). This reads from the APP_SECRETS_JSON environment variable or individual environment variables, generating strongly-typed configs that eliminate runtime parsing errors.

Which service best exemplifies the macro development patterns?

The [services/connection_gateway/src/lib.rs](https://github.com/macro-inc/macro/blob/main/services/connection_gateway/src/lib.rs) file serves as the canonical example, demonstrating the three-layer architecture, WebSocket handling, and integration with AWS services like DynamoDB and SQS. It exports core types including Tracker, Sender, and Connection while maintaining separation between routing and business logic.

How do services handle observability and tracing?

All services initialize an OpenTelemetry layer from worker-rs-otel via tracing_subscriber::registry().with(OtelLayer::new()). This standardizes logging, metrics, and distributed tracing across the entire platform without requiring individual services to implement telemetry protocols manually.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}