Advanced Features of Macro: A Deep Dive into Its Microservices Architecture and Capabilities

Macro is a Rust-based, cloud-native microservices platform with 80+ modular crates that combine document storage, AI-driven cognition, search, real-time messaging, and serverless processing into a single scalable workspace.

Macro, developed by macro-inc, is designed for teams building modern, cloud-first applications that demand high performance and flexibility. This guide explores the advanced features of Macro that make it a compelling choice for developers who need type-safe database interactions, extensible entity relationships, and event-driven background processing—all backed by a unified authentication and notification layer.

---

Modular Microservices Architecture with 80+ Crates

The foundation of Macro's advanced capabilities lies in its Cargo workspace architecture, which organizes the entire system into over 80 autonomous crates. Each crate represents a focused microservice under the crates/ directory, following a consistent Axum-based HTTP API pattern.

Key services include:

• Document Storage (crates/document-storage-service) — CRUD operations for files and metadata
• Document Cognition (crates/document-cognition-service) — Text extraction and AI-driven analysis
• Search Service (crates/search_service) — Full-text indexing powered by OpenSearch
• Static File Service (crates/static_file_service) — Optimized asset delivery
• Email Service (crates/email_service) — Inbound mail parsing, threading, and delivery
• Notification Service (crates/notification_service) — User-level push notifications
• Authentication Service (crates/authentication_service) — JWT-based secure authentication
• Connection Gateway (crates/connection_gateway) — WebSocket gateway for real-time updates

This modularity enables independent scaling, testing, and deployment of each component. The consistent Axum framework across services reduces cognitive overhead when contributing to multiple crates.

As documented in CLAUDE.md, this architecture supports both monorepo collaboration and independent service evolution.

---

Compile-Time SQL Safety with SQLx and Offline Mode

Macro leverages SQLx to enforce compile-time-checked PostgreSQL queries, eliminating an entire class of runtime database errors. The advanced feature of offline mode (SQLX_OFFLINE=true) allows builds to succeed without a live database connection, caching query metadata in .sqlx/ files.

Regenerate the cache after schema changes with:

just prepare_db

This workflow guarantees that every query remains synchronized with the actual database schema. The just prepare_db command introspects the database and updates the offline cache, which is then committed to version control for reproducible builds.

This approach is defined in CLAUDE.md under the SQLx workflow section.

---

Extensible Entity Mention System for Cross-Resource Linking

Originally limited to message mentions, Macro's generic EntityMention model now links **any source entity to any target entity—**a document can mention another document, a task can reference a file, or any combination thereof.

The migration involved:

1. Adding source_entity_type and source_entity_id columns to replace hardcoded assumptions
2. Migrating the legacy message_mentions table to the unified entity_mentions table
3. Updating all client code to use the type-agnostic API

The result is a future-proof linking mechanism that powers cross-document references, task tagging, and collaborative workflows without schema changes.

Create a generic entity mention using the macro_db_client crate:

use macro_db_client::entity_mentions::create_entity_mention;

/// Links a document (source) to another document (target).
async fn link_documents(
    db: &sqlx::Pool<sqlx::Postgres>,
    source_doc_id: i64,
    target_doc_id: i64,
) -> Result<(), sqlx::Error> {
    create_entity_mention(
        db,
        "Document",      // source_entity_type
        source_doc_id,   // source_entity_id
        "Document",      // target_entity_type
        target_doc_id,   // target_entity_id
    )
    .await
}

The implementation resides in crates/macro_db_client/src/entity_mentions.rs. This advanced feature case study is detailed in CLAUDE.md.

---

Serverless Background Processing with AWS Lambda

Macro offloads heavy or asynchronous work to AWS Lambda functions, enabling cost-effective scaling without maintaining persistent infrastructure. Lambda crates are compiled separately via just build_lambdas.

Primary Lambda workloads include:

• Document conversion — PDF-to-text extraction and DOCX unzipping via build_document_text_extractor
• Background indexing — Pushing new content into OpenSearch for searchability
• Deletion and cleanup — Polling SQS queues for asynchronous resource cleanup

Uploading a document triggers this pipeline automatically:

curl -X POST https://api.macro.dev/v1/documents \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@/path/to/report.pdf"

The document-storage-service stores the raw file in S3, writes metadata, and enqueues the document_text_extractor Lambda. This decouples API response latency from processing complexity.

The upload handlers are implemented in crates/document-storage-service/src/handlers.rs.

---

Integrated Communication Stack: Real-Time, Email, and Push

Macro's communication layer unifies multiple channels through a shared event bus (SQS) and Redis caching for low-latency state retrieval.

Components include:

• WebSocket gateway (connection_gateway) — Sub-second real-time chat and notifications
• Email service — Inbound parsing with thread maintenance and notification emission
• Notification service — Multi-channel push via WebSocket or external providers

Publish real-time notifications to specific users:

use connection_gateway::gateway::Gateway;

async fn notify_user(
    user_id: i64,
    payload: Notification,
) -> Result<(), anyhow::Error> {
    let gateway = Gateway::connect(env::gateway_url()).await?;
    gateway.send_to_user(user_id, payload).await
}

The gateway implementation is in crates/connection_gateway/src/gateway.rs. This architecture ensures that whether a user is connected via WebSocket or receives an email, the underlying event is processed once and propagated appropriately.

---

Centralized Configuration with macro_env_var

Macro prevents the anti-pattern of scattered std::env::var calls through the macro_env_var crate. All environment variables are declared and accessed through this centralized crate, ensuring:

• Consistent naming across services
• Compile-time visibility of required configuration
• Single point for secret handling integration with Doppler

This advanced feature promotes security and operational clarity across the 80-crate workspace.

---

Developer Tooling and Quality Enforcement

Macro ships with comprehensive tooling to maintain velocity at scale:

• Justfile commands — just build, just test, just setup_macrodb, and just build_lambdas cover the full development lifecycle
• Per-service SQL migrations — Isolated schema evolution prevents cross-service coupling
• Strict quality gates — #![deny(missing_docs)], Clippy, and rustfmt enforcement

Run the full test suite:

just test

These conventions are established in CLAUDE.md and reinforced through AGENTS.md for contributor onboarding.

---

Search Integration with OpenSearch

Macro's search capabilities are built on OpenSearch, wrapped by a typed client for ergonomic Rust usage:

use search_service::client::SearchClient;

async fn search_documents(query: &str) -> Result<Vec<Document>, anyhow::Error> {
    let client = SearchClient::new(env::search_endpoint());
    client.search_documents(query).await
}

The SearchClient in crates/search_service/src/client.rs handles query construction, result deserialization, and error mapping. This abstraction allows services to add search functionality without managing OpenSearch client complexity directly.

---

Summary

• 80+ crate workspace enables independent service scaling and consistent Axum-based APIs
• SQLx offline mode provides compile-time query safety with reproducible builds
• Generic EntityMention system creates flexible, type-agnostic cross-resource linking
• AWS Lambda integration handles heavy processing without impacting API latency
• Unified communication stack combines WebSocket, email, and push through SQS and Redis
• macro_env_var crate centralizes configuration and secret management
• Justfile and quality tooling maintain development velocity across a large codebase

---

Frequently Asked Questions

What programming language is Macro built in?

Macro is built entirely in Rust, leveraging its performance, memory safety, and async runtime capabilities. The codebase uses the Tokio ecosystem with Axum for HTTP services and SQLx for database interaction.

How does Macro handle database schema changes?

Macro uses per-service SQL migrations managed through SQLx. After modifying schemas, developers run just prepare_db to regenerate the offline query cache, ensuring all queries remain valid at compile time.

Can Macro run without AWS services?

While Macro is optimized for AWS (Lambda, SQS, S3, OpenSearch), the modular crate architecture allows replacing specific implementations. The core services in crates/ are cloud-agnostic, though production deployments currently target AWS infrastructure.

What makes Macro's entity mention system different from standard tagging?

Macro's generic EntityMention model decouples the linking mechanism from specific entity types. Unlike traditional tagging that assumes a fixed target type (e.g., tags on posts), Macro's system can relate any source entity type to any target entity type without schema modifications, enabling use cases like document-to-document references and task-to-file associations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}