# Advanced Features of Macro: A Deep Dive into Its Microservices Architecture and Capabilities

> Explore Macro's advanced features. Discover its cloud-native microservices architecture, AI cognition, real-time messaging, and serverless processing for scalable workspaces. Learn more about macro-inc/macro.

- Repository: [Macro/macro](https://github.com/macro-inc/macro)
- Tags: deep-dive
- Published: 2026-08-14

---

**Macro is a Rust-based, cloud-native microservices platform with 80+ modular crates that combine document storage, AI-driven cognition, search, real-time messaging, and serverless processing into a single scalable workspace.**

Macro, developed by macro-inc, is designed for teams building modern, cloud-first applications that demand high performance and flexibility. This guide explores the advanced features of Macro that make it a compelling choice for developers who need type-safe database interactions, extensible entity relationships, and event-driven background processing—all backed by a unified authentication and notification layer.

---

## Modular Microservices Architecture with 80+ Crates

The foundation of Macro's advanced capabilities lies in its **Cargo workspace architecture**, which organizes the entire system into over 80 autonomous crates. Each crate represents a focused microservice under the `crates/` directory, following a consistent **Axum-based HTTP API** pattern.

Key services include:

- **Document Storage** (`crates/document-storage-service`) — CRUD operations for files and metadata
- **Document Cognition** (`crates/document-cognition-service`) — Text extraction and AI-driven analysis
- **Search Service** (`crates/search_service`) — Full-text indexing powered by OpenSearch
- **Static File Service** (`crates/static_file_service`) — Optimized asset delivery
- **Email Service** (`crates/email_service`) — Inbound mail parsing, threading, and delivery
- **Notification Service** (`crates/notification_service`) — User-level push notifications
- **Authentication Service** (`crates/authentication_service`) — JWT-based secure authentication
- **Connection Gateway** (`crates/connection_gateway`) — WebSocket gateway for real-time updates

This modularity enables independent scaling, testing, and deployment of each component. The consistent Axum framework across services reduces cognitive overhead when contributing to multiple crates.

As documented in [`CLAUDE.md`](https://github.com/macro-inc/macro/blob/main/CLAUDE.md), this architecture supports both monorepo collaboration and independent service evolution.

---

## Compile-Time SQL Safety with SQLx and Offline Mode

Macro leverages **SQLx** to enforce **compile-time-checked PostgreSQL queries**, eliminating an entire class of runtime database errors. The advanced feature of **offline mode** (`SQLX_OFFLINE=true`) allows builds to succeed without a live database connection, caching query metadata in `.sqlx/` files.

Regenerate the cache after schema changes with:

```bash
just prepare_db

```

This workflow guarantees that every query remains synchronized with the actual database schema. The `just prepare_db` command introspects the database and updates the offline cache, which is then committed to version control for reproducible builds.

This approach is defined in [`CLAUDE.md`](https://github.com/macro-inc/macro/blob/main/CLAUDE.md) under the SQLx workflow section.

---

## Extensible Entity Mention System for Cross-Resource Linking

Originally limited to message mentions, Macro's **generic `EntityMention`** model now links **any source entity to any target entity—**a document can mention another document, a task can reference a file, or any combination thereof.

The migration involved:

1. Adding `source_entity_type` and `source_entity_id` columns to replace hardcoded assumptions
2. Migrating the legacy `message_mentions` table to the unified `entity_mentions` table
3. Updating all client code to use the type-agnostic API

The result is a **future-proof linking mechanism** that powers cross-document references, task tagging, and collaborative workflows without schema changes.

Create a generic entity mention using the `macro_db_client` crate:

```rust
use macro_db_client::entity_mentions::create_entity_mention;

/// Links a document (source) to another document (target).
async fn link_documents(
    db: &sqlx::Pool<sqlx::Postgres>,
    source_doc_id: i64,
    target_doc_id: i64,
) -> Result<(), sqlx::Error> {
    create_entity_mention(
        db,
        "Document",      // source_entity_type
        source_doc_id,   // source_entity_id
        "Document",      // target_entity_type
        target_doc_id,   // target_entity_id
    )
    .await
}

```

The implementation resides in [`crates/macro_db_client/src/entity_mentions.rs`](https://github.com/macro-inc/macro/blob/main/crates/macro_db_client/src/entity_mentions.rs). This advanced feature case study is detailed in [`CLAUDE.md`](https://github.com/macro-inc/macro/blob/main/CLAUDE.md).

---

## Serverless Background Processing with AWS Lambda

Macro offloads heavy or asynchronous work to **AWS Lambda functions**, enabling cost-effective scaling without maintaining persistent infrastructure. Lambda crates are compiled separately via `just build_lambdas`.

Primary Lambda workloads include:

- **Document conversion** — PDF-to-text extraction and DOCX unzipping via `build_document_text_extractor`
- **Background indexing** — Pushing new content into OpenSearch for searchability
- **Deletion and cleanup** — Polling SQS queues for asynchronous resource cleanup

Uploading a document triggers this pipeline automatically:

```bash
curl -X POST https://api.macro.dev/v1/documents \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@/path/to/report.pdf"

```

The `document-storage-service` stores the raw file in S3, writes metadata, and enqueues the `document_text_extractor` Lambda. This decouples API response latency from processing complexity.

The upload handlers are implemented in [`crates/document-storage-service/src/handlers.rs`](https://github.com/macro-inc/macro/blob/main/crates/document-storage-service/src/handlers.rs).

---

## Integrated Communication Stack: Real-Time, Email, and Push

Macro's communication layer unifies multiple channels through a **shared event bus (SQS)** and **Redis caching** for low-latency state retrieval.

Components include:

- **WebSocket gateway** (`connection_gateway`) — Sub-second real-time chat and notifications
- **Email service** — Inbound parsing with thread maintenance and notification emission
- **Notification service** — Multi-channel push via WebSocket or external providers

Publish real-time notifications to specific users:

```rust
use connection_gateway::gateway::Gateway;

async fn notify_user(
    user_id: i64,
    payload: Notification,
) -> Result<(), anyhow::Error> {
    let gateway = Gateway::connect(env::gateway_url()).await?;
    gateway.send_to_user(user_id, payload).await
}

```

The gateway implementation is in [`crates/connection_gateway/src/gateway.rs`](https://github.com/macro-inc/macro/blob/main/crates/connection_gateway/src/gateway.rs). This architecture ensures that whether a user is connected via WebSocket or receives an email, the underlying event is processed once and propagated appropriately.

---

## Centralized Configuration with macro_env_var

Macro prevents the anti-pattern of scattered `std::env::var` calls through the **`macro_env_var` crate**. All environment variables are declared and accessed through this centralized crate, ensuring:

- Consistent naming across services
- Compile-time visibility of required configuration
- Single point for secret handling integration with Doppler

This advanced feature promotes security and operational clarity across the 80-crate workspace.

---

## Developer Tooling and Quality Enforcement

Macro ships with comprehensive tooling to maintain velocity at scale:

- **Justfile commands** — `just build`, `just test`, `just setup_macrodb`, and `just build_lambdas` cover the full development lifecycle
- **Per-service SQL migrations** — Isolated schema evolution prevents cross-service coupling
- **Strict quality gates** — `#![deny(missing_docs)]`, Clippy, and rustfmt enforcement

Run the full test suite:

```bash
just test

```

These conventions are established in [`CLAUDE.md`](https://github.com/macro-inc/macro/blob/main/CLAUDE.md) and reinforced through [`AGENTS.md`](https://github.com/macro-inc/macro/blob/main/AGENTS.md) for contributor onboarding.

---

## Search Integration with OpenSearch

Macro's search capabilities are built on **OpenSearch**, wrapped by a typed client for ergonomic Rust usage:

```rust
use search_service::client::SearchClient;

async fn search_documents(query: &str) -> Result<Vec<Document>, anyhow::Error> {
    let client = SearchClient::new(env::search_endpoint());
    client.search_documents(query).await
}

```

The `SearchClient` in [`crates/search_service/src/client.rs`](https://github.com/macro-inc/macro/blob/main/crates/search_service/src/client.rs) handles query construction, result deserialization, and error mapping. This abstraction allows services to add search functionality without managing OpenSearch client complexity directly.

---

## Summary

- **80+ crate workspace** enables independent service scaling and consistent Axum-based APIs
- **SQLx offline mode** provides compile-time query safety with reproducible builds
- **Generic EntityMention system** creates flexible, type-agnostic cross-resource linking
- **AWS Lambda integration** handles heavy processing without impacting API latency
- **Unified communication stack** combines WebSocket, email, and push through SQS and Redis
- **macro_env_var crate** centralizes configuration and secret management
- **Justfile and quality tooling** maintain development velocity across a large codebase

---

## Frequently Asked Questions

### What programming language is Macro built in?

Macro is built entirely in **Rust**, leveraging its performance, memory safety, and async runtime capabilities. The codebase uses the Tokio ecosystem with Axum for HTTP services and SQLx for database interaction.

### How does Macro handle database schema changes?

Macro uses **per-service SQL migrations** managed through SQLx. After modifying schemas, developers run `just prepare_db` to regenerate the offline query cache, ensuring all queries remain valid at compile time.

### Can Macro run without AWS services?

While Macro is optimized for AWS (Lambda, SQS, S3, OpenSearch), the modular crate architecture allows replacing specific implementations. The core services in `crates/` are cloud-agnostic, though production deployments currently target AWS infrastructure.

### What makes Macro's entity mention system different from standard tagging?

Macro's **generic EntityMention model** decouples the linking mechanism from specific entity types. Unlike traditional tagging that assumes a fixed target type (e.g., tags on posts), Macro's system can relate any source entity type to any target entity type without schema modifications, enabling use cases like document-to-document references and task-to-file associations.