# What Is Macro by Macro-Inc? A Deep Dive Into the Rust-Based Document Platform

> Discover Macro by macro-inc, a Rust-based platform for cloud-native document management. Explore its crates for storage, search, and workflow automation.

- Repository: [Macro/macro](https://github.com/macro-inc/macro)
- Tags: deep-dive
- Published: 2026-08-14

---

**Macro by macro-inc is an open-source, cloud-native microservices platform built in Rust that handles document storage, processing, search, communication, and email workflows through 80+ specialized crates.**

This production-grade system provides a complete foundation for applications requiring robust document management, real-time collaboration, and integrated messaging. The entire codebase lives in the `macro-inc/macro` GitHub repository and is organized as a Cargo workspace with services for everything from AI-powered editing to WebSocket-driven notifications.

## Core Architecture of Macro by Macro-Inc

The Macro platform follows a **modular microservices design** where each responsibility is isolated into its own crate. This structure enables independent scaling, deployment, and maintenance of individual components.

### Workspace Organization

All services reside under the `crates/` directory and are coordinated through the top-level [`Cargo.toml`](https://github.com/macro-inc/macro/blob/main/Cargo.toml) workspace definition. The [`AGENTS.md`](https://github.com/macro-inc/macro/blob/main/AGENTS.md) file documents the conventions and service responsibilities for developers contributing to the codebase.

Key service categories include:

- **HTTP API services** – `document-storage-service`, `search_processing_service`, `email_service`
- **Background workers** – `worker_trigger`, `ai_editing_worker`, `document_text_extractor`
- **Shared libraries** – `macro_db_client`, `macro_queues`, `worker-rs-otel`

### Data Persistence Layer

Macro uses **PostgreSQL schemas** collectively referred to as **MacroDB** for core persistence. The `macro_db_client` crate centralizes all database interactions, with schema migrations stored in `crates/macro_db_client/migrations`.

```bash

# Initialize the database schema

just setup_macrodb

```

### External Infrastructure Dependencies

| Service | Purpose | Integration Point |
|---------|---------|-------------------|
| **S3** | Raw document file storage | `document_storage_service` |
| **Redis** | Caching and rate limiting | `connection_gateway` |
| **OpenSearch** | Full-text search indexing | `search_processing_service` |
| **DynamoDB** | WebSocket connection state | `connection_gateway` |

### Inter-Service Communication Patterns

Services within Macro by macro-inc communicate through four primary mechanisms:

1. **HTTP/Axum clients** – Synchronous request-response between services
2. **SQS queues** – Asynchronous processing for document extraction, email ingestion
3. **AWS Lambda triggers** – Event-driven background tasks
4. **WebSocket gateway** – Real-time notifications via `connection_gateway`

## Key Services in Macro by Macro-Inc

### Document Storage Service

The `document_storage_service` crate exposes HTTP endpoints for document lifecycle operations. The main API surface is defined in [`services/document_storage_service/src/api.rs`](https://github.com/macro-inc/macro/blob/main/services/document_storage_service/src/api.rs).

Example: Creating a document via the Rust client:

```rust
use macro_db_client::document::v2::create::{CreateDocumentArgs, create_document};

let args = CreateDocumentArgs {
    owner_id: "user-123".into(),
    title: "Project Proposal".into(),
    mime_type: "application/pdf".into(),
};

let db = macro_db_client::get_pool().await?;
let doc = create_document(&db, args).await?;
println!("Created document ID: {}", doc.document_id);

```

*Reference:* [`crates/macro_db_client/src/document/v2/create.rs`](https://github.com/macro-inc/macro/blob/main/crates/macro_db_client/src/document/v2/create.rs)

### Search Processing Service

Built on OpenSearch, this service handles full-text indexing and query execution. The Axum-based HTTP layer is implemented in [`services/search_processing_service/src/service.rs`](https://github.com/macro-inc/macro/blob/main/services/search_processing_service/src/service.rs).

```rust
#[axum::debug_handler]
async fn search_documents(
    State(state): State<AppState>,
    Json(query): Json<SearchQuery>,
) -> Result<Json<SearchResults>, AppError> {
    let results = state.search_service.search(&query).await?;
    Ok(Json(results))
}

```

*Reference:* [`services/search_processing_service/src/service.rs`](https://github.com/macro-inc/macro/blob/main/services/search_processing_service/src/service.rs)

### Email Service

The `email_service` crate manages both inbound and outbound email processing, including automatic contact linking and thread organization. Entry point: [`services/email_service/src/lib.rs`](https://github.com/macro-inc/macro/blob/main/services/email_service/src/lib.rs).

### Background Worker Infrastructure

The `worker_trigger` service ([`services/worker_trigger/src/main.rs`](https://github.com/macro-inc/macro/blob/main/services/worker_trigger/src/main.rs)) orchestrates background job execution, while specialized workers like `ai_editing_worker` handle domain-specific tasks. These can run as Lambda functions or long-running processes depending on deployment configuration.

## Queue-Based Asynchronous Processing

Macro by macro-inc leverages SQS for decoupling time-consuming operations. The `macro_queues` crate provides type-safe queue abstractions.

Publishing to the document text extractor queue:

```rust
use macro_queues::DocumentTextExtractorQueue;
use aws_sdk_sqs::Client as SqsClient;

let sqs = SqsClient::new(&aws_config::load_from_env().await);
let payload = serde_json::to_string(&my_doc_event).unwrap();

sqs.send_message()
    .queue_url(DocumentTextExtractorQueue::LOCAL)
    .message_body(payload)
    .send()
    .await?;

```

*Reference:* [`crates/macro_queues/src/document_text_extractor.rs`](https://github.com/macro-inc/macro/blob/main/crates/macro_queues/src/document_text_extractor.rs)

## Observability and Configuration

### OpenTelemetry Integration

The `worker-rs-otel` crate implements distributed tracing across all services. Spans are exported to an OTLP endpoint configured in the deployment workflow.

Entry point: [`crates/worker-rs-otel/src/lib.rs`](https://github.com/macro-inc/macro/blob/main/crates/worker-rs-otel/src/lib.rs)

### Environment Management

The `macro_env_var` crate enforces that all configuration is sourced from **Doppler** rather than standard environment variables, preventing accidental secret exposure.

## Deployment and Local Development

The `xtask` tooling crate automates CI/CD operations. The web application deployment logic lives in [`tooling/xtask/crates/xtask_workflows/src/workflows/deploy_web_app.rs`](https://github.com/macro-inc/macro/blob/main/tooling/xtask/crates/xtask_workflows/src/workflows/deploy_web_app.rs).

For local development, the repository includes comprehensive setup instructions:

```bash

# Clone and enter the repository

git clone https://github.com/macro-inc/macro.git
cd macro

# Set up the database

just setup_macrodb

# Run the full stack locally

# (see docs/RUNNING_LOCALLY.md for detailed steps)

```

Full local development guide: [`docs/RUNNING_LOCALLY.md`](https://github.com/macro-inc/macro/blob/main/docs/RUNNING_LOCALLY.md)

## Summary

- **Macro by macro-inc** is a Rust-based, open-source platform for document-centric applications with 80+ crates in a unified Cargo workspace
- **Core capabilities** include document storage, full-text search, email processing, real-time notifications, and AI-powered editing
- **Data layer** uses PostgreSQL (MacroDB), S3 for files, Redis for caching, OpenSearch for search, and DynamoDB for WebSocket state
- **Communication patterns** span HTTP, SQS queues, Lambda triggers, and WebSockets
- **Production features** include OpenTelemetry observability, Doppler-backed secrets management, and automated deployment tooling

## Frequently Asked Questions

### What programming language is Macro by macro-inc built with?

Macro is built entirely in **Rust**, leveraging the language's performance, memory safety, and async ecosystem. The project uses Axum for HTTP services, the AWS SDK for cloud integrations, and sqlx for type-safe database operations.

### How do I run Macro by macro-inc locally?

Clone the repository and follow [`docs/RUNNING_LOCALLY.md`](https://github.com/macro-inc/macro/blob/main/docs/RUNNING_LOCALLY.md). The `just setup_macrodb` command initializes the PostgreSQL schema, and the workspace-based structure allows running individual services with `cargo run -p <service-name>`. All external dependencies (S3, Redis, OpenSearch, DynamoDB) can be mocked or run via Docker for local development.

### Is Macro by macro-inc suitable for production use?

Yes. The platform includes production-grade features: structured logging and distributed tracing via OpenTelemetry, secrets management through Doppler integration, rate limiting, comprehensive error handling, and automated deployment workflows. The 80+ crate architecture enables independent scaling of services based on load.

### What makes Macro by macro-inc different from other document platforms?

Macro combines **document storage, search, email, and real-time collaboration** in a single unified Rust codebase. Unlike platforms that bolt together disparate services, Macro's microservices are co-designed with shared types, consistent observability, and unified deployment tooling—all while remaining fully open-source under the macro-inc organization.