What Is Macro by Macro-Inc? A Deep Dive Into the Rust-Based Document Platform

Macro by macro-inc is an open-source, cloud-native microservices platform built in Rust that handles document storage, processing, search, communication, and email workflows through 80+ specialized crates.

This production-grade system provides a complete foundation for applications requiring robust document management, real-time collaboration, and integrated messaging. The entire codebase lives in the macro-inc/macro GitHub repository and is organized as a Cargo workspace with services for everything from AI-powered editing to WebSocket-driven notifications.

Core Architecture of Macro by Macro-Inc

The Macro platform follows a modular microservices design where each responsibility is isolated into its own crate. This structure enables independent scaling, deployment, and maintenance of individual components.

Workspace Organization

All services reside under the crates/ directory and are coordinated through the top-level Cargo.toml workspace definition. The AGENTS.md file documents the conventions and service responsibilities for developers contributing to the codebase.

Key service categories include:

• HTTP API services – document-storage-service, search_processing_service, email_service
• Background workers – worker_trigger, ai_editing_worker, document_text_extractor
• Shared libraries – macro_db_client, macro_queues, worker-rs-otel

Data Persistence Layer

Macro uses PostgreSQL schemas collectively referred to as MacroDB for core persistence. The macro_db_client crate centralizes all database interactions, with schema migrations stored in crates/macro_db_client/migrations.

# Initialize the database schema

just setup_macrodb

External Infrastructure Dependencies

Service	Purpose	Integration Point
S3	Raw document file storage	document_storage_service
Redis	Caching and rate limiting	connection_gateway
OpenSearch	Full-text search indexing	search_processing_service
DynamoDB	WebSocket connection state	connection_gateway

Inter-Service Communication Patterns

Services within Macro by macro-inc communicate through four primary mechanisms:

1. HTTP/Axum clients – Synchronous request-response between services
2. SQS queues – Asynchronous processing for document extraction, email ingestion
3. AWS Lambda triggers – Event-driven background tasks
4. WebSocket gateway – Real-time notifications via connection_gateway

Key Services in Macro by Macro-Inc

Document Storage Service

The document_storage_service crate exposes HTTP endpoints for document lifecycle operations. The main API surface is defined in services/document_storage_service/src/api.rs.

Example: Creating a document via the Rust client:

use macro_db_client::document::v2::create::{CreateDocumentArgs, create_document};

let args = CreateDocumentArgs {
    owner_id: "user-123".into(),
    title: "Project Proposal".into(),
    mime_type: "application/pdf".into(),
};

let db = macro_db_client::get_pool().await?;
let doc = create_document(&db, args).await?;
println!("Created document ID: {}", doc.document_id);

Reference: crates/macro_db_client/src/document/v2/create.rs

Search Processing Service

Built on OpenSearch, this service handles full-text indexing and query execution. The Axum-based HTTP layer is implemented in services/search_processing_service/src/service.rs.

#[axum::debug_handler]
async fn search_documents(
    State(state): State<AppState>,
    Json(query): Json<SearchQuery>,
) -> Result<Json<SearchResults>, AppError> {
    let results = state.search_service.search(&query).await?;
    Ok(Json(results))
}

Reference: services/search_processing_service/src/service.rs

Email Service

The email_service crate manages both inbound and outbound email processing, including automatic contact linking and thread organization. Entry point: services/email_service/src/lib.rs.

Background Worker Infrastructure

The worker_trigger service (services/worker_trigger/src/main.rs) orchestrates background job execution, while specialized workers like ai_editing_worker handle domain-specific tasks. These can run as Lambda functions or long-running processes depending on deployment configuration.

Queue-Based Asynchronous Processing

Macro by macro-inc leverages SQS for decoupling time-consuming operations. The macro_queues crate provides type-safe queue abstractions.

Publishing to the document text extractor queue:

use macro_queues::DocumentTextExtractorQueue;
use aws_sdk_sqs::Client as SqsClient;

let sqs = SqsClient::new(&aws_config::load_from_env().await);
let payload = serde_json::to_string(&my_doc_event).unwrap();

sqs.send_message()
    .queue_url(DocumentTextExtractorQueue::LOCAL)
    .message_body(payload)
    .send()
    .await?;

Reference: crates/macro_queues/src/document_text_extractor.rs

Observability and Configuration

OpenTelemetry Integration

The worker-rs-otel crate implements distributed tracing across all services. Spans are exported to an OTLP endpoint configured in the deployment workflow.

Entry point: crates/worker-rs-otel/src/lib.rs

Environment Management

The macro_env_var crate enforces that all configuration is sourced from Doppler rather than standard environment variables, preventing accidental secret exposure.

Deployment and Local Development

The xtask tooling crate automates CI/CD operations. The web application deployment logic lives in tooling/xtask/crates/xtask_workflows/src/workflows/deploy_web_app.rs.

For local development, the repository includes comprehensive setup instructions:

# Clone and enter the repository

git clone https://github.com/macro-inc/macro.git
cd macro

# Set up the database

just setup_macrodb

# Run the full stack locally

# (see docs/RUNNING_LOCALLY.md for detailed steps)

Full local development guide: docs/RUNNING_LOCALLY.md

Summary

• Macro by macro-inc is a Rust-based, open-source platform for document-centric applications with 80+ crates in a unified Cargo workspace
• Core capabilities include document storage, full-text search, email processing, real-time notifications, and AI-powered editing
• Data layer uses PostgreSQL (MacroDB), S3 for files, Redis for caching, OpenSearch for search, and DynamoDB for WebSocket state
• Communication patterns span HTTP, SQS queues, Lambda triggers, and WebSockets
• Production features include OpenTelemetry observability, Doppler-backed secrets management, and automated deployment tooling

Frequently Asked Questions

What programming language is Macro by macro-inc built with?

Macro is built entirely in Rust, leveraging the language's performance, memory safety, and async ecosystem. The project uses Axum for HTTP services, the AWS SDK for cloud integrations, and sqlx for type-safe database operations.

How do I run Macro by macro-inc locally?

Clone the repository and follow docs/RUNNING_LOCALLY.md. The just setup_macrodb command initializes the PostgreSQL schema, and the workspace-based structure allows running individual services with cargo run -p <service-name>. All external dependencies (S3, Redis, OpenSearch, DynamoDB) can be mocked or run via Docker for local development.

Is Macro by macro-inc suitable for production use?

Yes. The platform includes production-grade features: structured logging and distributed tracing via OpenTelemetry, secrets management through Doppler integration, rate limiting, comprehensive error handling, and automated deployment workflows. The 80+ crate architecture enables independent scaling of services based on load.

What makes Macro by macro-inc different from other document platforms?

Macro combines document storage, search, email, and real-time collaboration in a single unified Rust codebase. Unlike platforms that bolt together disparate services, Macro's microservices are co-designed with shared types, consistent observability, and unified deployment tooling—all while remaining fully open-source under the macro-inc organization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}