# Main Components of the Macro Inc. Project: A Complete Architecture Guide

> Explore the main components of the Macro Inc. project. This Rust microservice platform features seven core layers for efficient development and deployment. Learn about storage, processing, communication, and more.

- Repository: [Macro/macro](https://github.com/macro-inc/macro)
- Tags: architecture
- Published: 2026-08-20

---

**The Macro Inc. project is a Rust-based microservice platform organized into seven core layers: storage services, processing services, communication services, infrastructure services, shared platform packages, front-end applications, and developer tooling.**

According to the `macro-inc/macro` source code, the repository hosts a cloud-native **Cargo workspace** containing more than 80 crates. Understanding the main components of the Macro Inc. project helps developers navigate its document-centric architecture and contribute to the right service or library.

## Core Storage Services in the Macro Inc. Project

The storage layer handles document persistence, metadata management, and search indexing.

- **Document Storage Service** – The central API for uploading, retrieving, and managing documents. Source files live in `services/document_storage_service/`, with service documentation defined in [`services/document_storage_service/README.md`](https://github.com/macro-inc/macro/blob/main/services/document_storage_service/README.md).
- **Document Cognition Service** – Performs OCR, classification, and enrichment on stored documents. Implemented in [`services/document_cognition_service/README.md`](https://github.com/macro-inc/macro/blob/main/services/document_cognition_service/README.md).
- **MacroDB Client** – A typed **PostgreSQL** client shared across services. Defined in [`crates/macro_db_client/README.md`](https://github.com/macro-inc/macro/blob/main/crates/macro_db_client/README.md).
- **OpenSearch Client** – A thin wrapper around the **OpenSearch** REST API for indexing and search queries. Located at [`crates/opensearch_client/README.md`](https://github.com/macro-inc/macro/blob/main/crates/opensearch_client/README.md).
- **OpenSearch Query Builder** – Helpers for constructing complex search queries. Defined in [`crates/opensearch_query_builder/README.md`](https://github.com/macro-inc/macro/blob/main/crates/opensearch_query_builder/README.md).

These services share a common **SQLX-offline schema** that is regenerated with `just prepare_db` after any migration change.

## Processing Services in the Macro Inc. Project

Processing services convert raw files into searchable content and prepare them for indexing.

- **Convert Service** – Converts PDFs, DOCX, ODT, and other formats into a canonical representation. Source available in [`services/convert_service/README.md`](https://github.com/macro-inc/macro/blob/main/services/convert_service/README.md).
- **Document Text Extractor** – Extracts raw text from uploaded files using engines such as PDFium and DOCX unzip logic. Found in [`services/document_text_extractor/README.md`](https://github.com/macro-inc/macro/blob/main/services/document_text_extractor/README.md).
- **Search Processing Service** – Builds and updates OpenSearch indexes after text extraction. Defined in [`services/search_processing_service/README.md`](https://github.com/macro-inc/macro/blob/main/services/search_processing_service/README.md).

Each processor runs as a short-lived Lambda or container, consuming jobs from **Amazon SQS** queues.

## Communication Services in the Macro Inc. Project

The communication layer manages email, real-time notifications, and webhook delivery.

- **Email Service** – Handles inbound and outbound email, attachment storage, and Gmail synchronization. CLI tools and binaries are located under `services/email_service/src/bin/`.
- **Notification Service** – Pushes real-time notifications to mobile and web clients. Configuration and logic are in [`services/notification_service/README.md`](https://github.com/macro-inc/macro/blob/main/services/notification_service/README.md).
- **Webhook Crate** – Provides generic inbound and outbound webhook handling, SQS delivery, and HTTP validation. The implementation resides in [`crates/webhook/README.md`](https://github.com/macro-inc/macro/blob/main/crates/webhook/README.md), with outbound SQS logic in [`crates/webhook/src/outbound/sqs_queue.rs`](https://github.com/macro-inc/macro/blob/main/crates/webhook/src/outbound/sqs_queue.rs) and HTTP delivery in [`crates/webhook/src/outbound/http_delivery.rs`](https://github.com/macro-inc/macro/blob/main/crates/webhook/src/outbound/http_delivery.rs).

These services integrate with external providers such as Slack and Gmail via **MCP client** and **Kafka** utilities.

## Infrastructure Services in the Macro Inc. Project

Infrastructure services support authentication, real-time connectivity, and background job orchestration.

- **Authentication Service** – Manages user login and password-less flows via **FusionAuth**. Implemented in `services/authentication_service/`.
- **Connection Gateway** – Acts as a **WebSocket** gateway for real-time collaboration. Source located at [`services/connection_gateway/README.md`](https://github.com/macro-inc/macro/blob/main/services/connection_gateway/README.md).
- **Contacts Service** – Manages user contacts and connections. Found in [`services/contacts_service/README.md`](https://github.com/macro-inc/macro/blob/main/services/contacts_service/README.md).
- **Worker Trigger** – Orchestrates ECS task runs for background jobs. Defined in [`services/worker_trigger/README.md`](https://github.com/macro-inc/macro/blob/main/services/worker_trigger/README.md).

Infrastructure is provisioned with **Pulumi** and organized under `infra/stacks/`.

## Platform Packages and SDKs

Platform packages contain shared utilities, SDKs, and client libraries used by both back-end services and the front-end.

- **SDK** – The public **Rust SDK** for consumers of Macro APIs. Located in [`packages/sdk/README.md`](https://github.com/macro-inc/macro/blob/main/packages/sdk/README.md).
- **Loro-Mirror** – Provides **CRDT** primitives for collaborative editing. Found in [`packages/loro-mirror/README.md`](https://github.com/macro-inc/macro/blob/main/packages/loro-mirror/README.md).
- **Lexical-Core** – Supplies text-editing primitives for the web UI. Defined in [`packages/lexical-core/README.md`](https://github.com/macro-inc/macro/blob/main/packages/lexical-core/README.md).
- **Collaboration** – Contains WebSocket-based sync primitives and adapters. Source available in [`packages/collaboration/README.md`](https://github.com/macro-inc/macro/blob/main/packages/collaboration/README.md).
- **Teams Crate** – Shared domain logic for team management. Located in `crates/teams/`.
- **Notification DB Client** – A typed database client specialized for notification persistence. Found in `crates/notification_db_client/`.

## Front-End Applications

The front-end layer delivers the user interface through multiple applications.

- **Web UI (Tauri)** – A desktop-like experience built with **React** and **Tauri**. The Tauri-specific code lives in `apps/web/tauri/`, with protocol handling implemented in [`apps/web/tauri/src-tauri/src/tauri_protocol.rs`](https://github.com/macro-inc/macro/blob/main/apps/web/tauri/src-tauri/src/tauri_protocol.rs).
- **Web Front-End (SPA)** – A browser-based client for document workspaces. Source resides in `apps/web/`, with core library documentation in [`apps/web/src/lib/core/README.md`](https://github.com/macro-inc/macro/blob/main/apps/web/src/lib/core/README.md).
- **Documentation Site** – A static site generated with **MDX**. Located in `apps/docs/`.

The UI consumes back-end APIs through the **SDK** and direct service calls such as `/api/documents` and `/api/search`.

## Developer Tooling for the Macro Inc. Project

Tooling ensures consistent builds, testing, and local development across the workspace.

- **xtask** – A unified CLI for building, testing, linting, and deploying the workspace. Defined in [`tooling/xtask/README.md`](https://github.com/macro-inc/macro/blob/main/tooling/xtask/README.md).
- **notification_sandbox** – A local harness for testing notification pipelines. Found in [`tooling/notification_sandbox/README.md`](https://github.com/macro-inc/macro/blob/main/tooling/notification_sandbox/README.md).
- **local_e2e_test_support** – Utilities for integration tests that spin up Docker containers. Located in [`crates/local_e2e_test_support/README.md`](https://github.com/macro-inc/macro/blob/main/crates/local_e2e_test_support/README.md).

All commands are wrapped in **justfiles** (for example, `just build`, `just test`, `just stack up`) to ensure repeatable environment setup.

## SDK Example: Creating a Document

The following runnable example demonstrates how to create a document using the public SDK:

```rust
use macro_sdk::client::MacroClient;
use macro_sdk::models::DocumentCreate;

#[tokio::main]
async fn main() -> Result<(), macro_sdk::Error> {
    // Initialise a client (assumes `MACRO_API_URL` is set in the env)
    let client = MacroClient::new_from_env()?;

    // Prepare a new document request
    let doc = DocumentCreate {
        title: "My First Document".into(),
        // binary payload would be uploaded separately; here we just set metadata
        metadata: Default::default(),
    };

    // Send the request to the Document Storage Service
    let created = client.documents().create(doc).await?;
    println!("Created document with ID: {}", created.id);
    Ok(())
}

```

The `MacroClient` is exported from `packages/sdk`, and this pattern is the standard entry point for back-end integrations with the Macro Inc. project.

## Summary

- The Macro Inc. project splits functionality into **storage**, **processing**, **communication**, and **infrastructure** microservices.
- Shared logic lives in **platform packages** such as the SDK, Loro-Mirror, and collaboration crates.
- The front-end consists of a **Tauri desktop app**, a **web SPA**, and a **documentation site**.
- **Developer tooling** (`xtask`, `notification_sandbox`, and local E2E support) keeps builds and tests reproducible via `just` commands.
- Services communicate via HTTP APIs, SQS queues, and WebSocket connections, backed by PostgreSQL, S3, Redis, OpenSearch, and DynamoDB.

## Frequently Asked Questions

### What programming language is the Macro Inc. project built with?

The Macro Inc. project is written primarily in **Rust** and organized as a Cargo workspace with over 80 crates. This includes service binaries, shared libraries, and front-end tooling.

### How do the services in the Macro Inc. project communicate?

Services communicate through **HTTP APIs**, **Amazon SQS queues**, and **WebSocket** connections. For example, the webhook crate uses [`crates/webhook/src/outbound/sqs_queue.rs`](https://github.com/macro-inc/macro/blob/main/crates/webhook/src/outbound/sqs_queue.rs) for queued delivery and [`crates/webhook/src/outbound/http_delivery.rs`](https://github.com/macro-inc/macro/blob/main/crates/webhook/src/outbound/http_delivery.rs) for direct HTTP transmission.

### What databases does the Macro Inc. project use?

The platform persists data in **PostgreSQL** (via the MacroDB and ContactsDB clients), **Amazon S3** for object storage, **Redis** for caching, **OpenSearch** for full-text search, and **DynamoDB** for select operational data stores. These backends are accessed through typed clients such as `crates/macro_db_client` and `crates/opensearch_client`.

### What is the purpose of the `xtask` tooling crate?

`xtask`, located in [`tooling/xtask/README.md`](https://github.com/macro-inc/macro/blob/main/tooling/xtask/README.md), is the unified command-line interface for the workspace. It wraps build, test, lint, and deployment workflows so that developers can run standardized commands such as `just build` and `just test` across all services.