What Database Does Macro-Inc's Macro Use? A Complete Data Architecture Breakdown

Macro-Inc's macro repository uses PostgreSQL as its primary relational database, complemented by DynamoDB for connection state, Redis for caching, OpenSearch for search, and Amazon S3 for blob storage.

The macro monorepo employs a polyglot persistence architecture that matches data stores to specific workload demands. While PostgreSQL serves as the central relational backbone—internally branded as MacroDB—the system strategically integrates multiple specialized databases and cloud services to optimize for performance, scalability, and query patterns.

Primary Database: PostgreSQL (MacroDB)

PostgreSQL functions as the core relational database for the macro platform. The repository includes a comprehensive migration system that defines schemas for communications, email, document management, and other domain services.

Schema Management

Migration scripts in crates/macro_db_client/migrations/ define the relational structure. For example, 20251104101012_comms_db_schema.sql establishes the comms_ table family for messaging and channel data:

• comms_messages — individual message records
• comms_channels — channel/group definitions
• comms_attachments — metadata for file attachments
• comms_participants — channel membership mappings

Connection Pool Implementation

Services across the monorepo establish PostgreSQL connectivity through sqlx::postgres::PgPoolOptions. This pattern appears consistently in service entry points.

// services/email_service/src/main.rs
use sqlx::postgres::PgPoolOptions;

let pool = PgPoolOptions::new()
    .max_connections(10)
    .connect(&std::env::var("DATABASE_URL")?)
    .await?;

The central configuration in infra/packages/service/src/macrodb.ts supplies the DATABASE_URL connection string to all deployed services, ensuring consistent authentication and routing across environments.

Supporting Data Stores: The Complete Stack

Beyond PostgreSQL, macro leverages four additional data technologies for specialized use cases:

Store	Purpose	Key Implementation File
DynamoDB	Connection gateway state, fast key-value operations	services/connection_gateway/src/service/dynamodb.rs
Redis	Session caching, ephemeral data, rate limiting	docker/docker-compose-databases.yml (service definition)
OpenSearch	Full-text search for documents and chat history	infra/stacks/document-storage/dynamodb.ts (co-located config)
Amazon S3	Binary large object (BLOB) storage for uploaded files	Various service integrations via AWS SDK

DynamoDB: Low-Latency State Management

The connection_gateway service uses DynamoDB to track real-time WebSocket gateway state. This avoids query overhead on PostgreSQL for ephemeral, high-churn data.

// services/connection_gateway/src/service/dynamodb.rs
let dynamo_client = aws_sdk_dynamodb::Client::new(&config);

let resp = dynamo_client
    .put_item()
    .table_name("connection_state")
    .item("gateway_id", gateway_id.into())
    .item("user_id", user_id.into())
    .item("expires_at", expiry.into())
    .send()
    .await?;

This implementation uses single-digit millisecond latency characteristics of DynamoDB to maintain user presence and routing information without burdening the primary relational store.

Redis: Caching Layer

Redis appears in docker/docker-compose-databases.yml alongside PostgreSQL, providing:

• Session token validation — avoiding repeated PostgreSQL lookups for authenticated requests
• Rate limit counters — sliding window implementations for API throttling
• Temporary job state — queuing and locking for background workers

Services connect via standard Redis clients with connection pooling configured through environment variables.

OpenSearch and S3: Search and Storage

OpenSearch indexes document content and chat message text for full-text search capabilities that PostgreSQL's tsvector cannot efficiently provide at scale. The configuration in the document storage infrastructure stack provisions both OpenSearch domains and supporting DynamoDB tables for metadata indexing.

Amazon S3 handles the actual binary content of uploaded documents. Services stream file data to S3 while storing reference URIs and metadata in PostgreSQL—a classic hybrid relational/object storage pattern.

Why This Database Architecture?

The macro team selected this multi-database approach based on operational requirements:

1. ACID compliance for core business data — PostgreSQL handles transactions requiring strict consistency (payments, permissions, audit trails)
2. Sub-10ms latency for connection state — DynamoDB replaces PostgreSQL for gateway routing tables that churn thousands of times per second
3. Cost-efficient full-text search — OpenSearch decouples expensive text indexing from the transactional database
4. Unlimited scale for file storage — S3 eliminates capacity planning for binary uploads

Local Development Setup

The Docker Compose configuration in docker/docker-compose-databases.yml provisions PostgreSQL, Redis, and supporting infrastructure for local development:

# Start all data stores locally

docker-compose -f docker/docker-compose-databases.yml up -d postgres redis

This matches production topology without requiring AWS credentials for DynamoDB or OpenSearch, letting developers run the full stack offline.

Summary

• PostgreSQL is the primary database in macro-inc/macro, branded internally as MacroDB, handling all relational data with sqlx::postgres::PgPoolOptions connection pools
• DynamoDB accelerates connection gateway operations via aws_sdk_dynamodb::Client in services/connection_gateway/src/service/dynamodb.rs
• Redis provides caching and session management through the Docker Compose stack
• OpenSearch and Amazon S3 extend capabilities for search and binary storage respectively
• Migration files in crates/macro_db_client/migrations/ version-control the PostgreSQL schema evolution

Frequently Asked Questions

Is macro-inc/macro a PostgreSQL-only application?

No. While PostgreSQL serves as the primary relational database, macro employs a polyglot persistence model with DynamoDB, Redis, OpenSearch, and S3 handling specialized workloads that fall outside PostgreSQL's optimal performance envelope.

How does macro handle database migrations?

Migrations are version-controlled SQL files in crates/macro_db_client/migrations/. The team applies these through standard Rust migration tools integrated with sqlx, ensuring schema changes are tracked alongside application code.

Why use DynamoDB instead of PostgreSQL for connection state?

Connection gateway state requires consistent sub-10 millisecond read/write latency across multiple availability zones. DynamoDB's global tables and on-demand scaling provide this without connection pool contention or replication lag that would affect PostgreSQL under similar load patterns.

Can I run the macro database stack locally without AWS services?

Partially. The Docker Compose configuration provides PostgreSQL and Redis locally. DynamoDB and OpenSearch functionality requires either LocalStack emulation, DynamoDB Local, or actual AWS credentials, depending on which services your development workflow exercises.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}