# What Database Does Macro-Inc's Macro Use? A Complete Data Architecture Breakdown

> Discover the exact databases used by Macro-Inc's macro repository. Learn about its complete data architecture including PostgreSQL, DynamoDB, Redis, OpenSearch, and S3.

- Repository: [Macro/macro](https://github.com/macro-inc/macro)
- Tags: architecture
- Published: 2026-08-15

---

**Macro-Inc's macro repository uses PostgreSQL as its primary relational database, complemented by DynamoDB for connection state, Redis for caching, OpenSearch for search, and Amazon S3 for blob storage.**

The macro monorepo employs a **polyglot persistence architecture** that matches data stores to specific workload demands. While PostgreSQL serves as the central relational backbone—internally branded as *MacroDB*—the system strategically integrates multiple specialized databases and cloud services to optimize for performance, scalability, and query patterns.

## Primary Database: PostgreSQL (MacroDB)

PostgreSQL functions as the **core relational database** for the macro platform. The repository includes a comprehensive migration system that defines schemas for communications, email, document management, and other domain services.

### Schema Management

Migration scripts in `crates/macro_db_client/migrations/` define the relational structure. For example, [`20251104101012_comms_db_schema.sql`](https://github.com/macro-inc/macro/blob/main/20251104101012_comms_db_schema.sql) establishes the `comms_` table family for messaging and channel data:

- `comms_messages` — individual message records
- `comms_channels` — channel/group definitions
- `comms_attachments` — metadata for file attachments
- `comms_participants` — channel membership mappings

### Connection Pool Implementation

Services across the monorepo establish PostgreSQL connectivity through **`sqlx::postgres::PgPoolOptions`**. This pattern appears consistently in service entry points.

```rust
// services/email_service/src/main.rs
use sqlx::postgres::PgPoolOptions;

let pool = PgPoolOptions::new()
    .max_connections(10)
    .connect(&std::env::var("DATABASE_URL")?)
    .await?;

```

The central configuration in [`infra/packages/service/src/macrodb.ts`](https://github.com/macro-inc/macro/blob/main/infra/packages/service/src/macrodb.ts) supplies the `DATABASE_URL` connection string to all deployed services, ensuring consistent authentication and routing across environments.

## Supporting Data Stores: The Complete Stack

Beyond PostgreSQL, macro leverages four additional data technologies for specialized use cases:

| Store | Purpose | Key Implementation File |
|-------|---------|------------------------|
| **DynamoDB** | Connection gateway state, fast key-value operations | [`services/connection_gateway/src/service/dynamodb.rs`](https://github.com/macro-inc/macro/blob/main/services/connection_gateway/src/service/dynamodb.rs) |
| **Redis** | Session caching, ephemeral data, rate limiting | [`docker/docker-compose-databases.yml`](https://github.com/macro-inc/macro/blob/main/docker/docker-compose-databases.yml) (service definition) |
| **OpenSearch** | Full-text search for documents and chat history | [`infra/stacks/document-storage/dynamodb.ts`](https://github.com/macro-inc/macro/blob/main/infra/stacks/document-storage/dynamodb.ts) (co-located config) |
| **Amazon S3** | Binary large object (BLOB) storage for uploaded files | Various service integrations via AWS SDK |

## DynamoDB: Low-Latency State Management

The **connection_gateway service** uses DynamoDB to track real-time WebSocket gateway state. This avoids query overhead on PostgreSQL for ephemeral, high-churn data.

```rust
// services/connection_gateway/src/service/dynamodb.rs
let dynamo_client = aws_sdk_dynamodb::Client::new(&config);

let resp = dynamo_client
    .put_item()
    .table_name("connection_state")
    .item("gateway_id", gateway_id.into())
    .item("user_id", user_id.into())
    .item("expires_at", expiry.into())
    .send()
    .await?;

```

This implementation uses **single-digit millisecond latency** characteristics of DynamoDB to maintain user presence and routing information without burdening the primary relational store.

## Redis: Caching Layer

Redis appears in [`docker/docker-compose-databases.yml`](https://github.com/macro-inc/macro/blob/main/docker/docker-compose-databases.yml) alongside PostgreSQL, providing:

- **Session token validation** — avoiding repeated PostgreSQL lookups for authenticated requests
- **Rate limit counters** — sliding window implementations for API throttling
- **Temporary job state** — queuing and locking for background workers

Services connect via standard Redis clients with connection pooling configured through environment variables.

## OpenSearch and S3: Search and Storage

**OpenSearch** indexes document content and chat message text for **full-text search capabilities** that PostgreSQL's `tsvector` cannot efficiently provide at scale. The configuration in the document storage infrastructure stack provisions both OpenSearch domains and supporting DynamoDB tables for metadata indexing.

**Amazon S3** handles the actual binary content of uploaded documents. Services stream file data to S3 while storing reference URIs and metadata in PostgreSQL—a classic **hybrid relational/object storage pattern**.

## Why This Database Architecture?

The macro team selected this **multi-database approach** based on operational requirements:

1. **ACID compliance for core business data** — PostgreSQL handles transactions requiring strict consistency (payments, permissions, audit trails)
2. **Sub-10ms latency for connection state** — DynamoDB replaces PostgreSQL for gateway routing tables that churn thousands of times per second
3. **Cost-efficient full-text search** — OpenSearch decouples expensive text indexing from the transactional database
4. **Unlimited scale for file storage** — S3 eliminates capacity planning for binary uploads

## Local Development Setup

The Docker Compose configuration in [`docker/docker-compose-databases.yml`](https://github.com/macro-inc/macro/blob/main/docker/docker-compose-databases.yml) provisions PostgreSQL, Redis, and supporting infrastructure for local development:

```bash

# Start all data stores locally

docker-compose -f docker/docker-compose-databases.yml up -d postgres redis

```

This matches production topology without requiring AWS credentials for DynamoDB or OpenSearch, letting developers run the full stack offline.

## Summary

- **PostgreSQL** is the primary database in macro-inc/macro, branded internally as MacroDB, handling all relational data with `sqlx::postgres::PgPoolOptions` connection pools
- **DynamoDB** accelerates connection gateway operations via `aws_sdk_dynamodb::Client` in [`services/connection_gateway/src/service/dynamodb.rs`](https://github.com/macro-inc/macro/blob/main/services/connection_gateway/src/service/dynamodb.rs)
- **Redis** provides caching and session management through the Docker Compose stack
- **OpenSearch** and **Amazon S3** extend capabilities for search and binary storage respectively
- Migration files in `crates/macro_db_client/migrations/` version-control the PostgreSQL schema evolution

## Frequently Asked Questions

### Is macro-inc/macro a PostgreSQL-only application?

No. While PostgreSQL serves as the primary relational database, macro employs a **polyglot persistence model** with DynamoDB, Redis, OpenSearch, and S3 handling specialized workloads that fall outside PostgreSQL's optimal performance envelope.

### How does macro handle database migrations?

Migrations are **version-controlled SQL files** in `crates/macro_db_client/migrations/`. The team applies these through standard Rust migration tools integrated with `sqlx`, ensuring schema changes are tracked alongside application code.

### Why use DynamoDB instead of PostgreSQL for connection state?

Connection gateway state requires **consistent sub-10 millisecond read/write latency** across multiple availability zones. DynamoDB's global tables and on-demand scaling provide this without connection pool contention or replication lag that would affect PostgreSQL under similar load patterns.

### Can I run the macro database stack locally without AWS services?

**Partially.** The Docker Compose configuration provides PostgreSQL and Redis locally. DynamoDB and OpenSearch functionality requires either **LocalStack emulation**, **DynamoDB Local**, or actual AWS credentials, depending on which services your development workflow exercises.