Macro Rust Service Deployment Patterns: CI-Driven Microservices with Nix, Crane, and Pulumi

Macro's Rust services deploy through a consistent, Nix-based CI pipeline that uses crane for Dockerized binaries, cargo-zigbuild for Lambda handlers, and Pulumi for infrastructure orchestration.

The macro-inc/macro codebase implements a service-agnostic deployment architecture designed for reproducibility and speed. Every Rust microservice—whether a long-lived binary or an event-driven Lambda—follows the same automated workflow from build to production. This article breaks down the deployment patterns driving Macro's Rust services, with direct references to source implementation files.

Binary Builds with Crane and Nix

Services running as persistent processes—such as document-storage-service and search_service—are compiled to native binaries inside a hermetic Nix environment. The crane library handles the build, producing slim Docker images containing only the compiled binary and its runtime dependencies.

In tooling/xtask/crates/xtask_workflows/src/workflows/reusable_deploy_service.rs, the binary build job chains several steps:

// tooling/xtask/crates/xtask_workflows/src/workflows/reusable_deploy_service.rs
Job::new("build-binaries")
    .add_step(build_with_crane())
    .add_step(package_docker_image())
    .add_output("image-tarball", "result/image.tar.gz")

This approach eliminates "works on my machine" problems by pinning the entire toolchain—Rust compiler, system libraries, and build utilities—in the Nix flake.

Lambda Builds with cargo-zigbuild

Event-driven handlers like upload_extractor_lambda_handler and docx_unzip_handler require a different build path. Macro uses cargo-zigbuild to cross-compile to AWS-compatible LLVM-IR without needing a full target toolchain installation.

The .github/scripts/build-cloud-storage-lambdas-nix.sh script orchestrates this:

#!/usr/bin/env bash
set -euo pipefail
nix build .#lambdaDeployCargoArtifacts
cp -L result/lambda-artifacts.tar.gz lambda-artifacts.tar.gz

The resulting lambda-artifacts.tar.gz contains the compiled artifacts ready for Lambda deployment. This build path is defined in reusable_deploy_service.rs as a discrete job type with its own caching strategy.

Artifact Hand-off via nsc

Both build paths converge on a standardized artifact hand-off mechanism. The CI job computes a SHA checksum of the output, then uses the nsc artifact command to register it for downstream consumption.

This pattern allows the build stage to remain completely decoupled from deployment. The nsc tool manages artifact versioning and availability across job boundaries in GitHub Actions.

Service Matrix Generation from JSON Config

Macro avoids rebuilding the entire workspace on every PR. The .github/services-config.json file declares which services have binaries (deploy_binaries) versus Lambdas (deploy_lambdas):

{
  "services": {
    "search_service": {
      "deploy_binaries": ["search-service"],
      "deploy_lambdas": []
    },
    "upload_extractor_lambda_handler": {
      "deploy_binaries": [],
      "deploy_lambdas": ["upload-extractor"]
    }
  }
}

The workflow parses this with jq to generate dynamic job matrices:

# .github/workflows/reusable_deploy_service.yml

- name: Generate service matrix
  id: matrix
  run: |
    cfg=.github/services-config.json
    lambdas=$(jq -c '[.services | to_entries[] | select((.value.deploy_lambdas // []) | length > 0) | .key]' "$cfg")
    echo "lambdas=$lambdas" >> "$GITHUB_OUTPUT"

Only affected services enter the build queue, minimizing CI time and resource usage.

Warm-Lambda Dependency Caching

Because cargo-zigbuild compiles from scratch for the x86_64-unknown-linux-musl target, dependency resolution is expensive. Macro solves this with a dedicated "warm-lambdas" job that pre-builds shared dependencies on a sticky Nix store.

Located in tooling/xtask/crates/xtask_workflows/src/workflows/deploy_all_services.rs, this job runs before any Lambda build:

• Establishes a warm Nix store with common crates cached
• Subsequent Lambda builds mount this store, reusing compiled dependencies
• Reduces typical Lambda build times from minutes to seconds

The cache invalidates on Cargo.lock changes or explicit cache-bust triggers.

Pulumi Preview and Apply Stages

After artifacts are built and handed off, infrastructure changes proceed through two-phase Pulumi execution:

Preview stage (validation):

- uses: ./.github/actions/preview-cloud-storage-pulumi
  with:
    stack: prod
    service: ${{ matrix.service }}

Apply stage (deployment):

• Updates Lambda function versions with new artifact ARNs
• Refreshes ECS task definitions for binary services
• Pushes new Docker images to the container registry

The preview step acts as a safety gate—failed previews block deployment. This logic resides in .github/actions/deploy-cloud-storage-pulumi/action.yml.

GitHub Actions Reusable Workflows

All deployment logic is encapsulated in composable GitHub Actions workflows:

Workflow	Purpose
reusable_deploy_service.yml	Core build-deploy sequence for any service
deploy_on_push.yml	Triggered on merge to main; determines affected services
deploy_all_services.yml	Full workspace deployment with warm-lambda preparation

These workflows expose typed inputs (service, environment, dry-run) enabling the same CI definition to service any workspace member.

Local Development with Just

Developers mirror CI behavior locally through just tasks defined in the top-level justfile:

# Build Lambda artifacts locally

just build-lambda-artifacts

# Deploy a specific service

just deploy-service search_service

# Full workspace deployment

just deploy-all

This alignment ensures that failures caught in CI are reproducible on developer machines, using identical Nix environments and build flags.

Summary

Macro's Rust service deployment patterns demonstrate how to scale microservice CI without sacrificing reproducibility or developer velocity:

• crane + Nix builds hermetic, minimal Docker images for long-lived services
• cargo-zigbuild cross-compiles Lambda handlers with Zig's toolchain
• nsc artifact provides decoupled artifact hand-off between build and deploy stages
• JSON-driven matrices scope CI work to changed services only
• Warm-lambda caching amortizes dependency build costs across PRs
• Pulumi two-phase deploy validates infrastructure before applying changes
• just task runner unifies local and CI workflows

Frequently Asked Questions

How does Macro achieve reproducible Rust builds across CI and local development?

Macro pins the entire build environment—including Rust version, system libraries, and build tools—inside a Nix flake. Both CI runners and local developer machines execute builds through Nix, ensuring identical outputs regardless of host system state.

What is cargo-zigbuild and why does Macro use it for Lambda functions?

cargo-zigbuild is a Cargo subcommand that uses Zig as the linker for cross-compilation. Macro uses it because Zig ships with libc implementations for multiple targets, eliminating the need to install musl cross-compilation toolchains. This produces smaller, fully static binaries that run on AWS Lambda's provided runtime.

How does Macro avoid rebuilding every service on every PR?

The .github/services-config.json file declares each service's deployment artifacts. CI parses this file to generate job matrices via jq, selecting only services with changes in the current branch. This matrix generation step runs before any expensive build operations.

Why warm Lambda dependencies separately from the main build?

Lambda builds with cargo-zigbuild compile all dependencies from source for the musl target—a slow operation. The warm-lambdas job runs once per workflow, populating a shared Nix store that subsequent Lambda builds mount. This cache reuse typically reduces Lambda build times by 60-90% according to the workflow implementation in deploy_all_services.rs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}