How to Implement a New AWS Lambda Handler for Asynchronous Processing in Macro

Create a new asynchronous Lambda handler by defining environment-variable schemas, bundling constructor arguments, provisioning IAM policies with SQS permissions, instantiating the generic Lambda<TEnv> component, and adding an EventSourceMapping to connect an SQS queue trigger.

The Macro repository uses a reusable Lambda component in infra/packages/lambda/src/lambda.ts to standardize AWS Lambda deployment. This guide walks through implementing a new asynchronous Lambda handler—one triggered by Amazon SQS for background job processing—following the established patterns from Upload Extractor and Docx Unzip handlers.

---

Define the Environment-Variable Schema

Every Lambda handler in Macro starts with explicit type safety for runtime configuration. Declare a TypeScript type that lists every environment variable your binary expects.

Reference pattern: UploadExtractorLambdaHandlerEnvVars in infra/stacks/bulk-upload/upload-extractor-lambda-handler.ts

export type MyAsyncHandlerEnvVars = {
  DATABASE_URL: pulumi.Output<string> | string;
  MY_QUEUE_URL: pulumi.Output<string> | string;
  RUST_LOG: pulumi.Output<string> | string;
  // Add any variables required by your compiled binary
};

Using pulumi.Output<string> | string ensures compatibility with both hardcoded values and references to other Pulumi-managed resources.

---

Create a Constructor Argument Type

Bundle environment variables with AWS resource references your handler needs. This pattern keeps the component interface clean and extensible.

Reference pattern: UploadExtractorLambdaHandlerArgs in the same Upload Extractor file

export interface MyAsyncHandlerArgs {
  envVars: MyAsyncHandlerEnvVars;
  queueArn: pulumi.Output<string> | string;
  tags: { [key: string]: string };
}

The queueArn parameter connects your handler to the specific SQS queue that triggers it, enabling true asynchronous processing where callers publish messages without waiting for execution.

---

Provision IAM Policies and Role

Asynchronous Lambda handlers need permission to receive and delete messages from SQS. Create minimal, least-privilege policies and attach them to a role with standard Lambda execution policies.

SQS policy implementation (lines 44-60 in Upload Extractor):

const sqsPolicy = new aws.iam.Policy(`${BASE_NAME}-sqs-policy`, {
  policy: pulumi.output({
    Version: '2012-10-17',
    Statement: [
      {
        Action: [
          'sqs:ReceiveMessage',
          'sqs:GetQueueAttributes',
          'sqs:DeleteMessage'
        ],
        Resource: [queueArn],
        Effect: 'Allow',
      },
    ],
  }),
  tags: this.tags,
});

Role construction with managed policies (lines 88-106):

const role = new aws.iam.Role(`${BASE_NAME}-role`, {
  assumeRolePolicy: JSON.stringify({
    Version: '2012-10-17',
    Statement: [{
      Action: 'sts:AssumeRole',
      Effect: 'Allow',
      Principal: { Service: 'lambda.amazonaws.com' }
    }],
  }),
  managedPolicyArns: [
    aws.iam.ManagedPolicy.AWSLambdaBasicExecutionRole,
    aws.iam.ManagedPolicy.AWSLambdaVPCAccessExecutionRole,
    sqsPolicy.arn,
  ],
  tags: this.tags,
});

This pattern—resource-specific policy plus standard AWS managed policies—appears consistently across Macro's Lambda implementations according to the source code.

---

Instantiate the Lambda and Async Trigger

Use the generic Lambda<TEnv> component from infra/packages/lambda/src/lambda.ts to create the function, then wire the SQS trigger using aws.lambda.EventSourceMapping.

const myLambda = new Lambda<MyAsyncHandlerEnvVars>(`${BASE_NAME}-lambda`, {
  baseName: BASE_NAME,
  handlerBase: `${REPO_ROOT}/services/${BASE_NAME}`,
  zipLocation: `${REPO_ROOT}/target/lambda/${BASE_NAME}/bootstrap.zip`,
  envVars,
  role,
  timeout: 300,
  memorySize: 1024,
  tags: this.tags,
});

EventSourceMapping for SQS trigger (lines 34-42 in Upload Extractor):

new aws.lambda.EventSourceMapping(`${BASE_NAME}-sqs-source`, {
  eventSourceArn: queueArn,
  functionName: myLambda.lambda.name,
  batchSize: 1,
  enabled: true,
});

The batchSize: 1 setting processes messages individually, simplifying error handling and retry logic for asynchronous jobs.

---

Add CloudWatch Alarms (Optional)

For production reliability, include throttling and error alarms using the setupLambdaAlarms() helper pattern found in DocxUnzipHandlerLambda.

Reference: infra/stacks/cloud-storage-service/docx-unzip-handler-lambda.ts

private setupLambdaAlarms() {
  // Alarms for Lambda errors, throttling, and DLQ activity
  // Mirror the implementation from UploadExtractorLambdaHandler
}

These alarms integrate with Macro's existing SNS topic for operations notifications, defined in infra/packages/shared/index.ts.

---

Summary

• Type-safe configuration: Define *EnvVars and *Args interfaces before implementation
• Least-privilege IAM: Create specific SQS policies and combine with AWSLambdaBasicExecutionRole
• Generic Lambda component: Use Lambda<TEnv> from infra/packages/lambda/src/lambda.ts for consistent infrastructure
• EventSourceMapping: Connect SQS queues with batchSize: 1 for reliable asynchronous processing
• Observability: Add setupLambdaAlarms() for production monitoring

---

Frequently Asked Questions

What triggers an asynchronous Lambda handler in Macro?

An SQS queue triggers asynchronous Lambda handlers via aws.lambda.EventSourceMapping. The Lambda polls the queue and invokes your function with batch messages, decoupling producers from execution timing. This differs from synchronous handlers invoked directly via API Gateway or ALB.

How does the generic Lambda component handle environment variables?

The Lambda<TEnv> component in infra/packages/lambda/src/lambda.ts accepts a generic type parameter enforcing compile-time validation of environment variables. It merges your provided envVars with internal defaults and handles the CloudFormation representation for deployment.

Why use batchSize: 1 for SQS event sources?

Setting batchSize: 1 ensures individual message processing, preventing partial batch failures from blocking other messages. This simplifies error handling and aligns with Macro's pattern of idempotent, single-job processing seen in Upload Extractor and Docx Unzip handlers.

Where are Lambda binaries compiled in Macro?

Compiled binaries reside at ${REPO_ROOT}/target/lambda/${BASE_NAME}/bootstrap.zip, following Rust's cargo-lambda output structure. The zipLocation parameter in the Lambda component points to this artifact, deployed via Pulumi's asset management.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}