# Nested CloudFormation Stack Architecture in AWS IDP: How the Patterns Interconnect

> Understand the nested CloudFormation stack architecture in AWS IDP. Learn how isolated processing patterns share core infrastructure through cross-stack parameter passing.

- Repository: [aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws)
- Tags: architecture
- Published: 2026-02-25

---

**The Accelerated Intelligent Document Processing on AWS solution uses a single top-level SAM template that orchestrates multiple nested CloudFormation stacks, where each processing pattern operates as an isolated nested stack while sharing core infrastructure like S3 buckets, DynamoDB tables, and IAM roles through cross-stack parameter passing.**

The `aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws` repository implements this nested CloudFormation stack architecture to enable modular deployment of intelligent document processing capabilities. This design allows organizations to select specific AI/ML patterns—such as Bedrock Data Automation or Textract with SageMaker—without deploying unnecessary resources, while maintaining a consistent foundation of shared services.

## Core Architecture Overview

### Top-Level Orchestration Template

The architecture centers on a single root template located at [`template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/template.yaml) in the repository root. This top-level stack acts as the orchestration layer, defining the foundational infrastructure and conditionally instantiating nested stacks based on deployment parameters.

The root template declares three primary pattern conditions—`IsPattern1`, `IsPattern2`, and `IsPattern3`—which evaluate the `IDPPattern` parameter to determine which nested stacks to provision. This conditional logic ensures that only the selected pattern's resources are deployed, optimizing cost and complexity.

### Shared Infrastructure Components

The top-level template provisions core resources that all patterns consume:

- **S3 Buckets**: `InputBucket` for document ingestion, `OutputBucket` for processed results, and `ConfigurationBucket` for workflow configurations
- **DynamoDB Tables**: `TrackingTable` for workflow state management across all patterns
- **IAM Roles**: Cross-service execution roles with least-privilege permissions
- **KMS Keys**: `CustomerManagedEncryptionKey` for encryption at rest

These resources are created once and referenced by all nested stacks through parameter passing, ensuring consistent security posture and data lineage across different processing patterns.

## Pattern-Specific Nested Stacks

Each processing pattern resides in its own nested stack under the `patterns/` directory, isolating pattern-specific Lambda functions, Step Functions state machines, and IAM policies.

### Pattern 1: Bedrock Data Automation

Located at [`patterns/pattern-1/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/patterns/pattern-1/template.yaml), this nested stack implements document processing using Amazon Bedrock Data Automation (BDA). The stack defines:

- An Amazon ECR repository for custom container images
- Docker-based Lambda functions for BDA invocation
- Step Functions workflows that orchestrate BDA project execution
- Pattern-specific IAM roles with permissions to invoke Bedrock models

The stack receives the `Pattern1BDAProjectArn` parameter from the top level, or optionally integrates with the `BDASampleProject` nested stack when users do not provide an existing BDA project ARN.

### Pattern 2: Textract and Bedrock

The [`patterns/pattern-2/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/patterns/pattern-2/template.yaml) nested stack combines Amazon Textract for OCR with Amazon Bedrock for intelligent classification and extraction. Key components include:

- Container image build resources for custom processing logic
- Lambda functions for Textract result processing and Bedrock prompt engineering
- Step Functions state machines that coordinate Textract async jobs with Bedrock inference
- Dedicated CloudWatch dashboards for pattern-specific metrics

This pattern consumes the shared `InputBucket` and `OutputBucket` references to read source documents and write structured extraction results.

### Pattern 3: Textract, SageMaker, and Bedrock

Found at [`patterns/pattern-3/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/patterns/pattern-3/template.yaml), this nested stack extends Pattern 2 by adding Amazon SageMaker endpoints for custom document understanding models (UDOP). The stack provisions:

- SageMaker model artifact handling and endpoint configuration
- Lambda functions for SageMaker inference integration
- Enhanced Step Functions workflows that branch between Textract, SageMaker, and Bedrock based on document characteristics
- Pattern-specific VPC configurations for SageMaker networking

All three pattern stacks export their `StateMachineArn` and `CloudWatchDashboardName` outputs, enabling the top-level stack to aggregate monitoring resources and UI integrations.

## Optional Supporting Stacks

The architecture includes several optional nested stacks under the `nested/` directory that enhance functionality without requiring pattern modifications.

### AppSync GraphQL API

The [`nested/appsync/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/nested/appsync/template.yaml) stack provisions a GraphQL API that powers the solution's web interface and enables external client integrations. The stack creates:

- AWS AppSync API and schema definitions
- DynamoDB data sources linked to the shared `TrackingTable`
- Lambda resolvers that invoke pattern Step Functions using the exported `StateMachineArn` references
- Cognito user pool integration for authentication

This stack is conditionally created when the `CreateAgentCoreLambda` parameter is set to true, and it receives the pattern state machine ARNs through the `Pattern1StateMachineArn`, `Pattern2StateMachineArn`, and `Pattern3StateMachineArn` parameters.

### Bedrock Knowledge Base

Located at [`nested/bedrockkb/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/nested/bedrockkb/template.yaml), this stack implements a Retrieval-Augmented Generation (RAG) pipeline for document search use cases. Components include:

- Amazon Bedrock Knowledge Base configuration
- OpenSearch Serverless vector store or Aurora PostgreSQL vector storage (configurable via `VectorStoreType`)
- S3 data source pointing to the shared `OutputBucket`
- IAM roles for Bedrock model invocation and vector store access

The stack is instantiated when `ShouldCreateDocumentKnowledgeBase` evaluates to true, and it integrates with pattern outputs to ensure processed documents are automatically indexed for semantic search.

### Sample BDA Project

The [`nested/bda-lending-project/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/nested/bda-lending-project/template.yaml) stack provides a ready-to-use Bedrock Data Automation project for users who do not have an existing BDA configuration. This stack:

- Creates a sample BDA project with pre-configured extraction schemas
- Outputs the `BDAProjectArn` for consumption by Pattern 1
- Includes sample blueprints for lending document processing

This optional stack bridges the gap for quick starts, allowing Pattern 1 to function immediately without manual BDA project creation.

## Cross-Stack Communication and Resource Sharing

### Parameter Passing Mechanism

The nested CloudFormation stack architecture relies on explicit parameter passing to share resources between the top-level stack and nested stacks. The top-level template declares each nested stack with a `Properties.Parameters` block that maps shared resource references:

```yaml
PATTERN2STACK:
  Type: AWS::CloudFormation::Stack
  Condition: IsPattern2
  Properties:
    TemplateURL: ./patterns/pattern-2/.aws-sam/packaged.yaml
    Parameters:
      InputBucket: !Ref InputBucket
      ConfigurationBucket: !Ref ConfigurationBucket
      OutputBucket: !Ref OutputBucket
      TrackingTable: !Ref TrackingTable
      CustomerManagedEncryptionKeyArn: !GetAtt CustomerManagedEncryptionKey.Arn

```

This mechanism ensures that nested stacks receive the actual resource ARNs and names at deployment time, enabling them to configure IAM policies and environment variables correctly without hardcoding values.

### Inter-Pattern Connectivity

While each pattern stack operates independently, they interconnect through the shared core infrastructure:

**Document Ingestion Flow**
The core `InputBucket` triggers a Queue Sender Lambda (defined in the top-level stack) that writes messages to an SQS queue. The Queue Processor Lambda then initiates the appropriate pattern's Step Functions execution using the state machine ARN exported by that pattern's nested stack.

**Workflow Tracking**
All pattern Lambdas read from and write to the shared `TrackingTable` DynamoDB table, maintaining a unified view of document processing status across different AI/ML approaches. The table schema is defined once in the top-level stack and consumed by all patterns.

**Result Storage**
Pattern-specific Lambdas write processed JSON outputs and PDFs to the shared `OutputBucket`. This centralization allows the optional Bedrock Knowledge Base stack to index results regardless of which pattern processed the document, and enables unified reporting through the core infrastructure.

**UI Integration**
The optional AppSync nested stack creates GraphQL resolvers that reference the pattern state machine ARNs (passed as parameters from the top-level stack). This allows a single web interface to trigger executions across Pattern 1, Pattern 2, or Pattern 3 without knowing the internal implementation details of each.

## Deployment Examples

### Deploying Pattern 2 via AWS CLI

To deploy the nested CloudFormation stack architecture with Pattern 2 (Textract + Bedrock) enabled, use the following command structure:

```bash
aws cloudformation deploy \
    --template-file template.yaml \
    --stack-name idp-accelerator \
    --parameter-overrides \
        IDPPattern=pattern-2 \
        AdminEmail=admin@example.com \
        MaxConcurrentExecutions=200 \
        ShouldCreateDocumentKnowledgeBase=true \
    --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND

```

The `IDPPattern=pattern-2` parameter triggers the `IsPattern2` condition, causing CloudFormation to instantiate the `PATTERN2STACK` nested stack from [`patterns/pattern-2/.aws-sam/packaged.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/patterns/pattern-2/.aws-sam/packaged.yaml).

### Referencing Pattern Outputs in Custom Resources

When extending the solution with custom monitoring Lambdas, reference the pattern state machine ARN using the nested stack output:

```yaml
Resources:
  CustomMonitoringFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/monitoring/
      Handler: index.handler
      Runtime: python3.12
      Environment:
        Variables:
          PATTERN_STATE_MACHINE_ARN: !GetAtt PATTERN2STACK.Outputs.StateMachineArn
          TRACKING_TABLE_NAME: !Ref TrackingTable

```

This configuration allows the monitoring function to poll Step Functions execution history using the exact ARN exported by the Pattern 2 nested stack, while reading status updates from the shared `TrackingTable` defined in the top-level stack.

## Summary

- The **Accelerated Intelligent Document Processing on AWS** solution uses a **single top-level SAM template** ([`template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/template.yaml)) to orchestrate multiple **nested CloudFormation stacks**, isolating pattern-specific resources while sharing core infrastructure.
- **Three primary patterns** (Bedrock Data Automation, Textract+Bedrock, Textract+SageMaker+Bedrock) each reside in their own nested stack under `patterns/pattern-{1,2,3}/template.yaml`, receiving shared resource references via parameters.
- **Optional nested stacks** for AppSync, Bedrock Knowledge Base, and sample BDA projects extend functionality without modifying core or pattern templates, connecting through the same parameter-passing mechanism.
- **Cross-stack communication** occurs through shared S3 buckets, DynamoDB tables, and explicit CloudFormation outputs, enabling unified workflow tracking and UI integration across disparate AI/ML processing paths.

## Frequently Asked Questions

### How does the top-level template decide which pattern stack to deploy?

The top-level [`template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/template.yaml) defines Boolean conditions such as `IsPattern1`, `IsPattern2`, and `IsPattern3` that evaluate the `IDPPattern` parameter value. When you specify `--parameter-overrides IDPPattern=pattern-2`, the `IsPattern2` condition becomes true, causing CloudFormation to create the `PATTERN2STACK` resource of type `AWS::CloudFormation::Stack` while skipping the others. This conditional logic ensures you deploy only the AI/ML pipeline required for your use case.

### What shared resources do all pattern stacks receive from the top-level template?

Every pattern nested stack receives critical infrastructure references through the `Parameters` property of the `AWS::CloudFormation::Stack` resource. These include the `InputBucket` (for document ingestion), `OutputBucket` (for results), `ConfigurationBucket` (for workflow settings), `TrackingTable` (DynamoDB for status tracking), and `CustomerManagedEncryptionKeyArn` (KMS for encryption). By passing these ARNs and names as parameters, the nested stacks can configure IAM policies and environment variables without hardcoding resource identifiers.

### How do the optional nested stacks integrate with the processing patterns?

Optional stacks such as [`nested/appsync/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/nested/appsync/template.yaml) and [`nested/bedrockkb/template.yaml`](https://github.com/aws-solutions-library-samples/accelerated-intelligent-document-processing-on-aws/blob/main/nested/bedrockkb/template.yaml) integrate through the same parameter-passing mechanism used by required patterns. The AppSync stack receives pattern state machine ARNs (e.g., `Pattern1StateMachineArn`) as parameters, allowing GraphQL resolvers to trigger specific workflows. The Bedrock Knowledge Base stack receives the `OutputBucket` parameter, enabling automatic indexing of documents processed by any active pattern. These stacks are conditionally created based on parameters like `ShouldCreateDocumentKnowledgeBase` and `CreateAgentCoreLambda`.

### Can I deploy multiple patterns simultaneously in the same account?

The current architecture supports deploying only one pattern per stack instance due to the `IDPPattern` parameter and mutually exclusive conditions (`IsPattern1`, `IsPattern2`, `IsPattern3`). However, you can deploy multiple instances of the entire solution by using different stack names (e.g., `idp-pattern1-prod` and `idp-pattern2-prod`) in the same AWS account and region. Each instance maintains its own isolated set of nested stacks while sharing no state, though you should verify service quotas for resources like Step Functions and Lambda concurrent executions when running multiple patterns concurrently.