# Highly Available and Resilient Multi-AWS VPC Architectures: Design Patterns for Private Subnets and Inter-VPC Communication > Learn key design patterns for highly available, resilient multi-AWS VPC architectures. Discover best practices for private subnets and inter-VPC communication using Transit Gateway and NAT Gateways. - Repository: [Arie Bregman/devops-exercises](https://github.com/bregman-arie/devops-exercises) - Tags: architecture - Published: 2026-02-28 --- **To build highly available and resilient multi-AWS VPC architectures, deploy workloads in private subnets with NAT Gateways per Availability Zone, use Transit Gateway for scalable inter-VPC communication, and distribute traffic across multiple AZs using load balancers and Route 53 health checks.** According to the `bregman-arie/devops-exercises` repository, enterprise-grade AWS networking requires isolating compute resources while maintaining robust connectivity between VPCs. The source code analysis reveals specific patterns for implementing fault-tolerant infrastructure that survives individual Availability Zone failures and supports complex multi-VPC topologies. ## Private Subnets and NAT Gateways: The Foundation of High Availability ### Isolating Workloads in Private Subnets Place all workload instances in **private subnets** to ensure they are never exposed directly to the Internet. As documented in [`topics/cloud/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/cloud/README.md), instances should be accessed exclusively through load balancers or bastion hosts, remaining "off the internet (in a private subnet behind a NAT)." This approach eliminates public IP exposure while permitting necessary outbound traffic for updates and package downloads. ### NAT Gateway Distribution Across Availability Zones Deploy **one NAT Gateway per Availability Zone** to eliminate single points of failure for egress traffic. This pattern ensures that if one AZ becomes unavailable, instances in private subnets of surviving AZs maintain outbound connectivity through their local NAT Gateway. The repository emphasizes that distributing NAT Gateways across multiple AZs prevents a failure in one zone from cutting off egress for the entire VPC. ## High-Availability Networking Primitives ### Internet Gateway Constraints and Configuration Each VPC can attach to **only one Internet Gateway (IGW)**, as explicitly noted in [`topics/aws/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/aws/README.md) (lines 84-86). While this creates a single point of egress for public traffic, you mitigate risks by deploying public-facing components across multiple AZs behind redundant load balancers. The IGW attaches to the VPC and routes public subnet traffic, but never directly to private subnet instances. ### Load Balancer Distribution and Auto-Scaling Deploy **multiple Application Load Balancers (ALBs) or Network Load Balancers (NLBs)** across Availability Zones, registering targets exclusively from private subnets. Combine this with **EC2 Auto-Scaling groups** configured with a minimum of two instances across at least two AZs. This configuration guarantees capacity during AZ failures and provides DNS-based failover through **Route 53 health-checked alias records** that route traffic to healthy endpoints, with fallback to secondary regions when necessary. ### Security Groups versus Network ACLs Implement layered security using **Security Groups** for instance-level allow-lists and **Network ACLs (NACLs)** for subnet-level deny/allow boundaries. The repository distinguishes these concepts in [`topics/aws/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/aws/README.md) (lines 55-60), recommending Security Groups for stateful firewall rules specific to individual instances and NACLs for broader subnet traffic policies. ## Inter-VPC Communication Patterns ### VPC Peering for Direct Connectivity Use **VPC Peering** for simple, same-region connectivity between two VPCs. This creates a direct routing path for private IPv4/IPv6 traffic, but note the critical limitation documented in [`topics/aws/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/aws/README.md) (lines 96-99): peering maintains a **one-to-one relationship** with **no transitive routing** support. If VPC A peers with VPC B, and VPC B peers with VPC C, VPC A cannot reach VPC C through B. ```hcl resource "aws_vpc_peering_connection" "peer_ab" { vpc_id = aws_vpc.vpc_a.id peer_vpc_id = aws_vpc.vpc_b.id auto_accept = true tags = { Name = "A-to-B-Peering" } } resource "aws_route" "a_to_b" { route_table_id = aws_vpc.vpc_a.main_route_table_id destination_cidr_block = aws_vpc.vpc_b.cidr_block vpc_peering_connection_id = aws_vpc_peering_connection.peer_ab.id } ``` ### AWS Transit Gateway for Scalable Hub-and-Spoke Implement **AWS Transit Gateway** when managing many VPCs (per-environment or per-team configurations) requiring **transitive routing**. This centralized hub connects dozens of VPCs and on-premises networks through a single routing table, supporting **bandwidth-based routing** and **inter-region peering** for cross-region high availability. Unlike VPC Peering, Transit Gateway allows VPC A to communicate with VPC C through the hub without direct peering between them. ```hcl resource "aws_ec2_transit_gateway" "core" { description = "Core TGW for multi-VPC hub" amazon_side_asn = 64512 } resource "aws_ec2_transit_gateway_vpc_attachment" "vpc_a" { transit_gateway_id = aws_ec2_transit_gateway.core.id vpc_id = aws_vpc.vpc_a.id subnet_ids = [aws_subnet.public_a.id, aws_subnet.public_b.id] } ``` ### AWS PrivateLink for Service-Oriented Access Expose specific services (RDS instances, internal APIs) via **Interface VPC Endpoints** using **AWS PrivateLink**. This pattern keeps traffic within the AWS network, eliminating the need for VPC peering for service consumption while maintaining strict network isolation. ### Shared Services VPC Architecture Create a **dedicated Shared Services VPC** hosting centralized resources such as NAT Gateways, shared DNS resolvers, and security-inspection appliances. Workload VPCs attach to this hub via Transit Gateway or peering, obtaining a **single point for egress/NAT** and **centralized security policy enforcement**. This design reduces operational overhead by consolidating shared infrastructure into one managed network boundary. ## Operational Considerations for Resilience ### Subnet Sizing and IP Address Management AWS reserves **five IP addresses** per subnet (network address, broadcast address, and three for AWS internal use). The repository references the "Kratos" example in [`topics/aws/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/aws/README.md) (lines 77-81), emphasizing the need to allocate sufficient CIDR blocks to accommodate future growth without requiring complex re-addressing. ### Monitoring and Failover Testing Enable **VPC Flow Logs** and **CloudWatch Alarms** on NAT Gateway and Transit Gateway health metrics. Implement **IAM least-privilege policies** granting only required permissions for IGW and NAT management. Regularly simulate AZ failures by stopping instances in private subnets to verify traffic continues routing through remaining AZs and NAT Gateways. ## Infrastructure as Code Examples ### Terraform VPC with Multi-AZ NAT Gateways The [`topics/terraform/exercises/vpc_subnet_creation/solution.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/terraform/exercises/vpc_subnet_creation/solution.md) file provides a complete implementation of a highly available VPC with public and private subnets across multiple Availability Zones: ```hcl resource "aws_vpc" "main" { cidr_block = "10.0.0.0/16" tags = { Name = "multi-az-vpc" } } resource "aws_subnet" "public_a" { vpc_id = aws_vpc.main.id cidr_block = "10.0.0.0/24" availability_zone = "us-east-1a" map_public_ip_on_launch = true } resource "aws_subnet" "public_b" { vpc_id = aws_vpc.main.id cidr_block = "10.0.1.0/24" availability_zone = "us-east-1b" map_public_ip_on_launch = true } resource "aws_subnet" "private_a" { vpc_id = aws_vpc.main.id cidr_block = "10.0.10.0/24" availability_zone = "us-east-1a" } resource "aws_subnet" "private_b" { vpc_id = aws_vpc.main.id cidr_block = "10.0.11.0/24" availability_zone = "us-east-1b" } resource "aws_eip" "nat_a" { vpc = true } resource "aws_nat_gateway" "nat_a" { allocation_id = aws_eip.nat_a.id subnet_id = aws_subnet.public_a.id } resource "aws_eip" "nat_b" { vpc = true } resource "aws_nat_gateway" "nat_b" { allocation_id = aws_eip.nat_b.id subnet_id = aws_subnet.public_b.id } resource "aws_route_table" "private_a" { vpc_id = aws_vpc.main.id route { cidr_block = "0.0.0.0/0" nat_gateway_id = aws_nat_gateway.nat_a.id } } resource "aws_route_table_association" "private_a" { subnet_id = aws_subnet.private_a.id route_table_id = aws_route_table.private_a.id } ``` This configuration establishes redundant egress paths, ensuring private subnet instances in `us-east-1a` route through `nat_a` while instances in `us-east-1b` use `nat_b`. ## Summary - **Private subnet isolation** is mandatory for secure workloads; combine with **NAT Gateways per AZ** to ensure highly available outbound connectivity without public IP exposure. - **AWS Transit Gateway** provides the only scalable solution for transitive routing between multiple VPCs, while **VPC Peering** suffices for simple two-VPC scenarios. - **Load balancers spanning multiple AZs** with **Route 53 health checks** deliver automatic failover capabilities for public-facing services. - **Internet Gateways** are limited to one per VPC, requiring careful architecture of public subnets to avoid congestion points. - **Subnet sizing** must account for AWS reserving five IP addresses per CIDR block, as documented in the repository's AWS exercises. - **Shared Services VPCs** centralize common infrastructure like NAT and DNS, reducing operational complexity across large multi-VPC estates. ## Frequently Asked Questions ### What is the difference between VPC Peering and Transit Gateway for inter-VPC communication? VPC Peering creates a direct, one-to-one network connection between exactly two VPCs with no transitive routing capability, meaning traffic cannot flow through an intermediary VPC to reach a third. Transit Gateway acts as a centralized hub that supports transitive routing, allowing dozens of VPCs to communicate through a single attachment point. According to [`topics/aws/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/aws/README.md) (lines 96-99), choose peering for simple two-VPC scenarios and Transit Gateway when managing complex multi-VPC architectures requiring many-to-many connectivity. ### Why should I deploy a NAT Gateway in every Availability Zone? Deploying one NAT Gateway per AZ eliminates a single point of failure for outbound internet traffic from private subnets. If an AZ experiences an outage, only the NAT Gateway in that specific zone fails; instances in private subnets of healthy AZs continue routing egress traffic through their local NAT Gateway. This pattern ensures your highly available and resilient multi-AWS VPC architecture maintains internet connectivity (for updates, patches, and external API calls) even during partial infrastructure failures. ### How do Security Groups and Network ACLs work together in a multi-VPC architecture? Security Groups operate at the instance level as stateful firewalls, automatically allowing return traffic for established connections. Network ACLs function at the subnet level as stateless filters, evaluating both inbound and outbound rules independently. The repository's [`topics/aws/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/aws/README.md) (lines 55-60) recommends using Security Groups for granular, instance-specific access controls while employing NACLs for broad subnet-level security boundaries, creating a defense-in-depth strategy across your VPC infrastructure. ### What are the IP addressing considerations when designing subnets for high availability? AWS reserves five IP addresses in every subnet: the network address, broadcast address, and three addresses for AWS internal use (router, DNS, and future use). When planning CIDR blocks for highly available architectures spanning multiple AZs, allocate sufficiently large subnets to accommodate future scaling without exhausting available IPs. The repository references this consideration in [`topics/aws/README.md`](https://github.com/bregman-arie/devops-exercises/blob/main/topics/aws/README.md) (lines 77-81) within the "Kratos" example, emphasizing that re-addressing production subnets requires significant downtime and should be avoided through proper initial sizing.