How to Direct AWS Lambda Logs to a Specific CloudWatch Log Group Using terraform aws_cloudwatch_log_group

Use the aws_cloudwatch_log_group resource to create your target log group, then reference it in the aws_lambda_function resource using the log_group argument (available in AWS provider v5.0+) to override the default "/aws/lambda/" behavior.

When managing serverless infrastructure with HashiCorp Terraform, controlling where AWS Lambda functions emit their execution logs is critical for observability and compliance. The terraform aws_cloudwatch_log_group resource enables you to pre-provision log groups with specific retention policies and naming conventions, while the aws_lambda_function resource can be configured to route logs to these designated groups instead of using auto-generated defaults. This approach ensures your logging infrastructure remains under full Infrastructure-as-Code control within the hashicorp/terraform workflow.

Understanding the Lambda Logging Architecture

By default, AWS Lambda automatically creates a CloudWatch Log Group named /aws/lambda/<function-name> on first invocation. While convenient, this default behavior prevents you from setting custom retention policies or naming conventions before the function executes. According to the hashicorp/terraform source code in internal/configs/configs.go, resource arguments like log_group are parsed as explicit configuration values that override runtime defaults. When you specify a log_group in the aws_lambda_function resource, Terraform delegates the log routing configuration to the AWS provider, which configures the Lambda runtime to use your pre-created destination.

Creating the CloudWatch Log Group Resource

Before associating a log group with your Lambda function, you must explicitly create it using the aws_cloudwatch_log_group resource. This resource, defined in the AWS provider schema referenced by internal/provider/registry/resource_schema.go, accepts parameters for name, retention, and KMS encryption.

resource "aws_cloudwatch_log_group" "lambda_logs" {
  name              = "/aws/lambda/my-custom-log-group"
  retention_in_days = 30
  
  tags = {
    Environment = "production"
    ManagedBy   = "terraform"
  }
}

Configuring aws_lambda_function to Use a Specific Log Group

To force the Lambda function to write to your specific log group rather than the default, use the log_group argument in the aws_lambda_function resource. As implemented in the AWS provider documentation at website/docs/r/lambda_function.html.markdown, this parameter (available from provider version 5.0 onward) accepts the name of an existing CloudWatch Log Group.

resource "aws_lambda_function" "my_function" {
  function_name = "my-function"
  role          = aws_iam_role.lambda_exec.arn
  handler       = "index.handler"
  runtime       = "nodejs18.x"
  filename      = "lambda.zip"

  log_group = aws_cloudwatch_log_group.lambda_logs.name
}

Terraform's dependency resolution logic, implemented in internal/plans/plan.go, automatically establishes that the Lambda function depends on the CloudWatch Log Group creation, ensuring proper resource ordering during apply operations.

IAM Permissions for CloudWatch Logging

The Lambda execution role must have permission to create log streams and put log events. While the AWS-managed policy AWSLambdaBasicExecutionRole typically suffices, custom roles require explicit permissions as shown in website/docs/r/cloudwatch_log_group.html.markdown.

data "aws_iam_policy_document" "lambda_logging" {
  statement {
    effect = "Allow"
    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents"
    ]
    resources = [
      aws_cloudwatch_log_group.lambda_logs.arn,
      "${aws_cloudwatch_log_group.lambda_logs.arn}:*"
    ]
  }
}

resource "aws_iam_role_policy" "lambda_logs" {
  name   = "lambda-logging-policy"
  role   = aws_iam_role.lambda_exec.id
  policy = data.aws_iam_policy_document.lambda_logging.json
}

Advanced Patterns for terraform aws_cloudwatch_log_group

Sharing Log Groups Across Multiple Functions

Multiple Lambda functions can route logs to a single CloudWatch Log Group by referencing the same aws_cloudwatch_log_group resource in each function's log_group argument. This pattern centralizes logs for microservices or related application components.

resource "aws_cloudwatch_log_group" "shared_logs" {
  name              = "/aws/lambda/shared-service-logs"
  retention_in_days = 14
}

resource "aws_lambda_function" "func_a" {
  function_name = "function-a"
  role          = aws_iam_role.lambda_exec.arn
  handler       = "a.handler"
  runtime       = "python3.11"
  filename      = "a.zip"
  log_group     = aws_cloudwatch_log_group.shared_logs.name
}

resource "aws_lambda_function" "func_b" {
  function_name = "function-b"
  role          = aws_iam_role.lambda_exec.arn
  handler       = "b.handler"
  runtime       = "python3.11"
  filename      = "b.zip"
  log_group     = aws_cloudwatch_log_group.shared_logs.name
}

Preventing Accidental Log Group Deletion

Protect critical log groups from accidental destruction using lifecycle rules. This guards against data loss when modifying or removing Lambda functions.

resource "aws_cloudwatch_log_group" "critical_logs" {
  name              = "/aws/lambda/critical-function"
  retention_in_days = 365

  lifecycle {
    prevent_destroy = true
  }
}

Handling Existing Log Groups

To avoid conflicts with manually created or pre-existing log groups, use conditional creation with count or for_each arguments, which Terraform processes through the configuration parser in internal/configs/configs.go.

variable "create_log_group" {
  default = true
  type    = bool
}

resource "aws_cloudwatch_log_group" "conditional" {
  count = var.create_log_group ? 1 : 0
  
  name              = "/aws/lambda/conditional-function"
  retention_in_days = 7
}

Summary

• Pre-create log groups using aws_cloudwatch_log_group to enforce naming conventions and retention policies before Lambda invocation.
• Reference explicitly using the log_group argument in aws_lambda_function (AWS provider v5.0+) to override the default "/aws/lambda/" behavior.
• Manage dependencies through Terraform's implicit dependency resolution in internal/plans/plan.go, ensuring log groups exist before Lambda functions reference them.
• Secure with IAM by attaching policies that grant logs:CreateLogStream and logs:PutLogEvents on the specific log group ARN.
• Protect data using lifecycle { prevent_destroy = true } on critical log groups to prevent accidental deletion.

Frequently Asked Questions

Can I use an existing CloudWatch Log Group created outside Terraform?

Yes. Use the data "aws_cloudwatch_log_group" data source to reference an existing log group by name, then pass its name attribute to the Lambda function's log_group argument. This avoids creating duplicate resources while still allowing Terraform to manage the function configuration.

What happens if I don't specify the log_group argument in aws_lambda_function?

If omitted, AWS Lambda automatically creates a log group named /aws/lambda/<function-name> on its first execution. This default group uses unlimited retention (never expires) and cannot be configured until after creation, potentially violating compliance requirements or cost optimization strategies.

How do I handle log retention policies in terraform aws_cloudwatch_log_group?

Set the retention_in_days argument to automatically expire old log events. Valid values range from 1 to 365 days, or 0 to retain indefinitely (not recommended for production). Changes to this attribute update the existing log group in-place without recreation.

Is the log_group argument available in all versions of the AWS provider?

No. The log_group argument in aws_lambda_function requires AWS provider version 5.0.0 or later. Earlier versions require alternative approaches such as naming the aws_cloudwatch_log_group resource to match the expected default path /aws/lambda/<function-name> or using environment variables to configure custom logging endpoints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}