How to Fix Terraform Permission Denied on AWS Lambda

Troubleshooting “Terraform Permission Denied” on AWS Lambda

As a Senior DevOps Engineer, encountering “Terraform Permission Denied” is a common roadblock when provisioning AWS resources. When this error surfaces specifically with AWS Lambda, it often points to a clear, albeit sometimes nuanced, issue with IAM permissions. This guide will walk you through diagnosing and resolving such issues effectively.

1. The Root Cause: Why This Happens on AWS Lambda

When Terraform attempts to provision or modify an AWS Lambda function, it acts on behalf of an AWS Identity and Access Management (IAM) entity – either an IAM User (via access keys) or an IAM Role (assumed by a CI/CD pipeline, an EC2 instance, etc.). The “Permission Denied” error means this IAM entity lacks one or more necessary permissions to perform the requested AWS API calls to manage Lambda resources.

The key distinction to understand is between:

  • Permissions for the Terraform Executor: The IAM User or Role running terraform apply. This entity needs permissions to create, update, or delete Lambda functions, their associated roles, and other related resources.
  • Permissions for the Lambda Execution Role: The IAM Role that the Lambda function itself assumes when it runs. This role dictates what AWS services the Lambda function can interact with (e.g., CloudWatch Logs, S3, DynamoDB). Terraform needs permission to pass this role to the Lambda service.

Common Missing Permissions (for the Terraform Executor):

  1. lambda:* actions: For creating, updating, or deleting functions (e.g., lambda:CreateFunction, lambda:UpdateFunctionConfiguration, lambda:DeleteFunction).
  2. iam:CreateRole, iam:AttachRolePolicy, iam:PutRolePolicy, iam:DeleteRole: If Terraform is also managing the Lambda function’s execution role and its associated policies.
  3. iam:PassRole: This is a frequent culprit. The IAM entity running Terraform must have iam:PassRole permissions for the specific IAM Role that you are assigning to the Lambda function. This prevents unauthorized entities from assigning highly privileged roles to services they control.
  4. s3:GetObject: If your Lambda function’s code is being sourced from an S3 bucket (e.g., s3_bucket and s3_key arguments in aws_lambda_function), the Terraform executor needs permission to read that object.
  5. cloudwatch:* or logs:*: If Terraform explicitly manages CloudWatch Log Groups for Lambda. (Often, Lambda implicitly creates a basic log group, but explicit management requires permissions.)

2. Quick Fix (CLI)

Before diving into detailed configuration, a quick CLI check can often pinpoint the issue, especially for the iam:PassRole permission.

Steps:

  1. Identify the IAM Entity: Determine which IAM User or Role is executing your terraform apply command. This is crucial.

    • If running locally: It’s usually the IAM User configured in your ~/.aws/credentials or via environment variables.
    • If in CI/CD: It’s the IAM Role assumed by your pipeline runner (e.g., GitHub Actions OIDC role, CodeBuild service role, EC2 instance profile).

  2. Check Current Permissions (Example for an IAM User):

    # Replace <IAM_USERNAME> with the actual username
aws iam list-attached-user-policies --user-name <IAM_USERNAME>
aws iam list-user-policies --user-name <IAM_USERNAME> # For inline policies

    If it’s an IAM Role:

    # Replace <IAM_ROLENAME> with the actual role name
aws iam list-attached-role-policies --role-name <IAM_ROLENAME>
aws iam list-role-policies --role-name <IAM_ROLENAME> # For inline policies

    You’ll need to then examine the JSON content of these policies.

  3. Temporarily Add Missing Permissions (for iam:PassRole example):

    • Identify the Lambda Execution Role ARN: Look in your Terraform code for the aws_iam_role resource that defines your Lambda function’s execution role. Get its ARN.
    • Create a temporary policy: 
      {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "<ARN_OF_LAMBDA_EXECUTION_ROLE>"
        }
    ]
}
    • Attach the policy to your Terraform Executor (e.g., IAM User): 
      # Save the JSON above to a file like temp_passrole_policy.json
aws iam put-user-policy --user-name <IAM_USERNAME> --policy-name TempPassRolePolicy --policy-document file://temp_passrole_policy.json
      Self-Correction: For production, attach an IAM Managed Policy or modify an existing one, rather than using inline policies or temporary fixes. This quick fix is for diagnosis.

  4. Re-run terraform plan: If the error changes or disappears, you’ve likely identified the missing permission.

Important: Always adhere to the principle of least privilege. Once you identify the specific missing permissions, refine them to grant only what’s necessary.

3. Configuration Check

A systematic review of your Terraform code and AWS IAM configurations will ensure long-term stability.

3.1. Terraform Code (.tf files)

Focus on the aws_lambda_function resource and any related IAM resources.

  1. aws_lambda_function resource:

    • function_name: Is it unique and valid? (Not a permission issue, but good to check).
    • handler: Correctly specified?
    • runtime: Supported?
    • role: This is critical. It must reference the ARN of the IAM Role that your Lambda function will assume at runtime. 
      resource "aws_lambda_function" "my_lambda" {
  function_name = "my-awesome-lambda"
  # ... other configurations ...
  role          = aws_iam_role.lambda_exec_role.arn
  # ...
}
    • s3_bucket, s3_key: If specified, ensure these point to a valid S3 object.

  2. aws_iam_role for Lambda Execution:

    • name: Ensure it’s descriptive.
    • assume_role_policy: This policy grants the Lambda service permission to assume this role. It must allow lambda.amazonaws.com to assume the role. 
      resource "aws_iam_role" "lambda_exec_role" {
  name = "my-lambda-execution-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      },
    ]
  })
}

  3. aws_iam_role_policy_attachment or aws_iam_policy (for Lambda Execution Role):

    • Ensure the lambda_exec_role has policies attached that grant it access to necessary resources (e.g., AWSLambdaBasicExecutionRole for CloudWatch Logs, S3, DynamoDB, etc.). While this doesn’t cause a “Terraform Permission Denied” error during provisioning, it’s good practice to ensure the role itself is correctly configured.

3.2. AWS IAM Configuration (Console / CLI)

Verify the permissions of the Terraform Executor (the IAM User or Role running Terraform).

  1. Explicit Denies: Check for any explicit Deny statements in attached policies. These override Allow statements.
  2. Wildcard Permissions: While * is convenient, it’s dangerous. Ensure any necessary wildcard permissions (lambda:*, iam:*) are intentionally granted, perhaps temporarily for debugging, and then narrowed down.
  3. Policy Evaluation Order: AWS evaluates policies based on Deny > Allow > Default Deny. Understand that a Deny in one policy will always win.
  4. iam:PassRole Scope: Double-check that the iam:PassRole permission (if granted) specifies the correct Resource (the ARN of the Lambda execution role). It should not be overly broad if possible.

4. Verification

After making changes, follow these steps to verify your fix.

  1. Run terraform plan:

    terraform plan

    This is your first line of defense. terraform plan communicates with AWS to determine the desired state and often catches permission issues before attempting to apply changes. Look for the “Plan: X to add, Y to change, Z to destroy” output without any “Permission Denied” errors.

  2. Run terraform apply:

    terraform apply

    If terraform plan executes successfully, terraform apply will attempt to provision the resources. Monitor the output closely. If the permission error re-appears, carefully read the exact error message as it may have changed, indicating a different missing permission.

  3. Verify in AWS Console:

    • Navigate to the Lambda service. Check if your function (my-awesome-lambda) has been created or updated.
    • Click on the function and go to the Configuration tab, then Permissions. Verify that the correct “Execution role” is assigned and that it points to the my-lambda-execution-role you defined.
    • Navigate to the IAM service. Find your my-lambda-execution-role. Check its “Permissions” tab to ensure it has the necessary policies for its runtime operations (e.g., AWSLambdaBasicExecutionRole).
    • Finally, locate the IAM User or Role that executed Terraform and confirm its policies.

By systematically addressing each point, you’ll not only resolve the “Terraform Permission Denied” error but also gain a deeper understanding of AWS IAM and Terraform’s interaction with Lambda. Remember to prioritize least privilege and regularly review your IAM policies.

⚡ Related Troubleshooting Guides