Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

Demystifying Lambda in VPC and Its Confusing Error

DZone 's Guide to

Demystifying Lambda in VPC and Its Confusing Error

Without understanding how the permissions work with Lambda running on a VPC, you could encounter this unusual error.

· Cloud Zone ·
Free Resource

Suddenly, my AWS Lambda function stopped working. Upon invocation, not a single line of code was executing, and I was just getting this error from the console:

"Calling the invoke API action failed with this message: Lambda was not able to access EC2's API using the Lambda Execution Role to set up the Lambda function."

This problem was confusing because:

  • My function’s code was not at all interacting with EC2.

  • My function was working properly a few hours ago. I didn’t change anything after successful invocations!

  • Googling the error message didn't help. This error message is rarely mentioned and even those search results weren’t helpful

This was the problem I encountered a few days ago and I would like to share the result with you to save your time from troubleshooting and to help you understand better what happens behind the scenes for Lambda within VPC.

It’s known that, currently, Lambda functions within VPC require Elastic Network Interface (ENI)*. The Lambda environment doesn’t let you create a new function within a VPC if it doesn’t have enough permission to interact with ENIs. So there is already a verification helping the user to avoid making a  mistake. But, I encountered the issue because I reduced my function’s ENI-related permissions, and the reduction caused the confusion. Here is what happens behind the scenes:

  1. When a user tries to create a Lambda function within VPC, the Lambda environment verifies whether the function has enough permission to create a new ENI. If it doesn’t have enough permission, Lambda returns an error and prevents the creation of a new function. Otherwise, Lambda creates a new function. Let's call this the initial verification.

  2. After the initial verification and once the new function is created, if the user reduces function's ENI-related permission, he doesn't get notified. So, a user can save the function’s new settings without any problem and will not see any errors.

  3. Once the new function is created and invoked, an ENI gets established; without invocation, no ENI gets established. The ENI is available for an uncertain amount of time, so even if the user reduces the permission (by updating the function), the function still works for a while. But after some uncertain amount of inactivity, ENI gets removed and the user can't invoke the function anymore, unless the user grants minimum permission.

To fix the problem, the user needs to grant enough permission to the function’s execution role enabling it to create, describe, and delete ENIs. An easy way is to add the policy AWSLambdaVPCAccessExecutionRole to function’s execution role. The policy is as follows: 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateNetworkInterface",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DeleteNetworkInterface"
            ],
            "Resource": "*"
        }
    ]
}


* Note: The current architecture will change in the near future.

Topics:
aws ,aws lambda ,lambda ,serverless ,faas ,lambda function ,cloud ,error ,permissions ,eni

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}