DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports
Events Video Library
Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
View Events Video Library
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Modern Digital Website Security: Prepare to face any form of malicious web activity and enable your sites to optimally serve your customers.

Containers Trend Report: Explore the current state of containers, containerization strategies, and modernizing architecture.

Low-Code Development: Learn the concepts of low code, features + use cases for professional devs, and the low-code implementation process.

E-Commerce Development Essentials: Considering starting or working on an e-commerce business? Learn how to create a backend that scales.

Related

  • How to Detect VPN Proxies With Python and IP2Location.io API
  • Building a Dynamic Chat Application: Setting up ChatGPT in FastAPI and Displaying Conversations in ReactJS
  • Optimizing Coroutine Execution
  • How To Implement Cosine Similarity in Python

Trending

  • Configuring Spark-Submit
  • Transitioning From Point-To-Point to an API-Centric Approach
  • Deployment of Spring MVC App on a Local Tomcat Server
  • Building a Dynamic Chat Application: Setting Up ChatGPT in FastAPI and Displaying Conversations in ReactJS
  1. DZone
  2. Coding
  3. Languages
  4. Serverless and Python: ''Unable to Import Module 'Handler'''

Serverless and Python: ''Unable to Import Module 'Handler'''

If you're a Python fan who enjoys using the Serverless library and virtualenv, you might be running into a dependency error. Here's one solution to the problem.

Mark Needham user avatar by
Mark Needham
·
Updated Aug. 09, 17 · Tutorial
Like (1)
Save
Tweet
Share
30.6K Views

Join the DZone community and get the full member experience.

Join For Free

I’ve been using the Serverless library to deploy and run some Python functions on AWS Lambda recently and was initially confused about how to handle my dependencies.

I tend to create a new virtualenv for each of my projects so let’s get that setup first:

Prerequisites

$ npm install serverless


$ virtualenv -p python3 a
$ . a/bin/activate


Now let’s create our Serverless project. I’m going to install the requests library so that I can use it in my function.

My Serverless Project

serverless.yaml:

service: python-starter-template

frameworkVersion: ">=1.2.0 <2.0.0"

provider:
  name: aws
  runtime: python3.6
  timeout: 180

functions:
  starter-function:
      name: Starter
      handler: handler.starter


handler.py:

import requests

def starter(event, context):
    print("event:", event, "context:", context)
    r = requests.get("http://www.google.com")
    print(r.status_code)


$ pip install requests


Ok, we’re now ready to try out the function. A nice feature of Serverless is that it lets us try out functions locally before we deploy them onto a cloud provider:

$ ./node_modules/serverless/bin/serverless invoke local --function starter-function
event: {} context: <__main__.FakeLambdaContext object at 0x10bea9a20>
200
null


So far so good. Next, we’ll deploy our function to AWS. I’m assuming you’ve already got your credentials set up, but if not, you can follow the tutorial on the Serverless page.

$ ./node_modules/serverless/bin/serverless deploy
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (26.48 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
.........
Serverless: Stack update finished...
Service Information
service: python-starter-template
stage: dev
region: us-east-1
api keys:
  None
endpoints:
  None
functions:
  starter-function: python-starter-template-dev-starter-function


Now let’s invoke our function:

$ ./node_modules/serverless/bin/serverless invoke --function starter-function
{
    "errorMessage": "Unable to import module 'handler'"
}

  Error --------------------------------------------------

  Invoked function failed

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           6.7.0
     Serverless Version:     1.19.0


Hmmm, that’s odd – I wonder why it can’t import our handler module? We can call the logs function to check. The logs are usually a few seconds behind, so we’ll have to be a bit patient if we don’t see them immediately.

$ ./node_modules/serverless/bin/serverless logs  --function starter-function
START RequestId: 735efa84-7ad0-11e7-a4ef-d5baf0b46552 Version: $LATEST
Unable to import module 'handler': No module named 'requests'

END RequestId: 735efa84-7ad0-11e7-a4ef-d5baf0b46552
REPORT RequestId: 735efa84-7ad0-11e7-a4ef-d5baf0b46552Duration: 0.42 msBilled Duration: 100 ms Memory Size: 1024 MBMax Memory Used: 22 MB


That explains it – the requests module wasn’t imported.

If we look in .serverless/python-starter-template.zip, we can see that the requests module is hidden inside the a directory and the instance of Python that runs on Lambda doesn’t know where to find it.

I’m sure there are other ways of solving this but, the easiest one I found is a Serverless plugin called serverless-python-requirements.

So how does this plugin work?

A Serverless v1.x plugin to automatically bundle dependencies from requirements.txt and make them available in your PYTHONPATH.

Doesn’t sound too tricky – we can use pip freeze to get our list of requirements and write them into a file. Let’s rework serverless.yaml to make use of the plugin.

My Serverless Project Using serverless-python-requirements

$ npm install --save serverless-python-requirements


$ pip freeze > requirements.txt
$ cat requirements.txt 
certifi==2017.7.27.1
chardet==3.0.4
idna==2.5
requests==2.18.3
urllib3==1.22


serverless.yaml:

service: python-starter-template

frameworkVersion: ">=1.2.0 <2.0.0"

provider:
  name: aws
  runtime: python3.6
  timeout: 180

plugins:
  - serverless-python-requirements

functions:
  starter-function:
      name: Starter
      handler: handler.starter

package:
  exclude:
    - a/** # virtualenv


We have two changes from before:

  • We added the serverless-python-requirements plugin
  • We excluded the a directory since we don’t need it

Let’s deploy again and run the function:

$ ./node_modules/serverless/bin/serverless deploy
Serverless: Parsing Python requirements.txt
Serverless: Installing required Python packages for runtime python3.6...
Serverless: Linking required Python packages...
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Unlinking required Python packages...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (14.39 MB)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
.........
Serverless: Stack update finished...
Service Information
service: python-starter-template
stage: dev
region: us-east-1
api keys:
  None
endpoints:
  None
functions:
  starter-function: python-starter-template-dev-starter-function


$ ./node_modules/serverless/bin/serverless invoke --function starter-function
null


Looks good. Let’s check the logs:

$ ./node_modules/serverless/bin/serverless logs --function starter-function
START RequestId: 61e8eda7-7ad4-11e7-8914-03b8a7793a24 Version: $LATEST
event: {} context: <__main__.LambdaContext object at 0x7f568b105f28>
200
END RequestId: 61e8eda7-7ad4-11e7-8914-03b8a7793a24
REPORT RequestId: 61e8eda7-7ad4-11e7-8914-03b8a7793a24Duration: 55.55 msBilled Duration: 100 ms Memory Size: 1024 MBMax Memory Used: 29 M


All good here as well, so we’re done!

Python (language)

Published at DZone with permission of Mark Needham, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Related

  • How to Detect VPN Proxies With Python and IP2Location.io API
  • Building a Dynamic Chat Application: Setting up ChatGPT in FastAPI and Displaying Conversations in ReactJS
  • Optimizing Coroutine Execution
  • How To Implement Cosine Similarity in Python

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Core Program
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 3343 Perimeter Hill Drive
  • Suite 100
  • Nashville, TN 37211
  • support@dzone.com

Let's be friends: