Getting Started with Pipenv

If you’re a Python developer, you probably know about pip and the different environment management solutions like virtualenv or venv. The pip tool is currently the standard way to install a Python package. Virtualenv has been a popular way of isolating Python environments for a long time. Pipenv combines the very best of these tools and brings us the one true way to install packages while keeping the dependencies from each project isolated. It claims to have brought the very best from all other packaging worlds (the package manager for other languages / runtimes / frameworks) to the Python world. From what I have seen so far, that claim is quite valid. And it does support Windows pretty well too.

How does Pipenv work?

Pipenv works by creating and managing a virtualenv and a Pipfile for your project. When we install / remove Python packages, the changes are reflected in the Pipfile. It also generates a lock file named Pipfile.lock¬†which is used to lock the version of dependencies and help us produce deterministic builds when needed. In a typical virtualenv¬†setup, we usually create and manage the virtualenv ourselves. Then we activate the environment and pip¬†just installs / uninstalls from that particular virtual environment. Packages like virtualenvwrapper¬†helps us easily create and activate virtualenv¬†with some of it’s handy features. But pipenv¬†takes things further by automating a large part of that. The Pipfile¬†would also make more sense if you have used other packaging systems like Composer, npm, bundler etc.

Getting Started

We need to start by installing pipenv globally. We can install it using pip from PyPi:

pip install pipenv

Now let’s switch to our project directory and try installing a package:

pipenv install flask

When you first run the pipenv install¬†command, you will notice it creates a virtualenv, Pipfile and Pipfile.lock¬†for you. Feel free to go ahead inspect their contents. If you’re using an IDE like PyCharm and want to configure your project interpreter, it would be a good idea to note down the virtualenv¬†path.

Since we have installed Flask, let’s try and running a sample app. Here’s my super simple REST API built with Flask:

from flask import Flask, jsonify

app = Flask(__name__)


@app.route('/')
def hello_world():
    return jsonify({"message": "Hello World!"})

Assuming that you have the FLASK_APP environment variable set to app.py (which contains the above code), we can just run the app like this:

pipenv run flask run

Any executables in the current environment can be run using the pipenv run¬†command. But I know what you might be thinking – we want to do just flask¬†run, not use the entire, long command. That’s easy too. We just need to activate the virtualenv with this command:

pipenv shell

Now you can just do flask run¬†or in fact run any executables in the way we’re used to doing.

Handling Dependencies

We can install and uninstall packages using the install¬†and uninstall¬†commands. When we install a new package, it’s added to our Pipfile¬†and the lock file is updated as well. When we uninstall a package, the Pipfile¬†and the lock files are again updated to reflect the change. The update¬†command uninstalls the packages and installs them again so we have the latest updates.

If you would like to check your dependency graph, just use the graph command which will print out the dependencies in a nice format, kind of like this:

PS C:\Users\Masnun\Documents\Python\pipenvtest> pipenv graph
celery==4.1.0
  - billiard [required: >=3.5.0.2,<3.6.0, installed: 3.5.0.3]
  - kombu [required: >=4.0.2,<5.0, installed: 4.1.0]
    - amqp [required: >=2.1.4,<3.0, installed: 2.2.2]
      - vine [required: >=1.1.3, installed: 1.1.4]
  - pytz [required: >dev, installed: 2017.3]
Flask==0.12.2
  - click [required: >=2.0, installed: 6.7]
  - itsdangerous [required: >=0.21, installed: 0.24]
  - Jinja2 [required: >=2.4, installed: 2.10]
    - MarkupSafe [required: >=0.23, installed: 1.0]
  - Werkzeug [required: >=0.7, installed: 0.12.2]

Pipenv is Awesome!

Trust me, it is! It packs a lot of cool and useful features that can help gear up your Python development workflow. There are just too many to cover in a blog post. I would recommend checking out the Pipenv Docs to get familiar with it more.

Deploying A Flask based REST API to AWS Lambda (Serverless) using Zappa

I have heard about AWS Lambda and all the cool things happening in the serverless world. I have also deployed Go functions using the Apex framework for serverless deployment. But recently I have started working on some Python projects again and decided to see how well the Python community is adapting to the serverless era. Not to my surprise, the Python community is doing great as usual. I quickly found an awesome framework named Zappa which makes deploying Python code to AWS Lambda very easy. Python is already natively supported on the AWS Lambda platform. But with the native support you need to configure the API Gateway, S3 bucket and other stuff on your own. Thanks to Zappa, these things are now automated to our convenience. We can easily deploy WSGI apps as well. That means, we can now take our Flask / Django / API Star apps and deploy them to AWS Lambda – with ease and simplicity. In this blog post, I will quickly walk through how to deploy a flask based rest api to the serverless cloud.

Setup Flask App

Before we can get started, we need to create a simple flask app. Here’s a quick rest api (that doesn’t do much):

from flask import Flask, jsonify

app = Flask(__name__)


@app.route('/')
def hello_world():
    return jsonify({"message": "Hello World!"})

Let’s save the above code in app.py. We can now install Flask using pip:

pip install flask

And then run the code:

FLASK_APP=app.py flask run

This should run our app and we should be able to visit http://127.0.0.1:5000/ to see the output.

Setting Up AWS

Please make sure you have an account for AWS where you have added your credit card and completed the sign up process. AWS Lambda has 1 million free requests per month which is promised to be always free (not for the first 12 months or anything). When you add your card, you shall also be eligible for a free tier of S3 for 12 months. So you won’t be charged for trying out a sample app deployment. So don’t worry about adding a card. In fact, adding a card is a requirement for getting the free tier.

Once you have your AWS account setup, click on your name (top right) and click “My Security Credentials”. From there, choose the “Access Keys” section and generate a new pair of key and secret. Store them in your ~/.aws/credentials¬†file. AWS Cli (and Zappa) will use these to connect to AWS services and perform required actions. The file should look like this:

[default]
aws_access_key_id=[...]
aws_secret_access_key=[...]

[masnun]
aws_access_key_id=[...]
aws_secret_access_key=[...]

I have created two profiles here. Named profiles are useful if you have more than one accounts/projects/environments to work with. After adding the credentials, add the region information in ~/.aws/config:

[default]
region=us-west-2
output=json

[profile masnun]
region=us-east-2
output=text

This will mostly help with choosing the default region for your app. Once you have these AWS settings configured, you can get started with Zappa.

Install and Configure Zappa

First install Zappa:

pip install zappa

Now cd into the project directory (where our flask app is). Then run:

zappa init

Zappa should guide you through the settings it needs. It should also detect the Flask app and auto complete the app (app.app¬†) for you. Once the wizard finishes, you’re ready to deploy your API.

zappa deploy dev

This should now deploy the app to the dev stage. You can configure different stages for your app in Zappa settings. Once you make some code changes, you can update the app:

zappa update

Both of these commands should print out the url for the app. In my case, the url is:¬†https://1gc1f80kb5.execute-api.us-east-2.amazonaws.com/dev¬†ūüôā

What’s Next?

Congratulations, you just deployed a Flask rest api to AWS Lambda using Zappa. You can make the url shorter by pointing a domain to it from your AWS console. To know more about Zappa and all the great things it can do, please check out Zappa on Github.