Author: masnun

Django: Building REST APIs

Recently, I have written a few pieces on REST API development. We have discussed the Fundamentals of REST APIs, Built a Simple REST API, Secured it with HTTP Basic Authentication and JSON Web Tokens. However, the example codes were all based on Flask. I chose Flask for those examples because the framework was minimal in it’s core, light weight yet popular and with a matured eco system – very well suited for demonstrating simple REST APIs. But if we consider popularity, probably no other framework is as popular as Django in the Python land. In my day to day work, I use the framework quite extensively. Over the years, I have grown to become a passionate fan of Django. Naturally a tutorial series on building REST APIs with Django was just a matter of time.

Tutorial Index

This is the first part of the series. I shall update the index here as I keep adding new contents.

Code: https://github.com/masnun/djmailer

Prerequisites

The tutorial series will be heavily focused on building REST APIs. So there will be less scope of discussing the fundamentals of Django as we go. I would expect the reader to have decent familiarity of Django before hand. If you are new to Django, please spend some time with the Official Tutorial. It is very well written.

The tutorial series will use Python 3.6.0 (but any versions above Python 3.0 should work fine). As for Django, I have just installed the latest release as of today –  1.11. . We would also be using Django REST Framework, I have 3.6.3 right now. The example code repository will have an up-to-date requirements.txt file – so don’t worry much about dependencies or their versions right now. Just make sure you use Python 3 and not Python 2.

Setting Up Django

Before we can start working on our awesome REST API, we first need to setup and configure our Django project. We need to install Django, install dependencies, start a project and an app, configure the database connection and write our first view. So what are we waiting for? Let’s get started!

Install Django and Dependencies

For now, we need the latest Django and a database driver. We will be using MySQL since it’s very popular and most people are familiar with it. In our production systems though, we mostly run PostgreSQL. We will discuss that choice in some later posts. Let’s install Django and the MySQL client using pip.

pip install django
pip install mysqlclient

Done? Cool. If Django installed successfully, you should be able to run the django-admin.py command. See if you can run it. If the command runs successfully, your installation worked. Time to move on to the next phase.

Create the Django Project

We want to build REST APIs for a mailing list. Let’s call it djmailer. We use the startproject admin command to start our project.

django-admin startproject djmailer

We would now have a directory named djmailer created in the current directory. Within that, there’s the default djmailer app directory which contains the settings and root urlconf and the uwsgi app.

Create a New App

For the API related stuff, we would now create a new app named api. Let’s do that. First cd into the project so you  can access the manage.py file. Now run this command:

python manage.py startapp api

Cool, now we have the api app created for us. Open up djmailer/settings.py and add api to the INSTALLED_APPS list.

Configure Database

In the default djmailer app, there’s a file named settings.py. Open it up in your favorite text editor. We need to add our database credentials here. We will notice that a sample database configuration is already generated for us. Something like:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

The above example uses SQLite. But we want to use MySQL. So first create a database named djmailer on your MySQL server and create the database user and password. Then replace the above code with these lines:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'djmailer',
        'USER': 'djmailer',
        'PASSWORD': 'suP3rS3CR3t!X$%^',
    }
}

Once you have configured the database, please run the migrate management command.

python manage.py migrate

This should create the tables Django needs for itself.

Hello World!

Time to create our first view. Open api/views.py file and create the view:

from django.http.response import JsonResponse


def hello_world(request):
    return JsonResponse({"message": "hello world!"})

Instead of the usual HttpResponse, we used JsonResponse because we want to render our response as JSON. Using this class ensures that the reponse is JSON and the appropriate content type is set.

We can add this view directly to the root urlconf in djmailer/urls.py but we would like to better organize our urls. So we are going to create a new file named urls.py in the api directory and put the following content:

from django.conf.urls import url

from .views import hello_world

urlpatterns = [
    url(r'^hello', hello_world, name="hello_world")
]

If we add urls from all apps to the root urlconf, it will soon become a mess. Also the apps will not be reusable. But if we keep our app specific routes inside the app, we can just import them from the root urlconf. This keeps things clean and the app can be easily plugged into a different project if needed.

Let’s edit djmailer/urls.py (the root urlconf) and import our api urls.

from django.conf.urls import url, include
from django.contrib import admin

from api import urls as api_urls

urlpatterns = [
    url(r'^api/', include(api_urls, namespace="api")),
    url(r'^admin/', admin.site.urls),
]

Focus on line 4 and 7. We are importing our api urls and putting them under the /api/ path. Run the django dev server:

python manage.py runserver

If we visit `http://localhost:8000/api/hello`, we would see the response:

{"message": "hello world!"}

Cool? We just built our first RESTful view with Django.

Introducing Django REST Framework (DRF)

Building REST APIs with plain old Django is very possible but we have to do a lot. We need to read incoming requests, parse them as JSON or XML (through content negotiation), in case of CRUD operations, we also have to work with models and finally we have to send back the response in appropriate format. It’s all possible in Django – but like I said, I have to do so many things on our own. Our initial hello world view was very basic, very simple. As the project grows and we have more and more complexities, things will start to get out of hands.

In our Flask examples, using a third party extension helped us get started faster and saved us much time and efforts. Django REST Framework is one such framework for Django. It provides so much functionality out of the box – I am a big fan of the framework. So while we can do everything without this framework, using it would make us productive and keep things sane.

So from now on, we would be using DRF to continue building our APIs. In the next blog post, we would be getting started with Django REST Framework.

What’s Next?

On our next post – Django REST Framework: Getting Started, we introduce you to the wonderful world of DRF and demonstrate how we can use APIView to build our endpoints.

In the mean time, please subscribe to our mailing list. We will keep you posted when we publish new content. 🙂

JWT Authentication with Python and Flask

In our blog post about HTTP Authentication, we promised we would next cover JSON Web Tokens aka JWT based authentication. So we wrote a detailed blog post on The Concepts of JWT explaining how the technology works behind the scene. And in this blog post, we would see how we can actually implement it in our REST API. In case you have missed them, we have also explained the basics of REST APIs  along with a Python / Flask tutorial walking through some of the best practices.

PyJWT or a Flask Extension?

In our last blog post on JWT, we saw code examples based on the PyJWT library. A quick Google search also revealed a couple of Flask specific libraries. What do we use?

We can implement the functionality with PyJWT alright. It will allow us fine grained control. We would be able to customize every aspect of how the authentication process works. On the other hand, if we use Flask extensions, we would need to do less since these libraries or extensions already provide some sort of integrations with Flask itself. Also personally, I tend to choose my framework specific libraries for a task. They reduce the amount of task required to get things going.

In this blog post, we would be using the Flask-JWT package.

Getting Started

Before we can begin, we have to install the package using pip.

pip install Flask-JWT

We also need an API end point that we want to secure. We can refer to the initial code we wrote for our HTTP Auth tutorial.

from flask import Flask
from flask_restful import Resource, Api

app = Flask(__name__)
api = Api(app, prefix="/api/v1")


class PrivateResource(Resource):
    def get(self):
        return {"meaning_of_life": 42}


api.add_resource(PrivateResource, '/private')

if __name__ == '__main__':
    app.run(debug=True)

Now we work on securing it 🙂

Flask JWT Conventions

Flask JWT has the following convention:

  • There needs to be two functions – one for authenticating the user, this would be quite similar to the verify function we wrote in our last tutorial (http auth tutorial). The second function’s job is to identify user from a token. Let’s call this function identity.
  • The authentication function must return an object instance that has an attribute named id.
  • To secure an endpoint, we use the @jwt_required decorator.
  • An API endpoint is setup at /auth that accepts username and password via JSON payload and returns access_token which is the JSON Web Token we can use.
  • We must pass the token as part of the Authorization header, like – JWT <token>.

Authentication and Identity

First let’s write the function that will authenticate the user. The function will take in username and password and return an object instance that has the id attribute. In general, we would use database and the id would be user id. But for this example, we would just create an object with an ID of our choice.

USER_DATA = {
    "masnun": "abc123"
}


class User(object):
    def __init__(self, id):
        self.id = id

    def __str__(self):
        return "User(id='%s')" % self.id


def verify(username, password):
    if not (username and password):
        return False
    if USER_DATA.get(username) == password:
        return User(id=123)

We are storing the user details in a dictionary like before. We have created User class with id attribute so we can fulfil the requirement of having id attribute. In our verify function, we compare the username and password and if it matches, we return an User instance with the id being 123. We will use this function to verify user logins.

Next we need the identity function that will give us user details for a logged in user.

def identity(payload):
    user_id = payload['identity']
    return {"user_id": user_id}

The identity function will receive the decoded JWT. An example would be like:

{'exp': 1494589408, 'iat': 1494589108, 'nbf': 1494589108, 'identity': 123}

Note the identity key in the dictionary. It’s the value we set in the id attribute of the object returned from the verify function. We should load the user details based on this value. But since we are not using the database, we are just constructing a simple dictionary with the user id.

Securing Endpoint

Now that we have a function to authenticate and another function to identify the user, we can start integrating Flask JWT with our REST API. First the imports:

from flask_jwt import JWT, jwt_required

Then we construct the jwt instance:

jwt = JWT(app, verify, identity)

We pass the flask app instance, the authentication function and the identity function to the JWT class.

Then in the resource, we use the @jwt_required decorator to enforce authentication.

class PrivateResource(Resource):
    @jwt_required()
    def get(self):
        return {"meaning_of_life": 42}

Please note the jwt_required decorator takes a parameter (realm) which has a default value of None. Since it takes the parameter, we must use the parentheses to call the function first – @jwt_required() and not just @jwt_required. If this doesn’t make sense right away, don’t worry, please do some study on how decorators work in Python and it will come to you 🙂

Here’s the full code:

from flask import Flask
from flask_restful import Resource, Api
from flask_jwt import JWT, jwt_required

app = Flask(__name__)
app.config['SECRET_KEY'] = 'super-secret'

api = Api(app, prefix="/api/v1")

USER_DATA = {
    "masnun": "abc123"
}


class User(object):
    def __init__(self, id):
        self.id = id

    def __str__(self):
        return "User(id='%s')" % self.id


def verify(username, password):
    if not (username and password):
        return False
    if USER_DATA.get(username) == password:
        return User(id=123)


def identity(payload):
    user_id = payload['identity']
    return {"user_id": user_id}


jwt = JWT(app, verify, identity)


class PrivateResource(Resource):
    @jwt_required()
    def get(self):
        return {"meaning_of_life": 42}


api.add_resource(PrivateResource, '/private')

if __name__ == '__main__':
    app.run(debug=True)

Looks good? Let’s try it out.

Trying it out

Run the app and try to access the secured resource:

$ curl -X GET http://localhost:5000/api/v1/private
{
  "description": "Request does not contain an access token",
  "error": "Authorization Required",
  "status_code": 401
}

Makes sense. The endpoint now requires authorization token. But we don’t have one, yet!

Let’s get one – we must send a POST request to /auth with a JSON payload containing username and password. Please note, the api prefix is not used, that is the url for the auth end point is not /api/v1/auth. But it is just /auth.

$ curl -H "Content-Type: application/json" -X POST -d '{"username":"masnun","password":"abc123"}' http://localhost:5000/auth
{
  "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE0OTQ1OTE4MjcsImlhdCI6MTQ5NDU5MTUyNywibmJmIjoxNDk0NTkxNTI3LCJpZGVudGl0eSI6MTIzfQ.q0p02opL0OxL7EGD7wiLbXbdfP8xQ7rXf7-3Iggqdi4"
}

Cool, we got the token. Now let’s use it to access the resource.

curl -X GET http://localhost:5000/api/v1/private -H "Authorization: JWT eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE0OTQ1OTE4MjcsImlhdCI6MTQ5NDU5MTUyNywibmJmIjoxNDk0NTkxNTI3LCJpZGVudGl0eSI6MTIzfQ.q0p02opL0OxL7EGD7wiLbXbdfP8xQ7rXf7-3Iggqdi4"
{
    "meaning_of_life": 42
}

Whoa, it worked! Amazing, now our JWT authentication is working great!

Getting the Authenticated User

Once our JWT authentication is functional, we can get the currently authenticated user by using the current_identity object.

Let’s add the import:

from flask_jwt import JWT, jwt_required, current_identity

And then let’s update our resource to return the logged in user identity.

class PrivateResource(Resource):
    @jwt_required()
    def get(self):
        return dict(current_identity)

The current_identity object is a LocalProxy instance which can’t be directly JSON serialized. But if we pass it to a dict() call, we can get a dictionary representation.

Now let’s try it out:

$ curl -X GET http://localhost:5000/api/v1/private -H "Authorization: JWT eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJleHAiOjE0OTQ1OTE4MjcsImlhdCI6MTQ5NDU5MTUyNywibmJmIjoxNDk0NTkxNTI3LCJpZGVudGl0eSI6MTIzfQ.q0p02opL0OxL7EGD7wiLbXbdfP8xQ7rXf7-3Iggqdi4"
{
    "user_id": 123
}

As we can see the current_identity object returns the exact same data our identity function returns because Flask JWT uses that function to load the user identity.

What’s Next?

Go ahead and implement the same functionality using PyJWT and your own code. You will need to create an endpoint that encodes current user data and returns the access token. Then you will need to intercept the http headers, parse the Authorization header and verify the JWT token. It should be a fun and yet excellent learning exercise.

 

Understanding JWT (JSON Web Tokens)

In the end of our last post (which was about Securing REST APIs) we mentioned about JWT. I made a promise that in the next post, we would discuss more about JWT and how we can secure our REST APIs using it. However, when I started drafting the post and writing the code, I realized the underlying concepts of JWT themselves deserve a dedicated blog post. So in this blog post, we will focus solely on JWT and how it works.

What is JWT?

We will ignore the text book definitions and try to explain the concepts in our own words. Don’t be afraid of the serious looking acronym, the concepts are rather simple to understand and comprehend. First let’s break down the term – “JSON Web Tokens”, so it has to do something with JSON, the web and of course tokens. Right? Let’s see.

Yes, a JWT mostly concerns with a Token that is actually a hashed / signed form of a JSON payload. The JSON payload is signed using a hashing algorithm along with a secret to produce a single (slightly long) string that works as a token. So a JWT is basically a string / token generated by processing a JSON payload in a certain way.

So how does JWT help? If you followed our last article, you now know why http basic auth is bad. You have to pass your username and password with every request. That is kind of bad, right? The more you send your username and password over the internet, the more likely it is to get compromised, no? Instead, on the first login, we can accept the username and password and return a token back to the client. The client passes that token with every request. We verify that token to see if it’s a logged in user or not. This is the idea behind Token based authentication.

Random Tokens  vs JWT

How would you generate such token? You could generate a nice random string and store it in database against that user. Right? This is how cookie based session works too, btw. Now what if your application is scaled across multiple servers and all requests are load balanced? One server will not recognize a token / session generated by another server. Unless of course you also have one central database active all the time, serving all the incoming requests from all the servers. That setup is tricky and difficult, no?

There is another work around using sticky sessions where the requests from one particular user is always directed to the same server by the load balancer. This work around is also not as simple as JWT. Even if all these work nicely, we still have to make database queries to validate the token / session. What if we want to provide single sign on (users from one service wants to access resources on a different service all together)? How does that work? We will need a central auth server and all services will have to talk to it to verify the user token.

The benefit of JWT is that it’s lightweight but at the same time it’s a self contained JSON payload. You can store user identity in the JSON, sign it and send the token to the clients. Since it’s signed we can verify and validate it with just our secret key. No database overhead. No need for sticky sessions. Just share the secret key privately and all your services can read the data stored inside the JWT. Others can’t tamper or forge a new, valid token for an user without that secret key. Single sign on just becomes a breeze and less complicated. Sounds good? Let’s see how JWTs are constructed.

Anatomy of JWT

A JSON Web Token consists of three major parts:

  • Header
  • Payload
  • Signature

These 3 parts are separated by dots (.). So a JWT looks like the following

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ

If you look closely, there are 3 parts here:

  • Header: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9
  • Payload: eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9
  • Signature: TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ

Okay, so we get the three parts, but what do they really do? Also those strings look like meaningless characters to me. What are they? How are they generated? Well, they are encoded in certain ways as we will see in the following sections.

Header

The header is a simple key value pair (dictionary / hashmap) data structure. It usually has two keys typ and alg short for type and algorithm. The standard is to have the keys at best 3 character long, so the generated token does not get too large.

Example:

{
  "alg": "HS256",
  "typ": "JWT"
}

The typ value is JWT since this is JWT we’re using. The HS256 is the most common and most popular hashing algorithm used with JWT.

Now we just need to base64 encode this part and we get the header string. You were wondering why the strings didn’t make sense. That’s because the data is base64 encoded.

Payload

Here comes our favorite part – the JSON payload. In this part, we put the data we want to store in the JWT. As usual, we should keep the keys and the overall structure as small as possible.

{
  "sub": "1234567890",
  "name": "John Doe",
  "admin": true
}

We can add any data we see fit. These fields / keys are called “claims”. There are some reserved claims – keys which can be interpreted in a certain way by the libraries which decode the JWT. For example, if we pass the exp  (expiry) claim with a timestamp, the decoding library will check this value and throw an exception if the time has passed (the token has expired). These can often be helpful in many cases. You can find the common standard fields on Wikipedia.

As usual, we base64 encode the payload to get the payload string.

Signature

The signature part itself is a hashed string. We concatenate the header and the payload strings (base 64 encoded header and payload) with a dot (.) between them. Then we use the hashing algorithm to hash this string with our secret key.

In pseudocode:

concatenated_string = base64encode(header) + '.' + base64encode(payload)
signature = hmac_sha256(concatenated_string, 'MY_SUPER_SECRET_KEY')

That would give us the last part of the JWT, the signature.

Glue it all together

As we discussed before, the JWT is the dot separated form of the three components. So the final JWT would be:

jwt = header + "." + payload + "." + signature

 Using a library

Hey! JSON Web Tokens sounded great but looks like there’s a lot of work involved! Well, it would seem that way since we tried to understand how a JSON Web Token is actually constructed. In our day to day use cases, we would just use a suitable library for the language / platform of our choice and be done with it.

If you are wondering what library you can use with your language / platform, here’s a comprehensive list of libraries – JSON Web Token Libraries.

Real Life Example with PyJWT

Enough talk, time to see some codes. Excited? Let’s go!

We will be using Python with the excellent PyJWT package to encode and decode our JSON Web Tokens in this example. Before we can use the library, we have to install it first. Let’s do that using pip.

pip install pyjwt

Now we can start generating our tokens. Here’s an example code snippet:

import jwt
import datetime

payload = {
    "uid": 23,
    "name": "masnun",
    "exp": datetime.datetime.utcnow() + datetime.timedelta(minutes=2)
}

SECRET_KEY = "N0TV3RY53CR3T"

token = jwt.encode(payload=payload, key=SECRET_KEY)

print("Generated Token: {}".format(token.decode()))

decoded_payload = jwt.decode(jwt=token, key=SECRET_KEY)

print(decoded_payload)

If we run the code, we will see:

python jwt_test.py
Generated Token: eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1aWQiOjIzLCJuYW1lIjoibWFzbnVuIiwiZXhwIjoxNDk0NDQ5OTQ0fQ.49okXifPSqc7n_n7wZRc9XVVqekTTeBIBBZdiH0nGJQ
{'uid': 23, 'name': 'masnun', 'exp': 1494449944}

So it worked – we encoded a payload and then decoded it back. All we needed to do is to call jwt.encode and jwt.decode with our secret key and the payload / token. So simple, no? parties.

Bonus Example – Expiry

In the following example, we will set the expiry to only 2 seconds. Then we will wait 10 seconds (so the token expires by then) and try to decode the token.

import jwt
import datetime
import time

payload = {
    "uid": 23,
    "name": "masnun",
    "exp": datetime.datetime.utcnow() + datetime.timedelta(seconds=2)
}

SECRET_KEY = "N0TV3RY53CR3T"

token = jwt.encode(payload=payload, key=SECRET_KEY)

print("Generated Token: {}".format(token.decode()))

time.sleep(10)  # wait 10 secs so the token expires

decoded_payload = jwt.decode(jwt=token, key=SECRET_KEY)

print(decoded_payload)

What happens after we run it? This happens:

jwt.exceptions.ExpiredSignatureError: Signature has expired

Cool, so we get an error mentioning that the signature has expired by now. This is because we used the standard exp claim and our library knew how to process it. This is how we use the standard claims to ease our job!

Using JWT for REST API Authentication

Now that we’re all convinced of the good sides of JSON Web Tokens, the question comes into mind – how can we use it in our REST APIs?

The idea is simple and straightforward. When the user logs in the first time, we verify his/her credentials and generate a JSON Web Token with necessary details. Then we return this token back to the user/client. The client will now send the token with every request, as part of the authorization header.

The server will decode this token and read the user data. It won’t have to access the database or contact another auth server to verify the user details, it’s all inside the decoded payload.  And since the token is signed and the secret key is “secret”, we can trust the payload.

But please make sure the secret key is not compromised. And of course use SSL (https) so that men in the middle can not hijack the token anyway.

What’s next?

JSON Web Token is not only about authentication. You can use it to securely transmit data from one party to another. However, it’s mostly used for authenticating REST APIs. In our next blog post, we shall go through that use case. We will see how we can authenticate our api using JWT.

In the mean time, you can subscribe to the mailing list so you can stay up to date with this blog. If you liked the article and/or learned something new, please don’t forget to share it with your friends.

Securing REST APIs: Basic HTTP Authentication with Python / Flask

In our last tutorial on REST API Best Practices, we designed and implemented a very simple RESTful mailing list API. However our API (and the data) was open to public, anyone could read / add / delete subscribers from our mailing list. In serious projects, we definitely do not want that to happen. In this post, we will discuss how we can use http basic auth to authenticate our users and secure our APIs.

PS: If you are new to REST APIs, please check out REST APIs: Concepts and Applications to understand the fundamentals.

Setup API and Private Resource

Before we can move on to authentication, we first need to create some resources which we want to secure. For demonstration purposes, we will keep things simple. We will have a very simple endpoint like below:

from flask import Flask
from flask_restful import Resource, Api

app = Flask(__name__)
api = Api(app, prefix="/api/v1")


class PrivateResource(Resource):
    def get(self):
        return {"meaning_of_life": 42}


api.add_resource(PrivateResource, '/private')

if __name__ == '__main__':
    app.run(debug=True)

If we launch the server and access the endpoint, we will get the expected output:

$ curl -X GET http://localhost:5000/api/v1/private
{
    "meaning_of_life": 42
}

Our API is for now public. Anyone can access it. Let’s secure it so it’s no longer publicly accessible.

Basic HTTP Authentication

The idea of Basic HTTP Authentication is pretty simple. When we request a resource, the server sends back a header that looks something like this: WWW-Authenticate →Basic realm=”Authentication Required”. Generally when we try to access such resources from a browser, the browser shows us a prompt to enter username and password. The browser then base64 encodes the data and sends back an Authorization header. The server parses the data and verifies the user. If the user is legit, the resource is accessible, otherwise we are not granted permission to access it. 

While using a REST Client, we would very often need to pass the credentials before hand, while we make the request. For example, if we’re using curl, we need to pass the --user option while running the command.

Basic HTTP Authentication is a very old method but quite easy to setup. Flask HTTPAuth is a nice extension that would help us with that.

Install Dependencies

Before we can start writing codes, we need to have the necessary packages installed. We can install the package using pip:

pip install Flask-HTTPAuth

Once the package is installed, we can use it to add authentication to our API endpoints.

Require Login

We will import the HTTPBasicAuth class and create a new instance named auth. It’s important to note that name because we will be using methods on this auth instance as decorators for various purposes.  For example, we will use the @auth.login_required decorator to make sure only logged in users can access the resource.

In our resource, we added the above mentioned decorator to our get method. So if anyone wants to GET that resource, s/he needs to login first. The code looks like this:

from flask import Flask
from flask_restful import Resource, Api
from flask_httpauth import HTTPBasicAuth

app = Flask(__name__)
api = Api(app, prefix="/api/v1")
auth = HTTPBasicAuth()


class PrivateResource(Resource):
    @auth.login_required
    def get(self):
        return {"meaning_of_life": 42}


api.add_resource(PrivateResource, '/private')

if __name__ == '__main__':
    app.run(debug=True)

If we try to access the resource without logging in, we will get an error telling us we’re not authorized. Let’s send a quick request using curl.

$ curl -X GET http://localhost:5000/api/v1/private
Unauthorized Access

So it worked. Our API endpoint is now no longer public. We need to login before we can access it. And from the API developer’s perspective, we need to let the users login before they can access our API. How do we do that?

Handling User Logins

We would generally store our users in a database. Well, a secured database. And of course, we would never store user password in plain text. But for this tutorial, we would store the user credentials in a dictionary. The password will be in plain text.

Flask HTTP Auth will handle the authentication process for us. We just need to tell it how to verify the user with his/her username and password. The @auth.verify_password decorator can be used to register a function that will receive the username and password. This function will verify if the credentials are correct and based on it’s return value, HTTP Auth extension will handle the user auth.

In the following code snippet, we register the verify function as the callback for verifying user credentials.  When the user passes the credentials, this function will be called. If the function returns True, the user will be accepted as authorized. If it returns False, the user will be rejected. We have kept our data in the USER_DATA dictionary.

USER_DATA = {
    "admin": "SuperSecretPwd"
}


@auth.verify_password
def verify(username, password):
    if not (username and password):
        return False
    return USER_DATA.get(username) == password

Once we have added the above code, we can now test if the auth works.

$ curl -X GET http://localhost:5000/api/v1/private --user admin:SuperSecretPwd
{
    "meaning_of_life": 42
}

But if we omit the auth credentials, does it work?

$ curl -X GET http://localhost:5000/api/v1/private
Unauthorized Access

It doesn’t work without the login. Perfect! We now have a secured API endpoint that uses basic http auth. But in all seriousness, it’s not recommended.  That’s right, do not use it in the public internet. It’s perhaps okay to use inside a private network. Why? Please read this thread.

Wrapping Up

With the changes made, here’s the full code for this tutorial:

from flask import Flask
from flask_restful import Resource, Api
from flask_httpauth import HTTPBasicAuth

app = Flask(__name__)
api = Api(app, prefix="/api/v1")
auth = HTTPBasicAuth()

USER_DATA = {
    "admin": "SuperSecretPwd"
}


@auth.verify_password
def verify(username, password):
    if not (username and password):
        return False
    return USER_DATA.get(username) == password


class PrivateResource(Resource):
    @auth.login_required
    def get(self):
        return {"meaning_of_life": 42}


api.add_resource(PrivateResource, '/private')

if __name__ == '__main__':
    app.run(debug=True)

As discussed in the last section, it’s not recommended to use basic http authentication in open / public systems. However, it is good to know how http basic auth works and it’s simplicity makes beginners grasp the concept of authentication / API security quite easily.

You might be wondering – “If we don’t use http auth, then what do we use instead to secure our REST APIs?”. In our next tutorial on REST APIs, we would demonstrating how we can use JSON Web Tokens aka JWT to secure our APIs. Can’t wait for that long? Go ahead and read the introduction.

And don’t forget to subscribe to the mailing list so when I write the next post, you get a notification!

Golang / The Go Programming Language

The Go Programming language (often written as golang) has become quite popular recently. Google is actively backing the project  but Golang has seen usage, contribution and success stories from many other popular brands and enterprises across the internet. Go promises a very simple, easy to learn syntax that allows us to build robust, reliable, efficient software. Once we have had invested some time writing  production grade code in the language, we would agree that Go delivers on it’s promises. It’s indeed a fantastic language – easy to learn, easy to read, reason about and of course maintain. You get superb performance without sacrificing much on productivity. Don’t just trust my words, give it a Go! (See what I did there?)

A little history

Work on Golang started back in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson at Google. So that kinds of make Golang 10 years old in 2017. Although work started in 2007, the language was however announced in 2009. It reached version 1.0 in 2012.

A little about the creators – Rob Pike was a member of the unix team and he is known for his work on Plan 9. Ken Thompson designed and implemented Unix. He also created the B language (from which C was inspired). He was also involved in the Plan 9 project.

Why did they start working on a new language? Because they were frustrated with the ones that existed. You can choose a dynamic language like Python / Javascript and enjoy the ease of programming in them. Or you can choose something like C/C++ to get performance. But at the same time you lose the ease of programming, productivity reduces and the compilation time can sometimes get too long. More and more developers were choosing dynamic languages in their projects, essentially favouring the ease of use over the safety and performance offered by the statically typed, compiled languages. There wasn’t any popular, easy to use, mainstream language that could ease these problems for the developers. You couldn’t get ease of programming, safety, efficiency, fast compilation – all from a single language.

The creators of Golang saw this problem as an opportunity to create a better language that could solve these problems at hand.

Why did Golang become so popular?

Go came with better offerings, specially with solutions to what many of us (including the Googlers) faced. When I read (or write) Go code, I feel like the following equation makes perfect sense => C + Python = Golang. Go is very very fast. Not just the language but also the compiler. Go compiles fast, runs fast. And you still feel quite productive, much more productive than you would feel in C++ or Java.

The syntax is simple. You won’t have to remember many keywords.  The static typing provides safety to a great extent. IDEs can provide better code completion and refactoring assistance. Compiler would help you reduce bugs by catching many potential errors before the program even starts to run (which is applicable to all statically typed languages, nothing Golang specific but with Go’s “light on keywords” design, it’s just more productive and enjoyable).

Go provides a nice, extensive standard library with all the batteries you might need for day to day system or network programming. You want to build an awesome web app? Go standard library has you (mostly) covered.

The major win for Go is perhaps the concurrency primitives. We can create very light weight threads called goroutine which are multiplexed on all available CPU cores. We can easily communicate between the goroutines using channels. I personally found the goroutine and channels based way of writing efficient concurrent programs very easy, elegant and pleasant. Don’t be afraid of writing highly concurrent programs any more!

Golang also compiles everything and generates a single binary that includes everything that you need to run the program. You do not need to have anything installed on your target machine to run the binary. This is a huge win in terms of deployment. Writing and distributing command line tools have never been easier!

How popular is Golang?

It’s quite popular. Golang is currently ranked 16 on the TIOBE Index. Tiobe also declared Go to be the language of the year in 2016. Interestingly they also chose Go as the language of the year back in 2009, the year it was released. It stands 15th on the RedMonk language ratings. If you compare active repositories using Githut, Go would be the 14th most active language on Github. For a new programming language, that has released it’s first major version in 2012, that ranking is quite impressive. If you think about the growth of the language, it’s been getting a lot of traction.

In the latest StackOverflow Survey (2017), Go is 3rd on the most wanted list and 5th on the most loved list. That tells us something, doesn’t it? 😀

gopher with a gun
Gopher with a Gun!

How’s the job market, you ask? Don’t worry, Go has jobs for you!

Also check out the HackerNews “Who is hiring?” threads. I see a lot of Golang related job postings there. And do not forget, Golang pays well.

I want to learn Golang

I am sure you do.  So where can we learn Golang by ourselves? These are my recommendations:

  • The Official Tour is excellent. I recommend it to all the beginners. You will have a very good basic once you complete the tour. The tour is also available offline in case you want to learn some golang while on vacation or on the go.
  • Go By Example is another excellent step by step tour. I always refer to this site when I need to lookup a syntax or need a quick reference.
  • Effective Go is a nice read if you want to learn the best practices and want to write idiomatic Go codes.
  • The Little Go Book is a great free book for learning Go.
  • Official Documentation is something we all should keep close to us while writing some Go code.
  • The Go Blog – This is something I really really love. The official Go blog has many interesting and detailed stuff. Trust me, there’s a lot to learn from there. Highly recommended.

If you need some help, please feel free to ask in Gophers Slack or the Go Forum maintained by the GoBridge people.

REST API Best Practices: Python & Flask Tutorial

In our last post about REST APIs, we have learned the basics of how REST APIs function. In this post, we would see how we can develop our own REST APIs. We would use Python and Flask for that. If you are new to Python, we have you covered with our Python: Learning Resources and Guidelines post.

Python / Flask code is pretty simple and easy to read / understand. So if you just want to grasp the best practices of REST API design but lack Python skills, don’t worry, you will understand most of it. However, I would recommend you try out the codes hands on. Writing codes by hand is a very effective learning method. We learn more by doing than we learn by reading or watching.

Installing Flask and Flask-RESTful

We will be using the Flask framework along with Flask-RESTful. Flask-RESTful is an excellent package that makes building REST APIs with Flask both easy and pleasant. Before we can start building our app, we first need to install these packages.

pip install flask
pip install flask-restful

Once we have the necessary packages installed, we can start thinking about our API design.

RESTful Mailing List

You see, I just recently started this Polyglot.Ninja() website and I am getting some readers to my site. Some of my readers have shown very keen interest to receive regular updates from this blog. To keep them posted, I have been thinking about building a mailing list where people can subscribe with their email address. These addresses get stored in a database and then when I have new posts to share, I email them. Can we build this mailing list “service” as a REST API?

The way I imagine it – we will have a “subscribers” collection with many subscriber. Each subscriber will provide us with their full name and email address. We should be able to add new subscriber, update them, delete them, list them and get individual data. Sounds simple? Let’s do this!

Choosing a sensible URL

We have decided to build our awesome mailing list REST API. For development and testing purposes, we will run the app on my local machine. So the base URL would be http://localhost:5000. This part will change when we deploy the API on a production server. So we probably don’t need to worry about it.

However, for API, the url path should make sense. It should clearly state it’s intent. A good choice would be something like /api/ as the root url of the API. And then we can add the resources, so for subscribers, it can be /api/subscribers. Please note that it’s both acceptable to have the resource part singular (ie. /api/subscriber) or plural (/api/subscribers). However, most of the people I have talked to and the articles I have read, more people like the plural form.

API Versioning: Header vs URL

We need to think about the future of the API before hand. This is our first iteration. In the future, we might want to introduce newer changes. Some of those changes can be breaking changes. If people are still using some of the older features which you can’t break while pushing new changes, it’s time you thought about versioning your API. It is always best practice to version your API from the beginning.

The first version of the api can be called v1. Now there are two common method of versioning APIs – 1) Passing a header that specifies the desired version of the API  2) Put the version info directly in the URL. There are arguments and counter arguments for both approaches. However, versioning using url is easier and more often seen in common public APIs.

So we accommodate the version info in our url and we make it – /api/v1/subscribers. Like discussed in our previous REST article, we will have two types of resources here – “subscriber collection” (ie. /subscribers) and “individual subscriber” elements (ie. /subscribers/17).  With the design decided upon and a bigger picture in our head, let’s get to writing some codes.

RESTful Hello World

Before we start writing our actual logic, let’s first get a hello world app running. This will make sure that we have got everything setup properly. If we head over to the Flask-RESTful Quickstart page, we can easily obtain a hello world code sample from there.

from flask import Flask
from flask_restful import Resource, Api

app = Flask(__name__)
api = Api(app)


class HelloWorld(Resource):
    def get(self):
        return {'hello': 'world'}


api.add_resource(HelloWorld, '/')

if __name__ == '__main__':
    app.run(debug=True)

Let’s save this code in a file named main.py and run it like this:

python main.py

If the code runs successfully, our app will launch a web server here – http://127.0.0.1:5000/. Let’s break down the code a bit:

  • We import the necessary modules (Flask and Flask-RESTful stuff).
  • Then we create a new Flask app and then wrap it in Api.
  • Afterwards, we declare our HelloWorld resource which extends Resource.
  • On our resource, we define what the get http verb will do.
  • Add the resource to our API.
  • Finally run the app.

What happens here, when we write our Resources, Flask-RESTful generates the routes and the view handlers necessary to represent the resource over RESTful HTTP. Now let’s see, if we visit the url, do we get the message we set?

If we visit the url, we would see the expected response:

{
    "hello": "world"
}

Trying out REST APIs

While we develop our api, it is essential that we can try out / test the API to make sure it’s working as expected. We need a way to call our api and inspect the output. If you’re a command line ninja, you would probably love to use curl. Try this on your terminal:

➜ curl -X GET http://localhost:5000
{
    "hello": "world"
}
➜

This would send a GET request to the URL and curl would print out the response on the terminal. It is a very versatile tool and can do a lot of amazing things. If you would like to use curl on a regular basis, you may want to dive deeper into the options / features / use cases. These can help you:

However, if you like command line but want a friendlier and easier command line tool, definitely look at httpie.

httpie vs curl
httpie vs curl command

 

Now what if you’re not a CLI person? And we can agree that sometimes GUI can be much more productive to use. Don’t worry, Postman is a great app!

If you are developing and testing a REST API, Postman is a must have app!

postman interface
Our newly created API in Postman

Back to Business

We now have a basic skeleton ready and we know how to test our API. Let’s start writing our mailing list logic. Let’s first layout our resources with some sample data. For this example, we shall not bother about persisting the data to some database. We will store the data in memory. Let’s use a list as our subscriber data source for now.

from flask import Flask
from flask_restful import Resource, Api

app = Flask(__name__)
api = Api(app, prefix="/api/v1")

users = [
    {"email": "[email protected]", "name": "Masnun", "id": 1}
]


class SubscriberCollection(Resource):
    def get(self):
        return {"msg": "All subscribers "}

    def post(self):
        return {"msg": "We will create new subscribers here"}


class Subscriber(Resource):
    def get(self, id):
        return {"msg": "Details about user id {}".format(id)}

    def put(self, id):
        return {"msg": "Update user id {}".format(id)}

    def delete(self, id):
        return {"msg": "Delete user id {}".format(id)}


api.add_resource(SubscriberCollection, '/subscribers')
api.add_resource(Subscriber, '/subscribers/<int:id>')

if __name__ == '__main__':
    app.run(debug=True)

What changes are notable here?

  • Note we added a prefix to the Api for versioning reason. All our urls will be prefixed by /api/v1.
  • We created a list named users to store the subscribers.
  • We created two resources – SubscriberCollection and Subscriber.
  • Defined the relevant http method handlers. For now the response just describes the intended purpose of that method.
  • We add both resources to our api. Note how we added the id parameter to the url. This id is available to all the methods defined on Subscriber.

Fire up the local development server and try out the API. Works fine? Let’s move on!

Parsing Request Data

We have to accept, validate and process user data. In our cases, they would be the subscriber information. Each subscriber would have an email address, a full name and ID. If we used a database, this ID would have been auto generated. Since we are not using a database, we would accept this as part of the incoming request.

For processing request data, the RequestParser can be very helpful. We will use it in our POST calls to /api/subscribers/ to validate incoming data and store the subscriber if the data is valid. Here’s the updated code so far:

from flask import Flask
from flask_restful import Resource, Api
from flask_restful.reqparse import RequestParser

app = Flask(__name__)
api = Api(app, prefix="/api/v1")

users = [
    {"email": "[email protected]", "name": "Masnun", "id": 1}
]

subscriber_request_parser = RequestParser(bundle_errors=True)
subscriber_request_parser.add_argument("name", type=str, required=True, help="Name has to be valid string")
subscriber_request_parser.add_argument("email", required=True)
subscriber_request_parser.add_argument("id", type=int, required=True, help="Please enter valid integer as ID")


class SubscriberCollection(Resource):
    def get(self):
        return users

    def post(self):
        args = subscriber_request_parser.parse_args()
        users.append(args)
        return {"msg": "Subscriber added", "subscriber_data": args}


class Subscriber(Resource):
    def get(self, id):
        return {"msg": "Details about user id {}".format(id)}

    def put(self, id):
        return {"msg": "Update user id {}".format(id)}

    def delete(self, id):
        return {"msg": "Delete user id {}".format(id)}


api.add_resource(SubscriberCollection, '/subscribers')
api.add_resource(Subscriber, '/subscribers/<int:id>')

if __name__ == '__main__':
    app.run(debug=True)

Here we have made two key changes:

  • We created a new instance of RequestParser and added arguments so it knows which fields to accept and how to validate those.
  • We added the request parsing code in the post method. If the request is valid, it will return the validated data. If the data is not valid, we don’t have to worry about it, the error message will be sent to the user.

Testing the request parser

If we try to pass invalid data, we will get error messages. For example, if we request without any data, we will get something like this:

{
  "message": {
    "email": "Missing required parameter in the JSON body or the post body or the query string",
    "id": "Please enter valid integer as ID",
    "name": "Name has to be valid string"
  }
}

But if we pass valid data, everything works fine. Here’s an example of valid data:

{"email": "[email protected]", "name": "John Smith", "id": 3}

This will get us the following response:

{
  "msg": "Subscriber added",
  "subscriber_data": {
    "email": "[email protected]",
    "id": 3,
    "name": "John Smith"
  }
}

Cool, now we know how to validate user data 🙂 Please remember – never trust user input. Always validate and sanitize user data to avoid security risks.

Next, we need to implement the user level updates.

Subscriber Views

We went ahead and completed the code for the rest of the methods. The updated code now looks like this:

from flask import Flask
from flask_restful import Resource, Api
from flask_restful.reqparse import RequestParser

app = Flask(__name__)
api = Api(app, prefix="/api/v1")

users = [
    {"email": "[email protected]", "name": "Masnun", "id": 1}
]


def get_user_by_id(user_id):
    for x in users:
        if x.get("id") == int(user_id):
            return x


subscriber_request_parser = RequestParser(bundle_errors=True)
subscriber_request_parser.add_argument("name", type=str, required=True, help="Name has to be valid string")
subscriber_request_parser.add_argument("email", required=True)
subscriber_request_parser.add_argument("id", type=int, required=True, help="Please enter valid integer as ID")


class SubscriberCollection(Resource):
    def get(self):
        return users

    def post(self):
        args = subscriber_request_parser.parse_args()
        users.append(args)
        return {"msg": "Subscriber added", "subscriber_data": args}


class Subscriber(Resource):
    def get(self, id):
        user = get_user_by_id(id)
        if not user:
            return {"error": "User not found"}

        return user

    def put(self, id):
        args = subscriber_request_parser.parse_args()
        user = get_user_by_id(id)
        if user:
            users.remove(user)
            users.append(args)

        return args

    def delete(self, id):
        user = get_user_by_id(id)
        if user:
            users.remove(user)

        return {"message": "Deleted"}


api.add_resource(SubscriberCollection, '/subscribers')
api.add_resource(Subscriber, '/subscribers/<int:id>')

if __name__ == '__main__':
    app.run(debug=True)

What did we do?

  • We added a helper function to find users from the list by it’s id
  • The update view works – we can update the user data. In our case we’re deleting the data and adding the new data. In real life, we would use UPDATE on the database.
  • Delete method works fine!

Feel free to go ahead and test the endpoints!

HTTP Status Codes

Our mailing list is functional now. It works! We have made good progress so far. But there’s something very important that we haven’t done yet. Our API doesn’t use proper http status codes. When we send response back to the client, we should also give it a status code. This code would help the client better interpret the results.

Have you ever visited a website and saw “404 Not found” error? Well, 404 is the status code, that means the document / resource you were looking for is not available. Saw any “500 Internal Server Error” lately? Now you know what that 500 means.

We can see the complete list of http status codes here: https://httpstatuses.com/.

Also depending on whether you’re a cat person or a dog enthusiast, these websites can explain things better:

So let’s fix our code and start sending appropriate codes. We can return an optional status code from our views. So when we add a new subscriber, we can send  201 Created like this:

return {"msg": "Subscriber added", "subscriber_data": args}, 201

And when we delete the user, we can send 204.

return None, 204

What’s next?

We have made decent progress today. We have designed and implemented a very basic API. We chose a sensible url, considered API versioning, did input validation and sent appropriate http status codes. We have done good. But what we have seen here is a very simple implementation. There are a lot of scope of improvements here. For example, our API is still open to public, there is no authentication enabled. So anyone with malicious intentions can flood / spam our mailing list database. We need to secure the API in that regard. We also don’t have a home page that uses HATEOAS to guide the clients. We don’t yet have documentation – always remember, the documentation is very important. We developers often don’t feel like writing documentation but well written documentation helps the consumers of your API consume it better and with ease. So do provide excellent docs!

I don’t know when – but in our next post on REST APIs, we shall explore more into the wonderful world of API development. And may be we shall also talk about some micro services? If you would like to know when I post those contents, do subscribe to the mailing list. You can find a subscription form on the sidebar.

And if you liked the post, do share with your friends 🙂

REST APIs: The concepts and applications

If you are into web programming you might have come across the terms “REST API” or “RESTful” and you might have felt curious as to what they really are. In this post, we shall take our time to go through some of the concepts / ideas behind REST and it’s applications.

The REST-less Developer

You might have noticed that the technology world is changing so fast. Back in the days, we used to love using Desktop applications. For example, we would probably want to check our emails on Outlook Express or Thunderbird then. Soon the web became popular and web applications started becoming extremely popular for the conveniences they offered. We now love Gmail, don’t we? But then we realized webmails are great and all but that’s not good enough. I want my emails on my phone and tablets. I for example, like to check my emails on my phone – I do that a lot because I can not be in front of a laptop all day. If you think carefully, in a few more years, we would want those emails on our wrist watches.

Now if you were the developer of such a popular email service, how would you serve these different types of devices? For example, the webmail can use HTML fine, mobile phones can browse HTML too but what about desktop and the smart watch apps? Also on phone, people would like native apps more than mobile web apps. So how to feed them data? Don’t be RESTless, some REST always helps! 😉

The RESTful Web

The good old web was working very well for us, for a time being. But the hypertext / HTML is not the native tongue for many devices that we have to accommodate into our services. Also the web is no longer about just “documents”, it is so much more. The REST architecture can shape the modern web in a way that provides uniform access to all our clients (devices). The core idea behind REST is that the client does not need to know anything before hand, it will connect to a server and the server will provide the client with available options via an agreed upon medium. For the web the medium is “HTML”.

For example, when your browser connected to this website, it didn’t know anything beforehand. The server served it a HTML page that has links to various posts and the search form to search content. By reading the source code (HTML), the client (browser) now knows what actions are available. Now, if you click a link or enter a search keyword and perform search, the browser would perform that action. But it has no idea what would happen next. When it performs the action, the server supplies new html telling it what it can do next. So the server is supplying information that the client can use to further interact with the server.

Hypertext or Hypermedia

But hey, not every device can understand HTML, no? Yes, you are absolutely right. That is why the server is no way just confined to HTML. It can provide other responses too, for example XML and JSON are also valid (and two popular) medium of communication. This is why when we describe REST, we usually say “hypermedia” instead of “hypertext”.

The principle that the client does not need to know anything before hand and the server dynamically generates hypermedia responses through which the client interacts with the server – this principle is aptly named “Hypermedia as the engine of application state” aka “HATEOAS“. That is one big name but if you read and think about it, it makes perfect sense. In this principle, the hypermedia generated by the server works as the “engine” of the application’s state. Cool, eh? HATEOAS is a key driving principle of the RESTful web but there’s more. Are you ready to dive in?

Fitting REST into HTTP and APIs

We now understand that in a REST like architecture, there will be a client and a server. The server will provide dynamically generated hypermedia on which the client will act upon. It all makes sense but how do we make our web APIs RESTful?

The idea of communicating over HTTP very often involves Verbs and Resources. Did you notice how very often the same URL can output different responses depending on which http method (GET or POST) we used? The URL can be considered as a resource and the http methods are the verbs. There’s more to just GET and POST. There are PUT, PATCH, DELETE etc.

The purpose/intent of the common http verbs are:

  • GET: The purpose is to literally get the data.
  • POST: This method translates to “create“.
  • PUT / PATCH: We use these methods to update data.
  • DELETE: Come on, do I even need to explain what this one does? 😀

Now while building our APIs, we can map these verbs to our resources. For example, we have User resources. You can access it on http://api.example.com/user. Now when someone makes a GET request, we can send them a list of available users. But when they send new user data via POST, we create a new user. What if they want to view / update / delete a single user instance?

Resources: Collections vs Elements

We can broadly classify the resources into two categories – “collections” and “elements” and apply the http verbs to them. Now we have two different kinds of resources – “user collection” as a whole and “individual users”. How do we map the different http verbs to them? Wikipedia has a nice chart for us.

For Collections: (/user)

  • GET – List all users
  • POST – Create a new user
  • PUT – Replace all users with these new users
  • DELETE – Delete all users

For Elements: (/user/123)

  • GET – Retrieve data about user with ID 123
  • POST – Generally not used and throws errors but can be used if the resource itself is a nested collection. In that case creates new element within that collection.
  • PUT – Replace the user data
  • DELETE – Delete the user

Is this Official?

Everything makes sense and sounds good. So I guess everyone on the web follows this standard? Well, no. Technically, REST is an architecture or architectural style/pattern. It is not a protocol or a standard itself (although it does use other standards like XML or JSON). The sad fact is that nobody has to follow the principles. This is why we often would come across APIs which would not adhere to these principles and design things their own way. And that is kind of alright given REST is not engraved in a holy stone.

But should we follow the principle? Of course we should, we want to be the good citizens of the RESTful web, don’t we?

How can I consume  / create REST APIs?

You do have a curious mind, don’t you? What good is our knowledge of REST if we are not using it ourselves? Of course we shall. This blog post just scratches the surface of it. There is so much more to learn about REST and we shall learn those things in time. I have plans to write detailed posts on consuming and creating REST APIs in the future. You just have to stay in touch! If you haven’t yet, it would be a good idea to subscribe to the mailing list and I will let you know when I write the next piece, deal?

If you have any feedback on this post, please feel free to let me know in the comments. I hope you liked reading it. Also don’t forget to tell your friends about the wonderful world of REST, will you? 🙂

Python: Learning Resources and Guidelines

The Python programming language is rapidly gaining popularity in all directions. Universities are teaching programming with Python. Jobs for the language is also on the rise. It has so many use cases – web development, data science, machine learning, command line tools, security research, desktop applications and what not.

Because of it’s massive popularity, mastery of the language has become a highly valuable skill in the job market these days. Many large enterprises are starting to adopt the language. Small startups are also betting their money on it. Google, Spotify, Instagram, NASA, Twilio, Disqus, Dropbox – all the cool brands are using it in their products / services.

The triumph in the academia is another major win for the programming language. This area has mostly been dominated by languages like C/C++/Java. Beginners have had the first taste of programming in those languages. However, I personally believe, Python is a much better choice in such cases. People who are just getting familiar with programming needs an easy and fun way to learn the basic concepts. C family or Java might require  much more efforts on their side. That is why the following chart makes me happy.

top us universities teaching python
Top US Universities are teaching more Python (Source)

Why is Python so popular?

That is an excellent question. What makes Python so special? Well, to begin with Python is a nice, easy to learn general purpose language. You can use it for many different purposes, it suits many use cases. You will find necessary libraries, frameworks, toolkits for Python which will make your work easier.  Python syntax is very clear and concise. It is  very readable and easy to comprehend. When you read some source code, it all makes sense so easily. The only thing that might confuse you in the beginning is the forced use of whitespace for indentation. This might be a little difficult to grasp in the beginning but once you get it, you will start appreciating the value of properly indented code. Your code will look so much better and cleaner. Don’t you want your code to look beautiful?

5th place on the tiobe index
Python has been in top 10 on the TIOBE Index for a while

 

Apart from the code being very nice and easy to read, Python codes can be very often shorter than other languages. The language is very expressive so you can actually do more while writing less. While reading Python code, you may often feel that you’re actually reading English. The ease of learning and the powerful syntax makes Python very fun to learn and use. We all love fun, no?

Python is dynamically and strongly typed. This will make things easier to handle. Don’t worry if you do not understand the concepts of dynamic or strong typing. You will learn more about them as you start learning the language.

Friendly Community

Probably the best thing about Python  is it’s community. The community has crafted a huge amount of well maintained resources that you can learn from. From free to read online books to free video tutorials on youtube – the freely available resources have you covered. Need to up your Python game? There are excellent paid books and courses which will help you level up your programming skills. There are places like StackOverflow where fellow Pythonistas will answer any questions you have. You can also hang around IRC or Slack rooms for help. The community is very friendly and will warmly welcome you. The good people there will also guide on your journey and help you make right choices from time to time. Got a question? Just ask!

Learning Resources

These are the resources I recommend for learning the language:

  • Automate The Boring Stuff with Python: Excellent book. Really really well written. The author will guide you through a series of interesting tasks which will also teach you various concepts of the language. It will be a very exciting journey for the beginners and an effective learning experience. This is the book I currently recommend to most beginners. You can read the book free online. The author is also creating free youtube videos based on the content.
  • Python 101: Another excellent book. This one is free too. The book will cover the basics of the language. The content is very well written and beginner friendly.
  • Official Tutorial: The official Python documentation is very user friendly. It contains it’s own getting started kind of tutorial. And of course freely available.
  • Python 202: Another good book from Mike Driscoll, the author of “Python101” mentioned above. Once you have learned the basics, this book will help you dive into some more advanced concepts and language features.
  • Pro Python: This is another recommended book for levelling up your Python skills.
  • Fluent Python: I can’t recommend this book enough. When it comes to learning advanced Python, this book is a must read. It is worth every penny!

Practice Makes Us Perfect

Reading books or watching videos are not enough. We need to practise what we learn. There’s a common belief that you need to work at least 10 thousands hours before you can become expert in an area. The same idea applies here too. Read the books, watch the tutorials but at the same time, write codes by hand, every time! You will learn far more by trying those codes yourself and making mistakes. Mistakes are okay – we all make them when we begin. Learning from the mistakes is far more important.

While learning to code, these websites can be of great help. They will help you practise what you have learned so far. Some of them will also get you started with problem solving and challenge your brain. I recommend you pick a site and continue with it.

  • Hackerrank: They have all sorts of problems to challenge you. From data structures to machine learning algorithms, they have you covered. They have excellent support for Python. They even have a dedicated track for Python where you can further shun your skills.
  • Codewars: They have a very diversified set of programming problems to solve. The problems are classified into difficulty levels, so you can work your way through the various levels.
  • Codecombat: Want to learn coding while playing games? Codecombat would allow you to program your character with your language of choice. You can even fight other players.
  • Project Euler: Solve different mathematical problems on this platform.

There are many other online judges for practising algorithmic problem solving. You can try those too. But some of them might not be very Python friendly.

How to get help?

It is very likely that when you are learning Python for the first time, you will be stuck and seeking help. Please remember, Google is your best friend. Searching in Google will often find you the solutions. If it does not, feel free to join StackOverflow and ask there. You can also join this Slack Community to get help from the people there. They are very friendly.

What’s Next?

Once you have had a solid grasp of Python, you probably want to dive into a certain industry. if you want to learn web development, start learning Django or Flask. If you are into Machine Learning or Data Science, start learning the relevant tools / frameworks.

Finally, always remember – learning one language will never be enough. You have to become a Polyglot Ninja!

 

Hello Polyglot Ninja!

If you’re like me, a programming / coding enthusiast, you would probably also start an introductory blog post with a “Hello World!” just like this one:

print("Hello World!")

So who am I? I am nobody significant, at least not yet. But I have big dreams. I learned programming out of passion. I started professionally with PHP, then learned Python and did a good amount of front end and backend Javascript on and off. These days, I mostly introduce myself as a full time Python developer. But deep inside, I am a Polyglot developer.

Who is a Polyglot Developer?

The word “Polyglot” refers to a person who knows and can use multiple languages. From that, we can safely assume that a person who is well versed in multiple programming language is a Polyglot Developer. We often see people around us say they are “PHP Developer”, “Python Developer”, “JavaScript Developer” etc. But hey, anyone working in the web industry for so long, probably knows JavaScript anyway. And if he’s using another language (ie. Ruby) in the backend, he knows 2 languages, right? So most of the full stack developers are polyglot anyway.

There are  people who love exploring new languages out of curiosity and passion. Learning a new language often teaches you new ways of thinking. You’re challenged to think in a different way. And as you gradually learn new techniques and arts to overcome these challenges, you become a better developer from inside. Clojure made me use map, reduce, filter and use recursions to iterate over a sequence – these were enlightenment to me. Soon I realized I have started applying the very same concepts in my Python code. The new ideas / concepts we come across in our newly found languages, we tend to bring them back to the languages we use day to day. And that very often results in better code.

Why should I become a Polyglot developer?

In the early days of my programming career, I had the very same question. Why should I “waste” my time learning Python if I can build all sorts of websites in PHP?

It is with experience and exploration, that I learned not every tool is suitable for every task. And programming is not just about building websites – there are so much more. Every programming language, every framework, every tool, every platform has it’s own use case. Remember – if X had no purpose, it would not have been created in the first place. And if Y has a decent user base / popularity, that means it at least does something better than the other available options.

You can not build a house just by using a hammer, or even if you can, you will have to go through a lot of agony, do extra work and the end result might not be good enough. Think. Every tool has it’s strength and a wise man uses the right tool for the job. He would use a Saw to cut through wood, not the hammer.

If you think about the use cases, you will notice, Python is very popular in Data Science and Machine Learning. There’s a very popular eco system of libraries, frameworks and lots of resources around Python in the machine learning or data science. While Ruby or say PHP can be used to implement some of the algorithms, it will be pain to do so. You will not find suitable, existing, matured libraries. Ruby or PHP will often be slow compared to Numpy or other libraries which are implemented internally in C. So if you’re smart, you would probably choose Python for such tasks. Python is also very popular in Web, System Administration, Desktop GUIs etc.

On the other hand, if you want to do front end of websites, you can not escape Javascript. Of course you can use TypeScript, ClojureScript – but to use them, you still need a certain level of basic knowledge of JS. And not to forget, those languages actually transpile to JS, that is they parse your code and produce JS from them.

If you’re into Big Data, Java/Scala is very prominent in that sector. Go and Rust are getting popular for performance and concurrency. Elixir is making web development fun again with it’s Phoenix framework. There’s a lot of programming languages – and none of them are truly useless. They all bring something to the table at the end of the day. And each one teaches us something new, improves our way of thinking.

But I can’t learn them all!

That is true. You can not learn them all, right now. But over time, with experience, you will be able to learn and get used to a significant number of popular programming languages. Also you probably don’t need to learn all of them either. If you come across a problem that can be best solved by a certain language, go ahead and learn it. Don’t just learn the syntax and the standard library, learn to write idiomatic code in the language. It will take time, give it time, don’t rush. Keep practising, you will get there.

 A Word of Caution

You want to become a Polyglot developer – we all do. That is a good thing. But don’t switch to a language because of the hype. Don’t start learning a language just because it’s a hip thing. Take your time, evaluate the language, check out the syntax, see what problems it can solve better, check out the community, maturity of the language and the eco system. Overall, think if learning the language would benefit you in anyway. You should only learn a language that you can use to solve a problem better. if you know PHP and you need to create a simple dynamic webpage, NodeJS won’t probably help you much.

Another very important thing would be learn one language very well before you start moving to the next one. When we learn our first language, we are not only learning a programming language but we’re also getting to know the concepts of programming for the first time. So take your time, learn all the concepts in depth. Gain some significant experience before exploring other languages.

 Which language should I start with?

You will get different suggestions on this one – some will say C, some will recommend Python, some people will advise Java and so on.

If you’re in your early days of programming, learning C has it’s benefits. You can better understand how things work under the hood. So do try C first and if you think you can pursue it, go ahead and get a good basic of C and C++.

If you didn’t like C – it looked very difficult and kind of scared you with all the memory management and pointer stuff, do try Python. It is a much nicer language, easier to get started.

When you have learned either of them well, go ahead and learn some new languages. May be Java? Golang? Rust? Well, everyone has his own preference. So make your own choice, try those languages and their use cases, pick the one you like for the job you want to use it for. Always remember – “right tool for the right job”.

 Where should I learn?

I will try to provide in depth guidelines / resources for different programming languages in the coming days. For now, Google for a good book and learn the syntax and standard library. There are plenty of free resources online. Once you have learned the basics, start solving problems on Hacker Rank or CodeWars. if you’re stuck or need help, ask in StackOverflow or Google for more. A good search engine like Google is a lifetime friend of a developer. So better get friendly with it. Learn some good techniques to find results fast. It will help you a lot in your coming days!

Where to go next?

Keep practising. Keep reading. Follow other fellow programmers on Twitter, see what they are up to. May be subscribe to some programming related sub reddits too?

And of course, don’t forget to subscribe to my mailing list. I don’t spam, you can unsubscribe any time. I shall be sending new post updates, essential guidelines and cool tips and tricks.

Happy learning!