Golang: Interface

In Go or Golang, declaring an interface is pretty simple and easy.

type Printer interface {
	Print(string)
}

We just defined an interface named Printer that required an implementer to have a method named Print which takes a string parameter and returns nothing. Interfaces are implemented implicitly in Go. Any type that has the Print(string) method implements the interface. There is no need to use any implements keyword or anything of that sort.

type Terminal struct {
}

func (t Terminal) Print(message string) {
	fmt.Println(message)
}

In the above example, the Terminal¬†type implements the Printer¬†interface because it implements the methods required by the interface. Here’s a runnable, full code example:

package main

import (
	"fmt"
)

type Printer interface {
	Print(string)
}

type Terminal struct {
}

func (t Terminal) Print(message string) {
	fmt.Println(message)
}

func main() {
	var printer Printer
	printer = Terminal{}

	printer.Print("Hello World!")
}

We declared our printer variable to be of type Printer which is the interface. Since the Terminal type implements the Printer interface, we can pass Terminal{} to the printer variable and later call the Print method on it.

Interface and the Method sets

As you can understand, a method set is a set of methods on a type. The method set of an interface type (for example Printer¬†here) is it’s interface, that is the Print¬†method in this example. The method set of a type T (for example Terminal) contains the methods which can take a T¬†type receiver. In our above code, the Print¬†method takes the type Terminal¬†so it’s included in Terminal‘s method set. The corresponding pointer type, *T¬†has a method set that includes all methods with a receiver type of *T¬†as well as the methods defined on the receiver type T. So *Terminal¬†type contains any method that either takes Terminal¬†or Terminal¬†as a receiver type. So the Print¬†method is also in the method set for *Terminal¬†.

Method Set of T includes all methods receiving just T.
Method Set of *T includes all methods receiving either T or *T.

So the method set of *T¬†includes the method set of T¬†anyway. But by now, you might be wondering why this is so important. It is very important to understand the method set of a type because whether it implements an interface or not depends on the method set. To understand things further, let’s take a quick look at the following example:

package main

import (
	"fmt"
)

type Printer interface {
	Print(string)
}

type Terminal struct {
}

func (t *Terminal) Print(message string) {
	fmt.Println(message)
}

func main() {
	var printer Printer
	printer = Terminal{}

	printer.Print("Hello World!")
}

If you try to run this code on the go playground or try to run/compile it on your machine, you shall get an error message like this:

.\main.go:20:10: cannot use Terminal literal (type Terminal) as type Printer in assignment:
	Terminal does not implement Printer (Print method has pointer receiver)

Can you guess what’s happening? Well, our Print¬†method has a receiver type of *Terminal¬†however we are trying to assign the type Terminal¬†to printer. The Print¬†method falls in the method set of *Terminal¬†and not Terminal. So in this particular example, *Terminal¬†type actually implements the interface, not the base Terminal¬†type. We can just assign &Terminal¬†to printer¬†and it will work fine. Try the codes here –¬†https://play.golang.org/p/MvyD0Ls8xb¬†ūüôā

Another interesting thing, since *Terminal¬†also includes the method set defined on Terminal, this could would be valid just fine –¬†https://play.golang.org/p/xDmNGBcwsM. This is why understanding the method set of a type is important to understand which interfaces it implements.

The Curious Case of Method Calls

We have seen how the method set of *T includes methods receiving both T and *T but the method set of T is confined to methods that only take T and not *T. Now you might be thinking РI have seen codes like the following snippet:

package main

import (
	"fmt"
)

type Printer interface {
	Print(string)
}

type Terminal struct {
}

func (t *Terminal) Print(message string) {
	fmt.Println(message)
}

func main() {
	var terminal Terminal
	terminal = Terminal{}
	terminal.Print("Hello!")
}

Here, the Print method receives a *Terminal type but how are we calling it on Terminal type? From what we have seen before, Terminal should not have the method set defined to take a *Terminal receiver, how is this call being made?

Well, the code x.m()¬†works fine if the m¬†method takes the x¬†type as receiver. That is fine with us. But if the method m¬†is to take the type *x¬†and we try to call x.m()¬†– that shouldn’t work, right? The proper call should be (&x).m()¬†– no? Yes, correct. But Go provides us a shortcut here.¬†If the method m¬†is defined to take a *x¬†type as receiver and base type x¬†is addressable, x.m()¬†works as a shortcut for (&x).m(). Go provides us with that shortcut to keep things simpler. So whether you have a pointer or a value, it doesn’t matter, as long as the type is addressable, you can call the method set of *x¬†on x¬†using the very same syntax. However, please remember that this shortcut is not available while working with interfaces.

The Empty Interface

The type interface{} has zero methods defined on it. And every type in Go implements zero or more methods. So their method set actually satisfies the emtpy interface aka interface{}. So if a variable is of type interface{}, we can pass any type to it.

package main

import (
	"fmt"
)

func main() {
	var x interface{}

	x = 2
	fmt.Println(x)

	x = "masnun"
	fmt.Println(x)

}

We want to store different types in the same slice? Map values can be of different types? Just use interface{}.

package main

import "fmt"

func main() {
	things := []interface{}{}

	things = append(things, "masnun")
	things = append(things, 42)

	fmt.Println(things)

	unKnownMap := map[string]interface{}{}

	unKnownMap["name"] = "masnun"
	unKnownMap["life"] = 42

	fmt.Println(unKnownMap)

}

So, when we’re not sure of a type, or we need the type to flexible / dynamic, we can use interface{}¬†to store them.

Type Assertion

While we can store any type in a interface{} type, not all types are the same. For example, you can not use the string functions on an integer type. Go would not accept if you blindly want to pass interface{} in an operation where a very specific type is expected. Take a look:

package main

import (
	"strings"
)

func main() {
	unKnownMap := map[string]interface{}{}

	unKnownMap["name"] = "masnun"
	unKnownMap["life"] = 42

	strings.ToUpper(unKnownMap["name"])

}

Even though we have a string value stored against the name key, Go actually stores it as a type interface{}¬†and thus it won’t allow us to use it like a string. Luckily, interface values do store the underlying value and type. So we can use type assertion to assert that the underlying value can behave like a certain type.

This works:

package main

import (
	"strings"
	"fmt"
)

func main() {
	unKnownMap := map[string]interface{}{}

	unKnownMap["name"] = "masnun"
	unKnownMap["life"] = 42

	fmt.Println(strings.ToUpper(unKnownMap["name"].(string)))

}

The¬†unKnownMap["name"].(string)¬†part – we’re doing the type assertion here. If the type assertion succeeds, we can use the value as a string. If it does not succeed, we will get a panic.

Getting Type of an interface{}

If you have an interface{} and want to know what it holds underneath, you can use the %T format in Printf family of calls.

package main

import (
	"fmt"
)

func main() {
	unKnownMap := map[string]interface{}{}

	unKnownMap["name"] = "masnun"
	unKnownMap["life"] = 42

	fmt.Printf("%T \n", unKnownMap["name"])
	fmt.Printf("%T \n", unKnownMap["life"])

}

Type Switch

You can also use a switch statement with an interface{} to deal with different possible types.

package main

import (
	"fmt"
)

func main() {
	unKnownMap := map[string]interface{}{}

	unKnownMap["name"] = "masnun"
	unKnownMap["life"] = 42

	TypeSwitch(unKnownMap["name"])
	TypeSwitch(unKnownMap["life"])

}

func TypeSwitch(i interface{}) {
	switch i.(type) {
	case string:
		fmt.Println("String Value: ", i.(string))

	case int:
		fmt.Println("Integer Value: ", i.(int))

	}

}

The i.(type) gets you the type of the variable. Please remember it only works with a switch statement.

Golang: Making HTTP Requests

Go aka Golang is a very promising programming language with a lot of potential. It’s very performant, easy to grasp and maintain, productive and backed by Google. In our earlier posts, we have tried to provide guidelines to learn Go¬†and later we saw how to work with JSON in Go. In this blog post, we’re going to see how we can make http requests using Go. We shall make use of the net/http¬†package in Go which provides all the stuff we need to make http requests or create new http servers. That is, this package would help you do all things “http”. To check / verify that we made correct requests, we would be using httpbin¬†which is a nice service to test our http client requests.

A Simple HTTP Request

Let’s make a very simple GET request and see how we can read the response. We would be sending a simple HTTP GET request to https://httpbin.org/get and read the response. For that we can just import the net/http¬†package and use the http.Get¬†function call. Let’s see an example:

package main

import (
	"net/http"
	"log"
	"io/ioutil"
)

func main() {
	MakeRequest()
}

func MakeRequest() {
	resp, err := http.Get("https://httpbin.org/get")
	if err != nil {
		log.Fatalln(err)
	}

	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		log.Fatalln(err)
	}

	log.Println(string(body))
}

We have created a separate MakeRequest¬†function and called it from our main function. So going ahead, we will just see the changes inside this function and won’t need to think about the entire program. Inside this function, we have passed the url to http.Get¬†and received two values – the response object and any errors that might have happened during the operation. We did a check to see if there were any errors. If there weren’t any errors, err¬†would be nil. Please note that this err¬†would be reported only if there was an issue connecting to the server and getting a response back. However, it would not be concerned about what http status code the server sent. For example, if the server sends a http 500 (which is internal server error), you will get that status code and error message on the resp¬†object, not on err.

Next, we read the resp.Body¬†which implements the io.ReadCloser¬†interface and we can use ioutil.ReadAll¬†to fully read the response. This function also returns two values – a byte slice ([]byte) and err. Again, we check for any potential errors in reading the response body. If there were no errors, we print out the body. Please note the string(body)¬†part. Here, we’re converting the byte slice to a string. If we don’t do it, log.Println¬†would print out representation of the byte slice, a list of all the bytes in that slice, individually. But we want a string representation. So we go ahead and make the conversion.

We would see the printed output is a JSON string. You will notice the httpbin service outputs JSON messages. So in the next example, we would see how we can send and read JSON messages.

JSON Requests and Responses

Now let’s send a JSON message. How do we do that? If you’re coming from Python / Node / Ruby, you might be used to passing a dictionary like structure to your favorite requests library and just mention it should be sent as JSON. Your library does the conversion for you and sends the request with required headers. In Go, however, things are more explicit, which is a good thing in fact. You will know what you’re doing, how you’re doing it.¬†If the JSON related functionality is new to you, please do check our blog post –¬†Golang: Working with JSON.

In Go, we would first convert our data structure to a byte slice containing the JSON representation of the data. Then we pass it to the request with the proper content type. Let’s see a code example:

func MakeRequest() {

	message := map[string]interface{}{
		"hello": "world",
		"life":  42,
		"embedded": map[string]string{
			"yes": "of course!",
		},
	}

	bytesRepresentation, err := json.Marshal(message)
	if err != nil {
		log.Fatalln(err)
	}

	resp, err := http.Post("https://httpbin.org/post", "application/json", bytes.NewBuffer(bytesRepresentation))
	if err != nil {
		log.Fatalln(err)
	}

	var result map[string]interface{}

	json.NewDecoder(resp.Body).Decode(&result)

	log.Println(result)
	log.Println(result["data"])
}

We first created message which is a map containing a string value, an integer value and another embedded map. Then we json.Marshal it to get the []byte out of it. We also check for any errors that might happen during the marshalling. Next, we make a POST request using the http.Post function. We pass the url, our content type, which is JSON and then we create and pass a new bytes.Buffer object from the bytes representation. Why do we need to create a buffer here? The http.Post function expects an implementation of io.Reader Рwhich is a brilliant design, anything that implements an io.Reader can be passed here. So we could even read this part from disk or network or any custom readers we want to implement. In our case, we can just create a bytes buffer which implements the io.Reader interface. We send the request and check for errors.

Next we declare another result¬†variable (which is also a map type) to store the results returned from the request. We could read the full body first (like previous example) and then do json.Unmarshal¬†on it. However, since the resp.Body¬†is an io.Reader, we can just pass it to json.NewDecoder¬†and then call Decode¬†on it. Remember, we have to pass a pointer to our map, so we passed &result¬†instead of just result.¬† The Decode¬†function returns an error too. But we assumed it would not matter and didn’t check. But best practice would have been to handle it as well. We logged the result¬†and result["data"]. The httpbin¬†service sends different information about the request as the response. You can see those in the result¬†map. If you want to see the data you sent, they will be in the data¬†key of the result¬†map.

Posting Form

In our last example, we have submitted JSON payload. What if we wanted to submit form values? We have the handy http.PostForm¬†function for that. This function takes the url and url.Values¬†from net/url¬†package. The url.Values¬†is a custom type which is actually¬†map[string][]string¬†internally. That is – it’s a map which contains string keys and against each key, there can be multiple string values ([]string). In a form request, you can actually submit multiple values against one field name. That’s the reason it’s a slice of string, instead of just a key to value mapping.

Here’s an example code snippet:

func MakeRequest() {

	formData := url.Values{
		"name": {"masnun"},
	}

	resp, err := http.PostForm("https://httpbin.org/post", formData)
	if err != nil {
		log.Fatalln(err)
	}

	var result map[string]interface{}

	json.NewDecoder(resp.Body).Decode(&result)

	log.Println(result["form"])
}

We would be reading the form key from the result map to retrieve our form values. We have seen how we can easily send form values using the net/http package. Next we would like to send a file along with typical form fields. For that we would also need to learn how to customize http requests on our own.

Custom Clients / Requests

The http.Get, http.Post¬†or http.PostForm¬†calls we have seen so far uses a default client already created for us. But now we are going to see how we can initialize our own Client¬†instances and use them to make our own Requests. Let’s first see how we can create our own clients and requests to do the same requests we have made before. A quick example follows:

func MakeRequest() {

	client := http.Client{}
	request, err := http.NewRequest("GET", "https://httpbin.org/get", nil)
	if err != nil {
		log.Fatalln(err)
	}

	resp, err := client.Do(request)
	if err != nil {
		log.Fatalln(err)
	}

	var result map[string]interface{}
	json.NewDecoder(resp.Body).Decode(&result)
	log.Println(result)
}

As you can see, we just take a new instance of http.Client¬†and then create a new request by calling http.NewRequest¬†function. It takes the http method, url and the request body. In our case, it’s a plain GET request, so we pass nil¬†for the body. We then call the Do¬†method on the client¬† and parse the response body. So that’s it – create a client, create a request and then let the client Do¬†the request.¬†Interestingly the client¬†also has convenient methods like Get, Post, PostForm¬†– so we can directly use them. That’s what http.Get, http.Post, http.PostForm and other root level functions actually do. They call these methods on the DefaultClient¬†which is already created beforehand. In effect, we could just do:

func MakeRequest() {

	client := http.Client{}
	resp, err := client.Get("https://httpbin.org/get")
	if err != nil {
		log.Fatalln(err)
	}

	var result map[string]interface{}
	json.NewDecoder(resp.Body).Decode(&result)
	log.Println(result)
}

And it would work similarly. Now you might be wondering Рwhy not just use the DefaultClient, why create our own? What is the benefit?

Customizing the Client

If we look at the definition of the http.Client structure, it has these fields:

type Client struct {
	Transport RoundTripper
	CheckRedirect func(req *Request, via []*Request) error
	Jar CookieJar
	Timeout time.Duration
}

If we want, we can set our own transport implementation, we can control how the redirection is handled, pass a cookie jar to save cookies and pass them to the next request or simply set a timeout. The timeout part is often very significant in making http requests. The DefaultClient does not set a timeout by default. So if a malicious service wants, it can start blocking your requests (and your goroutines) indefinitely, causing havoc in your application. Customizing the client gives us more control over how the requests are sent.

File Upload

For uploading files while sending a http request, we need to use the mime/multipart¬†package with the net/http¬†package. We will first see the code example and then walk through it to understand what we’re doing. The code might seem a lot (it includes a lot of error handling) and complex. But please bear with me, once you go through the code and understand what’s happening, it will seem so simpler ūüôā

func MakeRequest() {

	// Open the file
	file, err := os.Open("name.txt")
	if err != nil {
		log.Fatalln(err)
	}
	// Close the file later
	defer file.Close()

	// Buffer to store our request body as bytes
	var requestBody bytes.Buffer

	// Create a multipart writer
	multiPartWriter := multipart.NewWriter(&requestBody)

	// Initialize the file field
	fileWriter, err := multiPartWriter.CreateFormFile("file_field", "name.txt")
	if err != nil {
		log.Fatalln(err)
	}

	// Copy the actual file content to the field field's writer
	_, err = io.Copy(fileWriter, file)
	if err != nil {
		log.Fatalln(err)
	}

	// Populate other fields
	fieldWriter, err := multiPartWriter.CreateFormField("normal_field")
	if err != nil {
		log.Fatalln(err)
	}

	_, err = fieldWriter.Write([]byte("Value"))
	if err != nil {
		log.Fatalln(err)
	}

	// We completed adding the file and the fields, let's close the multipart writer
	// So it writes the ending boundary
	multiPartWriter.Close()

	// By now our original request body should have been populated, so let's just use it with our custom request
	req, err := http.NewRequest("POST", "https://httpbin.org/post", &requestBody)
	if err != nil {
		log.Fatalln(err)
	}
	// We need to set the content type from the writer, it includes necessary boundary as well
	req.Header.Set("Content-Type", multiPartWriter.FormDataContentType())

	// Do the request
	client := &http.Client{}
	response, err := client.Do(req)
	if err != nil {
		log.Fatalln(err)
	}

	var result map[string]interface{}

	json.NewDecoder(response.Body).Decode(&result)

	log.Println(result)
}

So what are we doing here?

  • First we are opening the file we want to upload. In our case, I have created a file named “name.txt” that just contains my name.
  • We create a bytes.Buffer¬†to hold the request body we will be passing with our http.Request¬†later on.
  • We create a multipart.Writer¬†object and pass a pointer to our bytes.Buffer¬†object so the multipart writer can write necessary bytes to it.
  • The multipart writer has convenient methods to create a form file or a form field. It gives us back a writer to which we can write our file content or the field values. We create a file field and copy our file contents to it. Then we create a normal field and write “Value” to it.
  • Once we have written our file and normal form field, we call the Close method on the multipart writer object. Closing it writes the final, ending boundary to the underlying bytes.Buffer¬†object we passed to it. This is necessary, otherwise the request body may remain incomplete.
  • We create a new post request like we saw before. We passed the bytes.Buffer¬†we created as the request body. The body now contains the multi part form data written with the help of the mime/multipart¬†package.
  • We send the request as before. But we set the content type by calling¬†multiPartWriter.FormDataContentType()¬†– which ensures the correct content type and boundary being set.
  • We decode the response from httpbin and check the output.

If everything goes well, we will see the form field and the file name in the response we received from httpbin. The concept here is simple. We are sending a http request with a custom body. We could construct the request body ourselves but we just took the help of the mime/multipart package to construct it in a relatively easier fashion.

Always Close The Response Body

Here’s a lesson I learned the hard way. When we make a http request, we get a response and an error back. We may feel lazy and decide not to check for errors or close the response body (just like in the examples above). And from the laziness comes disaster. If we do not close the response body, the connection may remain open and cause resource leak. But if the error is not nil that is in case of an error, the response can be nil. So we can’t just do a defer resp.Body.Close()¬†in this case. We have to properly check error and then close the response body.

client := http.DefaultClient
resp, err := client.Do(req)
if err != nil {
    return nil, err
}
defer resp.Body.Close()

Always Use a Timeout

Try to use your own http client and set a timeout. Not setting a timeout can block the connection and the goroutine and thus cause havoc. So do something like this:

timeout := time.Duration(5 * time.Second)
client := http.Client{
    Timeout: timeout,
}
client.Get(url)

 

Golang: Working with JSON

We previously wrote an introductory blog post on Golang describing why and how the language is gaining traction quickly. Go aka Golang is being used by many large scale systems for various purposes. One thing common in most complex systems is that we have to communicate with other systems / services. JSON is a very popular data interchange format for this kind of scenarios. In fact, it is so popular that you may even go ahead and call it the de-facto data interchange format on the internet.  If you have followed any of our REST API tutorials (Django / Flask), you might have also noticed they all output JSON. In this blog post, we will see the different ways we can work with JSON in Golang.

Creating JSON

We can use the encoding/json¬†package to easily create JSON from our Go data structures. There’s a few things to consider while creating JSON in Go. One of them is the choice of data structure.

From Structs

Structs are very convenient to use and often come as cheaper compared to map or other complex structures. We can pass any instance of a struct to the json.Marshal function and get back the JSON as a slice of bytes ([]byte).

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string
	Age    int
	Emails []string
}

func main() {
	masnun := Person{
		Name:   "Abu Ashraf Masnun",
		Age:    27,
		Emails: []string{"[email protected]", "[email protected]"},
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s",json_bytes)
}

Output should be:

{"Name":"Abu Ashraf Masnun","Age":27,"Emails":["[email protected]","[email protected]"]}

Things to note here:

  • The field name has to start with a capital letter – this is important and will be discussed soon
  • You can nest other structures, in this example our Emails¬†field contain a list of strings
  • The json.Marshal¬†function returns the JSON and error, don’t forget to handle the error. You may not often get errors from the Marshal¬†function but in some cases where some types can not be converted into JSON, you will get errors. So look out for it.
  • The returned JSON is in the form of bytes list. So if you want to use it as string, you will need to convert it.

Choosing Field Names

If you have seen the output, the keys/fields in the created JSON structure all start with a capital letter (our struct fields were similar, so this is no surprise). If you have a curious and adventurous mind, you might have tried to go ahead and convert the struct fields into lower caps. But that doesn’t work, does it? Check it out –¬†https://play.golang.org/p/93eDoFSjnW¬†– because if a field / member name does not start with a capital letter, it is not exported. External code can not access it. This is why our name¬†and age¬†fields are not part of the generated JSON structure.

But don’t worry, there’s a simple way of “tagging” our struct fields so we can describe how to marshal our structs. Let’s modify our codes to look like this:

type Person struct {
	Name   string   `json:"name"`
	Age    int      `json:"age"`
	Emails []string `json:"emails"`
}

Next to each field, we have provided tags to describe how this field should be marshalled or unmarshalled. Now we can see you expected results here:¬†https://play.golang.org/p/xlcjU1_VSE¬†ūüôā

If you don’t understand the tags, don’t worry, try reading this answer on StackOverflow –¬†https://stackoverflow.com/questions/10858787/what-are-the-uses-for-tags-in-go. We will understand the concepts more when we use structs for various purposes (for example mapping to database tables, but for now, let’s not worry).

Omitting Empty Fields

What if some of the fields are empty? Let’s try that.

func main() {
	masnun := Person{}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

The output would be:

{"name":"","age":0,"emails":null}

If a field is empty, we might want to omit that from our JSON. We can do so by using the omitempty flag in our json field tags.

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string      `json:"name,omitempty"`
	Age    int         `json:"age,omitempty"`
	Emails []string    `json:"emails,omitempty"`
}

func main() {
	masnun := Person{
		Name: "Abu Ashraf Masnun",
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

Now if we check the output again:

{"name":"Abu Ashraf Masnun"}

Nice, no?

Skipping Fields

Let’s say in our struct, we need to keep the Age field. But we don’t want it to be a part of the produced JSON. We can use the -¬†flag in the json tag for that particular field.

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string      `json:"name,omitempty"`
	Age    int         `json:"-"`
	Emails []string    `json:"emails,omitempty"`
}

func main() {
	masnun := Person{
		Name: "Abu Ashraf Masnun",
		Age:  27,
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

The output would be:

{"name":"Abu Ashraf Masnun"}

Even though the struct had the Age field set, it wasn’t included in the output. This comes very handy in some cases, for example when a¬†User¬†struct has a Password¬†field that we don’t want to serialize into JSON.

Using Maps and Slices

So far we have used structs. We can also use maps and slices instead. Here’s a quick code example:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	masnun := map[string]interface{}{

		"name": "Abu Asharf Masnun",
		"age":  27,
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

And using slices:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	emails := []string{"[email protected]", "[email protected]"}
	json_bytes, _ := json.Marshal(emails)
	fmt.Printf("%s", json_bytes)
}

They both work as expected. But in most codebases I have come across, structs are more widely used.

 

Parsing JSON

We have so far seen how we can generate JSON from our go data. Now we will see the opposite. We will be parsing JSON into Go data structures.

Into Structs

We will first see how we can parse JSON data into structs. It’s quite similar to what we did earlier. We will be using the Unmarshal¬†function which takes bytes and pointer to any interface{}¬†type. It reads through the JSON and stores the data in the struct we pass as the second parameter. Let’s see an example:

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string   `json:"name"`
	Age    int      `json:"age"`
	Emails []string `json:"emails"`
}

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"]
		}
	`)

	masnun := Person{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	fmt.Println(masnun.Name, masnun.Age, masnun.Emails)

}

Here json_bytes hold the JSON we want to process. We already have a Person type with tagged fields. We just need to pass this json_bytes and a pointer to an instance of Person to the Unmarshal function. Please note the pointer is important. We have to pass a pointer otherwise the parser would not be able to write to the struct.

If the struct doesn’t have some fields which are present in the JSON, those will be silently ignored. In the same way, if the struct has fields which are not available in the JSON, they will be ignored too.

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name    string   `json:"name"`
	Age     int      `json:"age"`
	Emails  []string `json:"emails"`
	Address string
}

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun Person
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	fmt.Println(masnun.Address)

}

In the above example, the struct has a field named Address¬†which the JSON doesn’t provide. On the other hand, the JSON has the Score¬†key which the struct knows nothing about. In this case, masnun.Address¬†will be empty string.

Into Maps / Slices

We have previously mentioned how structs are cheaper and more widely used than maps. But there’s these use cases where we can not be certain about the structure of the JSON data we want to parse. In such cases, maps can be very useful. Let’s see:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun map[string]interface{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	fmt.Println(masnun["Name"], masnun["Age"], masnun["Emails"], masnun["Score"])

}

See? We have passed map[string]interface{} and received all the data in JSON. But please remember, the values to each key in the map will be of type interface{}. If we want to extract part of the data, for example, one of the emails and then use it as a string, I will have to manually convert it to a string.

For example this code will fail:

package main

import (
	"encoding/json"
	"fmt"
	"strings"
)

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun map[string]interface{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	var firstEmail string
	firstEmail = masnun["Emails"][0]

	fmt.Println(strings.ToUpper(firstEmail))

}

We will get an error:

# command-line-arguments
./main.go:27: invalid operation: masnun["Emails"][0] (type interface {} does not support indexing)

That’s what I was trying to explain just above. The Emails¬†key has a value of interface{}¬†type. Let’s cast it to a list of interface{} first. Then we can take an element (which will be again interface{}¬†type). We further cast it to a string.

package main

import (
	"encoding/json"
	"fmt"
	"strings"
)

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun map[string]interface{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	var emails []interface{}
	var firstEmail string
	emails = masnun["Emails"].([]interface{})
	firstEmail = emails[0].(string)

	fmt.Println(strings.ToUpper(firstEmail))

}

You may be wondering why couldn’t we just get¬†Emails¬†as¬†[]string? Well, Go doesn’t know the types of the values in our JSON. So it uses interface{}¬†. That is when it stores Emails, it stores it as a list of unknown types or a list of interface{}. That is why we first need to get it as a list and then we can take individual items and further convert them to the type we want.

Now it works fine ūüôā

Streaming JSON Encoding and Decoding

The json package offers NewEncoder and NewDecoder functions which would get us Encoder and Decoder types. These types can work with other objects that support io.Reader and io.Writer interfaces to offer streaming support.

Streaming JSON from a File

We can open a JSON file using the os.Open¬†function and stream it using the json.NewDecoder¬†function. Here’s a quick example:

package main

import (
	"encoding/json"
	"fmt"
	"os"
)

func main() {

	fileReader, _ := os.Open("masnun.json")
	var masnun map[string]interface{}
	json.NewDecoder(fileReader).Decode(&masnun)
	fmt.Println(masnun)

}

We opened a file which implements the io.Reader¬†interface. So we can use it with our Decoder¬†type. We created a decoder with the file reader and then called the Decode¬†method on it. That’s all we needed ūüôā

Streaming JSON into a File

Writing JSON is also very similar. We need to open a file in write mode, grab the io.Writer and pass it to json.NewEncoder Рthen we can pass our data to the Encode method to stream the json into the file.

package main

import (
	"encoding/json"
	"os"
)

type Person struct {
	Name   string
	Age    int
	Emails []string
}

func main() {
	masnun := Person{
		Name:   "Abu Ashraf Masnun",
		Age:    27,
		Emails: []string{"[email protected]", "[email protected]"},
	}

	fileWriter, _ := os.Create("output.json")
	json.NewEncoder(fileWriter).Encode(masnun)
}

 

Custom Marshal / Unmarshal

If we want to change how our own types are marshalled or unmarshalled, we can implement the json.Marshaler¬†and json.Unmarshaler¬†interfaces. It’s actually simple. We need to define the MarshalJSON¬†and UnmarshalJSON¬†methods on our structs and we’re done. Here’s an example from the official documentation:

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"strings"
)

type Animal int

const (
	Unknown Animal = iota
	Gopher
	Zebra
)

func (a *Animal) UnmarshalJSON(b []byte) error {
	var s string
	if err := json.Unmarshal(b, &s); err != nil {
		return err
	}
	switch strings.ToLower(s) {
	default:
		*a = Unknown
	case "gopher":
		*a = Gopher
	case "zebra":
		*a = Zebra
	}

	return nil
}

func (a Animal) MarshalJSON() ([]byte, error) {
	var s string
	switch a {
	default:
		s = "unknown"
	case Gopher:
		s = "gopher"
	case Zebra:
		s = "zebra"
	}

	return json.Marshal(s)
}

func main() {
	blob := `["gopher","armadillo","zebra","unknown","gopher","bee","gopher","zebra"]`
	var zoo []Animal
	if err := json.Unmarshal([]byte(blob), &zoo); err != nil {
		log.Fatal(err)
	}

	census := make(map[Animal]int)
	for _, animal := range zoo {
		census[animal] += 1
	}

	fmt.Printf("Zoo Census:\n* Gophers: %d\n* Zebras:  %d\n* Unknown: %d\n",
		census[Gopher], census[Zebra], census[Unknown])

}

Pretty nice, no?

Golang / The Go Programming Language

The Go Programming language (often written as golang)¬†has become quite popular recently. Google¬†is actively backing the project ¬†but Golang has seen usage, contribution and success stories from many other popular brands and enterprises across the internet. Go promises a very simple, easy to learn syntax that allows us to build robust, reliable, efficient software. Once we have had invested some time writing ¬†production grade code in the language, we would agree that Go delivers on it’s promises. It’s indeed a fantastic language – easy to learn, easy to read, reason about and of course maintain.¬†You get superb performance without sacrificing much on productivity. Don’t just trust my words, give it a Go! (See what I did there?)

A little history

Work on Golang started back in 2007 by Robert Griesemer, Rob Pike, and Ken Thompson at Google. So that kinds of make Golang 10 years old in 2017. Although work started in 2007, the language was however announced in 2009. It reached version 1.0 in 2012.

A little about the creators – Rob Pike was a member of the unix team and he is known for his work on Plan 9. Ken Thompson designed and implemented Unix. He also created the B language (from which C was inspired). He was also involved in the Plan 9 project.

Why did they start working on a new language? Because they were frustrated with the ones that existed.¬†You can choose a dynamic language like Python / Javascript and enjoy the ease of programming in them. Or you can choose something like C/C++ to¬†get performance. But at the same time you lose the ease of programming, productivity reduces and the compilation time can sometimes get too long. More and more developers were choosing dynamic languages in their projects, essentially favouring the ease of use over the safety and performance offered by the statically typed, compiled languages.¬†There wasn’t any popular, easy to use, mainstream language that could ease these problems for the developers. You couldn’t get ease of programming, safety, efficiency, fast compilation – all from a single language.

The creators of Golang saw this problem as an opportunity to create a better language that could solve these problems at hand.

Why did Golang become so popular?

Go came with better offerings, specially with solutions to what many of us (including the Googlers) faced. When I read (or write) Go code, I feel like the following equation makes perfect sense => C + Python = Golang. Go is very very fast. Not just the language but also the compiler. Go compiles fast, runs fast. And you still feel quite productive, much more productive than you would feel in C++ or Java.

The syntax is simple. You won’t have to remember many keywords. ¬†The static typing provides safety to a great extent. IDEs can provide better code completion and refactoring assistance.¬†Compiler would help you reduce bugs by¬†catching many potential errors before the program even starts to run (which is applicable to all statically typed languages, nothing Golang specific but with Go’s “light on keywords” design, it’s just more productive and enjoyable).

Go provides a nice, extensive standard library with all the batteries you might need for day to day system or network programming. You want to build an awesome web app? Go standard library has you (mostly) covered.

The major win for Go is perhaps the concurrency primitives. We can create very light weight threads called goroutine which are multiplexed on all available CPU cores. We can easily communicate between the goroutines using channels. I personally found the goroutine and channels based way of writing efficient concurrent programs very easy, elegant and pleasant.¬†Don’t be afraid of writing highly concurrent programs any more!

Golang also compiles everything and generates a single binary that includes everything that you need to run the program. You do not need to have anything installed on your target machine to run the binary. This is a huge win in terms of deployment. Writing and distributing command line tools have never been easier!

How popular is Golang?

It’s quite popular. Golang is currently ranked 16 on the TIOBE Index. Tiobe also declared Go to be the language of the year in 2016. Interestingly they also chose Go as the language of the year back in 2009, the year it was released. It stands 15th on the RedMonk language ratings. If you compare¬†active repositories using Githut, Go would be the 14th most active language on Github.¬†For a new programming language, that has released it’s first major version in 2012, that ranking is quite impressive. If you think about the growth of the language, it’s been getting a lot of traction.

In the latest StackOverflow Survey (2017), Go is 3rd on the most wanted list and 5th on the most loved list. That tells us something, doesn’t it? ūüėÄ

gopher with a gun
Gopher with a Gun!

How’s the job market, you ask? Don’t worry, Go has jobs for you!

Also check out the HackerNews “Who is hiring?” threads. I see a lot of Golang related job postings there.¬†And do not¬†forget, Golang pays well.

I want to learn Golang

I am sure you do.  So where can we learn Golang by ourselves? These are my recommendations:

  • The Official Tour is excellent. I recommend it to all the beginners. You will have a very good basic once you complete the tour. The tour is also available offline in case you want to learn some golang while on vacation or on the go.
  • Go By Example¬†is another excellent step by step tour. I always refer to this site when I need to lookup a syntax or need a quick reference.
  • Effective Go is a nice read if you want to learn the best practices and want to write idiomatic Go codes.
  • The Little Go Book¬†is a great free book for learning Go.
  • Official Documentation¬†is something we all should keep close to us while writing some Go code.
  • The Go Blog¬†– This is something I really really love. The official Go blog has many interesting and detailed stuff. Trust me, there’s a lot to learn from there. Highly recommended.

If you need some help, please feel free to ask in Gophers Slack or the Go Forum maintained by the GoBridge people.