Learning JavaScript

You really don’t have to check where JavaScript stands in the TIOBE index, it is undoubtedly the most popular programming language in today’s world. JavaScript started big in the browser land but with the inception and rise of NodeJS, it has come out of it’s browser context and now being used in all sorts of tools and toys. From modern progressive web apps to desktop apps, from command line tools to IoT – JavaScript is literally everywhere. The language has it’s quirks, it’s not perfect and it gets a lot hatred from time to time. Despite the backlash, JavaScript continues to grow and get more and more mind share. Specially if you’re interested in web development, sooner or later you would need some level of skills in the language. So in this blog post, I will go ahead and list some of the resources that can help beginners in the JS world.

Learning The Language

JavaScript, the language can be used in the browser or on the server side through a framework like NodeJS. You can also use JS in different other places for scripting purposes. Depending on where we’re using the language, the libraries or the APIs can vary. But the language – it remains more or less the same. So before you want to start learning the next modern front end framework or the latest MVC framework in the Node land, you probably want to learn the language first. You should learn basics very well and have a solid grasp of the language before you choose your platform / stack.

  • Speaking JavaScript¬†is an excellent book to get started with ES5. The same author brings you Exploring ES6.¬†¬†And that’s not all, he also has another book – Exploring ES2016 and ES2017. You can probably already guess – the author, Dr. Axel Rauschmayer is pretty awesome. Reading these books will definitely get your basics strong. You can read these books free online.
  • Eloquent Javascript¬†is also free online¬†and should help you learn both the language and how it works in the browser.
  • JavaScript: The Good Parts is another excellent book. The book is not very big in terms of volume but it covers the language pretty well.
  • You Don’t Know JS¬†is the title of a series of books written by Kyle Simpson. You can read the books free online. This series is pretty awesome!
  • JavaScript Allong√©, the “Six” Edition – can also be read free online and packs lots of excellent content.

You may consult multiple books to understand the same topic. But it would be a good idea to start one book and stick to it.

Practising JavaScript

When reading through the concepts, please do try out the codes yourself. Just reading through won’t help much if you don’t practise. You can practise JavaScript in these places:

  • HackerRank – solve programming problems with JS here.
  • You can also train JS at CodeWars.
  • How about learning JS while playing some games? CodeCombat has you covered!
  • You can solve problems at Project Euler too!

What’s Next?

Once you have learned the language, you can choose to work on the front end or the backend or you can choose to be a full stack JS developer! The choice is yours. Do try both and see which ones you like. Having a solid understanding of the language will make it easier for you to pick up any stack / framework. So put extra focus on learning the concepts well and taking your JS skills to intermediate level!

Golang: Working with JSON

We previously wrote an introductory blog post on Golang describing why and how the language is gaining traction quickly. Go aka Golang is being used by many large scale systems for various purposes. One thing common in most complex systems is that we have to communicate with other systems / services. JSON is a very popular data interchange format for this kind of scenarios. In fact, it is so popular that you may even go ahead and call it the de-facto data interchange format on the internet.  If you have followed any of our REST API tutorials (Django / Flask), you might have also noticed they all output JSON. In this blog post, we will see the different ways we can work with JSON in Golang.

Creating JSON

We can use the encoding/json¬†package to easily create JSON from our Go data structures. There’s a few things to consider while creating JSON in Go. One of them is the choice of data structure.

From Structs

Structs are very convenient to use and often come as cheaper compared to map or other complex structures. We can pass any instance of a struct to the json.Marshal function and get back the JSON as a slice of bytes ([]byte).

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string
	Age    int
	Emails []string
}

func main() {
	masnun := Person{
		Name:   "Abu Ashraf Masnun",
		Age:    27,
		Emails: []string{"[email protected]", "[email protected]"},
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s",json_bytes)
}

Output should be:

{"Name":"Abu Ashraf Masnun","Age":27,"Emails":["[email protected]","[email protected]"]}

Things to note here:

  • The field name has to start with a capital letter – this is important and will be discussed soon
  • You can nest other structures, in this example our Emails¬†field contain a list of strings
  • The json.Marshal¬†function returns the JSON and error, don’t forget to handle the error. You may not often get errors from the Marshal¬†function but in some cases where some types can not be converted into JSON, you will get errors. So look out for it.
  • The returned JSON is in the form of bytes list. So if you want to use it as string, you will need to convert it.

Choosing Field Names

If you have seen the output, the keys/fields in the created JSON structure all start with a capital letter (our struct fields were similar, so this is no surprise). If you have a curious and adventurous mind, you might have tried to go ahead and convert the struct fields into lower caps. But that doesn’t work, does it? Check it out –¬†https://play.golang.org/p/93eDoFSjnW¬†– because if a field / member name does not start with a capital letter, it is not exported. External code can not access it. This is why our name¬†and age¬†fields are not part of the generated JSON structure.

But don’t worry, there’s a simple way of “tagging” our struct fields so we can describe how to marshal our structs. Let’s modify our codes to look like this:

type Person struct {
	Name   string   `json:"name"`
	Age    int      `json:"age"`
	Emails []string `json:"emails"`
}

Next to each field, we have provided tags to describe how this field should be marshalled or unmarshalled. Now we can see you expected results here:¬†https://play.golang.org/p/xlcjU1_VSE¬†ūüôā

If you don’t understand the tags, don’t worry, try reading this answer on StackOverflow –¬†https://stackoverflow.com/questions/10858787/what-are-the-uses-for-tags-in-go. We will understand the concepts more when we use structs for various purposes (for example mapping to database tables, but for now, let’s not worry).

Omitting Empty Fields

What if some of the fields are empty? Let’s try that.

func main() {
	masnun := Person{}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

The output would be:

{"name":"","age":0,"emails":null}

If a field is empty, we might want to omit that from our JSON. We can do so by using the omitempty flag in our json field tags.

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string      `json:"name,omitempty"`
	Age    int         `json:"age,omitempty"`
	Emails []string    `json:"emails,omitempty"`
}

func main() {
	masnun := Person{
		Name: "Abu Ashraf Masnun",
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

Now if we check the output again:

{"name":"Abu Ashraf Masnun"}

Nice, no?

Skipping Fields

Let’s say in our struct, we need to keep the Age field. But we don’t want it to be a part of the produced JSON. We can use the -¬†flag in the json tag for that particular field.

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string      `json:"name,omitempty"`
	Age    int         `json:"-"`
	Emails []string    `json:"emails,omitempty"`
}

func main() {
	masnun := Person{
		Name: "Abu Ashraf Masnun",
		Age:  27,
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

The output would be:

{"name":"Abu Ashraf Masnun"}

Even though the struct had the Age field set, it wasn’t included in the output. This comes very handy in some cases, for example when a¬†User¬†struct has a Password¬†field that we don’t want to serialize into JSON.

Using Maps and Slices

So far we have used structs. We can also use maps and slices instead. Here’s a quick code example:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	masnun := map[string]interface{}{

		"name": "Abu Asharf Masnun",
		"age":  27,
	}

	json_bytes, _ := json.Marshal(masnun)
	fmt.Printf("%s", json_bytes)
}

And using slices:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {
	emails := []string{"[email protected]", "[email protected]"}
	json_bytes, _ := json.Marshal(emails)
	fmt.Printf("%s", json_bytes)
}

They both work as expected. But in most codebases I have come across, structs are more widely used.

 

Parsing JSON

We have so far seen how we can generate JSON from our go data. Now we will see the opposite. We will be parsing JSON into Go data structures.

Into Structs

We will first see how we can parse JSON data into structs. It’s quite similar to what we did earlier. We will be using the Unmarshal¬†function which takes bytes and pointer to any interface{}¬†type. It reads through the JSON and stores the data in the struct we pass as the second parameter. Let’s see an example:

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name   string   `json:"name"`
	Age    int      `json:"age"`
	Emails []string `json:"emails"`
}

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"]
		}
	`)

	masnun := Person{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	fmt.Println(masnun.Name, masnun.Age, masnun.Emails)

}

Here json_bytes hold the JSON we want to process. We already have a Person type with tagged fields. We just need to pass this json_bytes and a pointer to an instance of Person to the Unmarshal function. Please note the pointer is important. We have to pass a pointer otherwise the parser would not be able to write to the struct.

If the struct doesn’t have some fields which are present in the JSON, those will be silently ignored. In the same way, if the struct has fields which are not available in the JSON, they will be ignored too.

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	Name    string   `json:"name"`
	Age     int      `json:"age"`
	Emails  []string `json:"emails"`
	Address string
}

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun Person
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	fmt.Println(masnun.Address)

}

In the above example, the struct has a field named Address¬†which the JSON doesn’t provide. On the other hand, the JSON has the Score¬†key which the struct knows nothing about. In this case, masnun.Address¬†will be empty string.

Into Maps / Slices

We have previously mentioned how structs are cheaper and more widely used than maps. But there’s these use cases where we can not be certain about the structure of the JSON data we want to parse. In such cases, maps can be very useful. Let’s see:

package main

import (
	"encoding/json"
	"fmt"
)

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun map[string]interface{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	fmt.Println(masnun["Name"], masnun["Age"], masnun["Emails"], masnun["Score"])

}

See? We have passed map[string]interface{} and received all the data in JSON. But please remember, the values to each key in the map will be of type interface{}. If we want to extract part of the data, for example, one of the emails and then use it as a string, I will have to manually convert it to a string.

For example this code will fail:

package main

import (
	"encoding/json"
	"fmt"
	"strings"
)

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun map[string]interface{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	var firstEmail string
	firstEmail = masnun["Emails"][0]

	fmt.Println(strings.ToUpper(firstEmail))

}

We will get an error:

# command-line-arguments
./main.go:27: invalid operation: masnun["Emails"][0] (type interface {} does not support indexing)

That’s what I was trying to explain just above. The Emails¬†key has a value of interface{}¬†type. Let’s cast it to a list of interface{} first. Then we can take an element (which will be again interface{}¬†type). We further cast it to a string.

package main

import (
	"encoding/json"
	"fmt"
	"strings"
)

func main() {

	json_bytes := []byte(`
		{
			"Name":"Abu Ashraf Masnun",
			"Age":27,
			"Emails":["[email protected]","[email protected]"],
			"Score":97
		}
	`)

	var masnun map[string]interface{}
	err := json.Unmarshal(json_bytes, &masnun)
	if err != nil {
		panic(err)
	}

	var emails []interface{}
	var firstEmail string
	emails = masnun["Emails"].([]interface{})
	firstEmail = emails[0].(string)

	fmt.Println(strings.ToUpper(firstEmail))

}

You may be wondering why couldn’t we just get¬†Emails¬†as¬†[]string? Well, Go doesn’t know the types of the values in our JSON. So it uses interface{}¬†. That is when it stores Emails, it stores it as a list of unknown types or a list of interface{}. That is why we first need to get it as a list and then we can take individual items and further convert them to the type we want.

Now it works fine ūüôā

Streaming JSON Encoding and Decoding

The json package offers NewEncoder and NewDecoder functions which would get us Encoder and Decoder types. These types can work with other objects that support io.Reader and io.Writer interfaces to offer streaming support.

Streaming JSON from a File

We can open a JSON file using the os.Open¬†function and stream it using the json.NewDecoder¬†function. Here’s a quick example:

package main

import (
	"encoding/json"
	"fmt"
	"os"
)

func main() {

	fileReader, _ := os.Open("masnun.json")
	var masnun map[string]interface{}
	json.NewDecoder(fileReader).Decode(&masnun)
	fmt.Println(masnun)

}

We opened a file which implements the io.Reader¬†interface. So we can use it with our Decoder¬†type. We created a decoder with the file reader and then called the Decode¬†method on it. That’s all we needed ūüôā

Streaming JSON into a File

Writing JSON is also very similar. We need to open a file in write mode, grab the io.Writer and pass it to json.NewEncoder Рthen we can pass our data to the Encode method to stream the json into the file.

package main

import (
	"encoding/json"
	"os"
)

type Person struct {
	Name   string
	Age    int
	Emails []string
}

func main() {
	masnun := Person{
		Name:   "Abu Ashraf Masnun",
		Age:    27,
		Emails: []string{"[email protected]", "[email protected]"},
	}

	fileWriter, _ := os.Create("output.json")
	json.NewEncoder(fileWriter).Encode(masnun)
}

 

Custom Marshal / Unmarshal

If we want to change how our own types are marshalled or unmarshalled, we can implement the json.Marshaler¬†and json.Unmarshaler¬†interfaces. It’s actually simple. We need to define the MarshalJSON¬†and UnmarshalJSON¬†methods on our structs and we’re done. Here’s an example from the official documentation:

package main

import (
	"encoding/json"
	"fmt"
	"log"
	"strings"
)

type Animal int

const (
	Unknown Animal = iota
	Gopher
	Zebra
)

func (a *Animal) UnmarshalJSON(b []byte) error {
	var s string
	if err := json.Unmarshal(b, &s); err != nil {
		return err
	}
	switch strings.ToLower(s) {
	default:
		*a = Unknown
	case "gopher":
		*a = Gopher
	case "zebra":
		*a = Zebra
	}

	return nil
}

func (a Animal) MarshalJSON() ([]byte, error) {
	var s string
	switch a {
	default:
		s = "unknown"
	case Gopher:
		s = "gopher"
	case Zebra:
		s = "zebra"
	}

	return json.Marshal(s)
}

func main() {
	blob := `["gopher","armadillo","zebra","unknown","gopher","bee","gopher","zebra"]`
	var zoo []Animal
	if err := json.Unmarshal([]byte(blob), &zoo); err != nil {
		log.Fatal(err)
	}

	census := make(map[Animal]int)
	for _, animal := range zoo {
		census[animal] += 1
	}

	fmt.Printf("Zoo Census:\n* Gophers: %d\n* Zebras:  %d\n* Unknown: %d\n",
		census[Gopher], census[Zebra], census[Unknown])

}

Pretty nice, no?