Subscribe to the Ardan Labs Insider

You’ll get our FREE Video Series & special offers on upcoming training events along with notifications on our latest blog posts.

Included in your subscription
  • Access to our free video previews
  • Updates on our latest blog posts
  • Discounts on upcoming events

Valid email required.

Submit failed. Try again or message us directly at

Thank You for Subscribing

Check your email for confirmation.

Decode JSON Documents In Go

Author image
William Kennedy

We are working on a project where we have to make calls into a web service. Many of the web calls return very large documents that contain many sub-documents. The worst part is, we usually only need a handful of the fields for any given document and those fields tend to be scattered all over the place.

Here is a sample of a smaller document:

var document string = {<br /> &quot;userContext&quot;: {<br /> &nbsp; &nbsp; &quot;conversationCredentials&quot;: {<br /> &nbsp; &nbsp; &nbsp; &nbsp; &quot;sessionToken&quot;: &quot;06142010_1:75bf6a413327dd71ebe8f3f30c5a4210a9b11e93c028d6e11abfca7ff&quot;<br /> &nbsp; &nbsp; },<br /> &nbsp; &nbsp; &quot;valid&quot;: true,<br /> &nbsp; &nbsp; &quot;isPasswordExpired&quot;: false,<br /> &nbsp; &nbsp; &quot;cobrandId&quot;: 10000004,<br /> &nbsp; &nbsp; &quot;channelId&quot;: -1,<br /> &nbsp; &nbsp; &quot;locale&quot;: &quot;en_US&quot;,<br /> &nbsp; &nbsp; &quot;tncVersion&quot;: 2,<br /> &nbsp; &nbsp; &quot;applicationId&quot;: &quot;17CBE222A42161A3FF450E47CF4C1A00&quot;,<br /> &nbsp; &nbsp; &quot;cobrandConversationCredentials&quot;: {<br /> &nbsp; &nbsp; &nbsp; &nbsp; &quot;sessionToken&quot;: &quot;06142010_1:b8d011fefbab8bf1753391b074ffedf9578612d676ed2b7f073b5785b&quot;<br /> &nbsp; &nbsp; },<br /> &nbsp; &nbsp; &quot;preferenceInfo&quot;: {<br /> &nbsp; &nbsp; &nbsp; &nbsp; &quot;currencyCode&quot;: &quot;USD&quot;,<br /> &nbsp; &nbsp; &nbsp; &nbsp; &quot;timeZone&quot;: &quot;PST&quot;,<br /> &nbsp; &nbsp; &nbsp; &nbsp; &quot;dateFormat&quot;: &quot;MM/dd/yyyy&quot;,<br /> &nbsp; &nbsp; &nbsp; &nbsp; &quot;currencyNotationType&quot;: {<br /> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;currencyNotationType&quot;: &quot;SYMBOL&quot;<br /> &nbsp; &nbsp; &nbsp; &nbsp; },<br /> &nbsp; &nbsp; &nbsp; &nbsp; &quot;numberFormat&quot;: {<br /> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;decimalSeparator&quot;: &quot;.&quot;,<br /> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;groupingSeparator&quot;: &quot;,&quot;,<br /> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &quot;groupPattern&quot;: &quot;###,##0.##&quot;<br /> &nbsp; &nbsp; &nbsp; &nbsp; }<br /> &nbsp; &nbsp; &nbsp;}<br /> },<br /> &quot;lastLoginTime&quot;: 1375686841,<br /> &quot;loginCount&quot;: 299,<br /> &quot;passwordRecovered&quot;: false,<br /> &quot;emailAddress&quot;: &quot;;,<br /> &quot;loginName&quot;: &quot;sptest1&quot;,<br /> &quot;userId&quot;: 10483860,<br /> &quot;userType&quot;: {<br /> &nbsp; &nbsp; &quot;userTypeId&quot;: 1,<br /> &nbsp; &nbsp; &quot;userTypeName&quot;: &quot;normal_user&quot;<br /> }<br /> }

It is not scalable for us to create all the structs and embedded structs to unmarshal the different JSON documents using json.Unmarshal and working directly with a map was out of the question. What we needed was a way to decode these JSON documents into structs that just contained the fields we needed, regardless where those fields lived in the JSON document.

Luckily we came a across a package by Mitchell Hashimoto called mapstructure and we forked it. This package is able to take a JSON document that is already unmarshaled into a map and decode that into a struct. Unfortunately, you still needed to create all the embedded structs if you wanted the data at the different levels. So I studied the code and build some functionality on top that allowed us do what we needed.

The first function we added is called DecodePath. This allows us to specify the fields and sub-documents we want from the JSON document and store them into the structs we need. Let’s start with a small example using the JSON document above:

type UserType struct {
    UserTypeId int
    UserTypeName string

type User struct {
    Session   string   jpath:&quot;userContext.cobrandConversationCredentials.sessionToken&quot;
    CobrandId int      jpath:&quot;userContext.cobrandId&quot;
    UserType  UserType jpath:&quot;userType&quot;
    LoginName string   jpath:&quot;loginName&quot;

docScript := []byte(document)
docMap := map[string]interface{}{}
json.Unmarshal(docScript, &docMap)

user := User{}
DecodePath(docMap, &user)

fmt.Printf("%#v", user)

If we run this program we get the following output:


The "jpath" tag is used to find the map keys and set the values into the struct. The User struct contains fields from three different layers of the JSON document. We only needed to define two structs to pull the data out of the map we needed.

We can also map entire structs the same way a normal unmarshal would work. Just name the fields in the struct to match the field names in the JSON document. The names of the fields in the struct don’t need to be in the same case as the fields in the JSON document.

Here is a more complicated example using an anonymous field in our struct:

type NumberFormat struct {
    DecimalSeparator  string jpath:&quot;userContext.preferenceInfo.numberFormat.decimalSeparator&quot;
    GroupingSeparator string jpath:&quot;userContext.preferenceInfo.numberFormat.groupingSeparator&quot;
    GroupPattern      string jpath:&quot;userContext.preferenceInfo.numberFormat.groupPattern&quot;

type User struct {
    LoginName string jpath:&quot;loginName&quot;

docScript := []byte(document)
docMap := map[string]interface{}{}
json.Unmarshal(docScript, &docMap)

user := User{}
DecodePath(docMap, &user)

fmt.Printf("%#v", user)

If we run this program we get the following output:


We can also use an anonymous field pointer:

type User struct {
    LoginName string jpath:&quot;loginName&quot;

In this case DecodePath will instantiate an object of that type and perform the decode, but only if a mapping can be found.

We now have great control over decoding JSON documents into structs. What happens when the JSON you get back is an array of documents?

There are times when the web api returns an array of JSON documents:

var document = [{&quot;name&quot;:&quot;bill&quot;},{&quot;name&quot;:&quot;lisa&quot;}]

In this case we need to decode the slice of maps into a slice of objects. We added another function called DecodeSlicePath that does just that:

type NameDoc struct {
    Name string jpath:&quot;name&quot;

sliceScript := []byte(document)
sliceMap := []map[string]interface{}{}
json.Unmarshal(sliceScript, &sliceMap)

var myslice []NameDoc
DecodeSlicePath(sliceMap, &myslice)

fmt.Printf("%#v", myslice)

Here is the output:


The function DecodeSlicePath creates the slice based on the length of the map and then decodes each JSON document, one at a time.

If it were not for Mitchell I would not have been able to get this to work. His package is brilliant and handles all the real technical issues around decoding maps into structs. The two functions I have built on top of mapstructure provides a nice convenience factor we needed for our project. If you’re dealing with some of the same issue, please try out the package.

Go Training

We have taught Go to thousands of developers all around the world since 2014. There is no other company that has been doing it longer and our material has proven to help jump start developers 6 to 12 months ahead of their knowledge of Go. We know what knowledge developers need in order to be productive and efficient when writing software in Go.

Our classes are perfect for both experienced and beginning engineers. We start every class from the beginning and get very detailed about the internals, mechanics, specification, guidelines, best practices and design philosophies. We cover a lot about "if performance matters" with a focus on mechanical sympathy, data oriented design, decoupling and writing production software.

Capital One
Red Ventures

Interested in Ultimate Go Corporate Training and special pricing?

Let’s Talk Corporate Training!

Join Our Online
Education Program

Our courses have been designed from training over 30,000 engineers since 2013, and they go beyond just being a language course. Our goal is to challenge every student to think about what they are doing and why.