Introduction
In part 1 we took a higher level view on serialization in general and JSON in specific. In part 2 we looked at emitting JSON.
In part 3, we’ll look at an issue you might encounter when consuming JSON, Zero vs NULL field values. To clarify the definition of NULL, this means the absence of value.
So here is the question:
Given a field in a Go struct set to its zero value, how do you know that zero value was set by the user or it’s zero because it was never provided?
By the way, this issue is not just restricted to JSON, you can have the same issue when working with maps, channels, database queries, and more.
Simple Unmarshalling
Let’s start with understanding basic unmarshalling.
Say we have a start VM (VM = virtual machine) request. In JSON, the request may look like this:
Listing 1: Start VM request
01 {
02 "image": "debian:bookworm-slim",
03 "count": 1
04 }
Listing 1 shows the “start VM” request in JSON format. On line 02, we have the image name and on line 03, we have the machine count.
So on the Go side, the first thing we need is to define a StartVM struct:
Listing 2: StartVM
08 type StartVM struct {
09 Image string
10 Count int
11 }
Listing 2 shows the StartVM struct. It has two fields: Image and Count.
The encoding/json package APIs only work with exported fields and when JSON field tags are not provided, the package will use a case-insensitive match of the field name.
Note: To apply JSON field tags review this json.Marshal. documentation
And now we can unmarshal data into this struct:
Listing 3: Unmarshalling JSON
14 data := `
15 {
16 "image": "debian:bookworm-slim",
17 "count": 1
18 }`
19
20 var req StartVM
21 if err := json.Unmarshal([]byte(data), &req); err != nil {
22 return err
23 }
24
25 fmt.Printf("%#v\n", req)
26 return nil
Listing 3 shows a simple example of unmarshalling JSON into a Go struct. On lines 14-18, we define a JSON document. On line 20, we create a variable of type StartVM and then on lines 21-23, we use the json.Unmarshal function to extract the data from the JSON document and populate the fields in the req variable.
This code prints:
Listing 4:
main.StartVM{Image:"debian:bookworm-slim", Count:1}
Zero Values vs NULL
The Unmarshal function will not fail if fields are not present from the JSON document or not present from the Go struct. This allows us to define only the fields we are interested in, but this can cause issues at times when it comes to NULL vs Zero value.
Note: The json.Decoder type has a DisallowUnknownFields method that causes the decoder to return an error if the JSON contains fields that are not present in the struct. This approach works well if you need to validate the exact JSON input.
Let’s see an example:
Listing 5: Missing “count”
08 type StartVM struct {
09 Image string
10 Count int
11 }
31 data := []byte(`{"image": "debian:bookworm-slim"}`)
32
33 var req StartVM
34 if err := json.Unmarshal(data, &req); err != nil {
35 return err
36 }
37
38 fmt.Printf("%#v\n", req)
39 return nil
Listing 5 shows the unmarshalling of a JSON document where the count field is missing from the JSON.
This code will print:
Listing 6:
main.StartVM{Image:"debian:bookworm-slim", Count:0}
How can you know if the user didn’t send the count field as part of the JSON document (NULL) or if it was included and set to 0 (Zero Value)? In some cases, this might not be an issue. But assume that the requirements for this API endpoint are: “If the user does not pass count, use 1 as the count, otherwise validate that the count is bigger than 0.”
I’ll show you three ways to deal with the “Zero Value vs NULL” problem:
- Using Pointer Semantics
- Using A Map
- Using Default Values
Using Pointer Semantics
Using pointer semantics will allow you to determine if a field was provided in the JSON document by leveraging the semantic of nil for NULL.
Listing 7: StartVM Using Pointer Fields
08 type StartVM struct {
09 Image string //
10 Count *int // Changed to a pointer of type int
11 }
48 data := []byte(`{"image": "debian:bookworm-slim"}`)
49
50 var req StartVM
51 if err := json.Unmarshal(data, &req); err != nil {
52 return err
53 }
54
55 if req.Count == nil { // User didn't send "count", use default value
56 v := 1
57 req.Count = &v
58 }
59
60 if *req.Count < 1 {
61 return fmt.Errorf("bad count: %d", *req.Count)
62 }
63
64 fmt.Printf("%#v\n", req)
65 return nil
Listing 7 shows how to use pointer semantics to know if a field was not provided in the JSON document. On lines 08-11, we change the fields in the StartVM struct to use pointer semantics.
Then on lines 50-53, we unmarshal the JSON document and then check if the Count field is nil (meaning the user didn’t provide the count field). If the Count field is nil, the Count field is defaulted to 1. Finally on line 60-62, we validate if the Count field is valid by checking it’s not less than 1.
This code will print:
Listing 8:
main.StartVM{Image:"debian:bookworm-slim", Count:0xc000098170}
Many Go developers choose this option because it’s quick and simple to implement. You do need to be careful of nil pointer errors like dereferencing a nil pointer and causing a panic.
Using A Map
Using a map is a second option. We’ll use a map declared as map[string]any in Go since this represents the type system of a JSON document.
Listing 9: Using A Map
08 type StartVM struct {
09 Image string
10 Count int
11 }
69 data := []byte(`{"image": "debian:bookworm-slim"}`)
70
71 var m map[string]any
72 if err := json.Unmarshal(data, &m); err != nil {
73 return err
74 }
75
76 if _, ok := m["count"]; !ok { // User didn't send "count", use default value
77 m["count"] = 1.0
78 }
79
80 image, ok := m["image"].(string)
81 if !ok || image == "" {
82 return fmt.Errorf("bad image: %#v", m["image"])
83 }
84
85 count, ok := m["count"].(float64)
86 if !ok {
87 return fmt.Errorf("bad count: %#v", m["count"])
88 }
89
90 if count < 1 {
91 return fmt.Errorf("bad count: %f", count)
92 }
93
94 req := StartVM{
95 Image: image,
96 Count: int(count),
97 }
98
99 fmt.Printf("%#v\n", req)
100 return nil
Listing 9 shows how to use a Go map. On lines 71-74, we unmarshal the JSON document into a Go map. Then on lines 76-78, we check if the count field was provided in the JSON document and if not, the count key in the map is set to 1.0.
On lines 80-92, we validate the JSON document provided values we expect in the proper type.
One line 94-97 we define a variable of type StartVM and set the values from the map.
_Note: Instead of manually converting the map to a struct, you can use this cool package called mapstructure.`
This approach is more tedious and complex, but it will work. It’s important to note that by default encoding/json converts numbers to float64 and you’ll need to convert them to int (or any other type you require).
Using Default Values
Using default values can help to simplify the Zero Value vs NULL issues when this is an option. For this to work, the default value can’t match the zero value for the specified field.
Listing 10: Using Default Values
08 type StartVM struct {
09 Image string
10 Count int
11 }
104 data := []byte(`{"image": "debian:bookworm-slim"}`)
105
106 req := StartVM{
107 Count: 1,
108 }
109
110 if err := json.Unmarshal(data, &req); err != nil {
111 return err
112 }
113
114 if req.Count < 1 {
115 return fmt.Errorf("bad count: %d", req.Count)
116 }
117
118 fmt.Println(req)
119
120 return nil
Listing 10 shows how to use default values. On lines 106-108, we create a StartVM variable and set the Count field to 1 as part of the construction. Then on lines 110-112, we unmarshal the JSON document and then verify as before the final value of the count field. If the count field is provided in the JSON document, the default value will be replaced with what was provided. In this example the count field is not provided so the count field will be 1.
When you have a situation where a default value can be used and it won’t represent the zero value for the field, this is a great option.
Conclusion
The “Zero Value vs NULL” problem can present itself in several places when coding in Go, including the process of unmarshalling JSON. In this article I’ve shown you three ways to solve this issue. I hope you learned the power of default values and consider their use in the future.
Do you have other methods of dealing with this issue? I’d love to hear, mail me at miki@ardanlabs.com.
