Subscribe to the Ardan Labs Insider

You’ll get our FREE Video Series & special offers on upcoming training events along with notifications on our latest blog posts.

Included in your subscription
  • Access to our free video previews
  • Updates on our latest blog posts
  • Discounts on upcoming events

Valid email required.

Submit failed. Try again or message us directly at

Thank You for Subscribing

Check your email for confirmation.

Building A Weather App Using Go

Author image
William Kennedy

Screen Shot At Ardan Studios we have spent the last 6 months, in our spare time and on weekends, building a consumer based mobile application called OutCast. The mobile application is tailored towards those who like spending time outdoors, whether that be fishing, hunting or any other type of activity.

Screen Shot This first release of OutCast shows the conditions for the buoy stations and marine forecasts areas within the United States. All this information is updated every 10 minutes and there are map views with traditional grids and search.

The backend processing for buoy and marine data is built using Go and running at Iron.IO as a scheduled worker task. The buoy processing downloads a text file from the NOAA website and rips through it, updating MongoDB with any changes. The marine processing is a bit more complicated. This requires pulling down multiple web pages from the NOAA website and parsing out all the text. Go made building and running these tasks a breeze.

Another important aspect of OutCast is real time weather radar for the last 50 minutes. This has been very challenging on multiple levels. Mainly because we needed a real good image library that would run on Linux and could be integrated with Go. We were fortunate to find ImageMagick’s MagickWand C API and their Go package that provides the CGO bindings (

Processing images is an intense piece of work. Sometimes it takes 3 seconds to clean a single image. With 155 radar stations that need to be processed every 5 minutes, it took us several refactors to get things working well. The MagickWand library can only handle processing one image at a time. This restriction places a lot of stress on getting things done accurately within an acceptable amount of time.

Here is a sample of a radar image before and after processing:

There is another interesting constraint. NOAA updates the images every 120 seconds on different time boundaries. If the program can’t download all the images very quickly, the application could have images out of sync when they are animated across the map. Many radar images cross each other like in Orlando, FL. On this area of the map we have 3 radar images overlapping each other.

Sometimes the images are not available. The program goes out to get the image and it doesn’t exist. This creates problems with gaps in the timeline. When this happens, there is an alternate image location the program attempts to use. If the image is still not available, then the image from the previous run is used. Unless you are looking for it, you usually can’t tell.

Then you have the issue of updating S3 storage and MongoDB for each image. This again needs to happen quickly to prevent image overlays from being out of sync. At the end of the day, you want to do your best to make sure that the images for all the radar stations are in sync. This will provide the best user experience.

Radar image processing happens in three stages and runs on a VM at Digital Ocean with 2 Gig of memory and 2 Cores.

So how does Go help make this all happen every 5 minutes all day and all night without fail?

Stage 1: Download Images
There are 155 images that have to be downloaded. The images range from 1k to 20k in size depending on the activity of the weather at that moment. In this stage, the program spawns a Go routine for each image that needs to be downloaded. The Go program consistently does all 155 downloads in less than one second:

15:15:02 radar.go:425: main : downloadImages : Started

– Spawn Go Routines
15:15:02 radar.go:431: main : downloadImages : Info : Image [1] of [155]
15:15:02 radar.go:431: main : downloadImages : Info : Image [155] of [155]

– Sample Download Per Image
Worker-JAX : downloadImage : Info : HEADER : Url , N0R/JAX_N0R_0.gif
Worker-JAX : downloadImage : Info : HEADER : Last-Modified , [Fri, 06 Dec 2013 20:12]
Worker-JAX : downloadImage : Info : HEADER : Content-Type , [image/gif]
Worker-JAX : downloadImage : Info : HEADER : Vary , [Accept-Encoding]
Worker-JAX : downloadImage : Info : HEADER : Cache-Control , [max-age=180]
Worker-JAX : downloadImage : Info : HEADER : Expires , [Fri, 06 Dec 2013 20:18:02 GMT]
Worker-JAX : downloadImage : Info : HEADER : Date , [Fri, 06 Dec 2013 20:15:02 GMT]
Worker-JAX : downloadImage : Info : HEADER : Connection , [keep-alive]
Worker-JAX : downloadImage : Info : HEADER : Server , [Apache/2.2.15 (Red Hat)]
Worker-JAX : downloadImage : Info : HEADER : Accept-Ranges , [bytes]
Worker-JAX : downloadImage : Info : HEADER : Content-Length , -1
Worker-JAX : downloadImage : Info : HEADER : Image-Length , 6873

– All Images Complete
15:15:02 radar.go:445: main : downloadImages : Completed

Stage 2: Image Cleanup
Now that all 155 images have been downloaded, they need to be cleaned using the ImageMagick API. Unfortunately, this can only be done with a single Go routine. Trying to clean more than one image at a time causes the program to pull a lot of memory. It really slows things down and can cause the program to be terminated by the OS. I have also seen other very odd behavior. The program will consistently complete this work in 90 seconds or less:

15:15:02 radar.go:453: main : cleanImages : Started
15:15:02 radar.go:457: main : cleanImages : Info : Image [1] of [155]

– Sample Processing Per Image
Worker-RIW : cleanImage : Started
Worker-RIW : cleanImage : Info : ReadImageBlob
Worker-RIW : cleanImage : Info : TransparentPaintImage
Worker-RIW : cleanImage : Info : WaveImage
Worker-RIW : cleanImage : Info : Crop
Worker-RIW : cleanImage : Info : Resize
Worker-RIW : cleanImage : Info : EqualizeImage
Worker-RIW : cleanImage : Info : GaussianBlurImage
Worker-RIW : cleanImage : Info : BrightnessContrastImage
Worker-RIW : cleanImage : Info : ResetIterator
Worker-RIW : cleanImage : Info : GetImageBlob
Worker-RIW : cleanImage : Completed
Worker-RIW : cleanImage : Info : Defer : PixelWand Destroy
Worker-RIW : cleanImage : Info : Defer : MagicWand Destroy

– All Images Complete
15:16:20 radar.go:477: main : cleanImages : Completed

Stage 3: Upload To S3 and Update MongoDB
The last stage requires uploading all the cleaned images to S3 storage, removing the old images from S3 and then updating MongoDB with the new list of available image files. I am using the goamz package from Ubuntu for accessing S3 and the mgo package from Gustavo Neimeyer to access MongoDB.

Just like when we download the images, this stage spawns a Go routine for each image that needs to be uploaded and recorded. The Go program consistently performs this work in one second:

15:16:20 radar.go:485: main : updateImages : Started
15:16:20 radar.go:491: main : updateImages : Info : Image [1] of [155]
15:16:20 radar.go:491: main : updateImages : Info : Image [2] of [155]
15:16:20 radar.go:491: main : updateImages : Info : Image [3] of [155]

– Sample Processing Per Image
collateImage : Started : StationId[RIW] FileName[US/WY/RIW/20131206-2015.gif]
collateImage : Info : Remove : Minutes[50.01] FileName[US/WY/RIW/20131206-1935]
collateImage : Info : Keep : Minutes[45.00] FileName[US/WY/RIW/20131206-1940.gif]
collateImage : Info : Keep : Minutes[40.00] FileName[US/WY/RIW/20131206-1945.gif]
collateImage : Info : Keep : Minutes[35.00] FileName[US/WY/RIW/20131206-1940.gif]
collateImage : Info : Keep : Minutes[30.01] FileName[US/WY/RIW/20131206-1945.gif]
collateImage : Info : Keep : Minutes[25.02] FileName[US/WY/RIW/20131206-1950.gif]
collateImage : Info : Keep : Minutes[20.01] FileName[US/WY/RIW/20131206-1955.gif]
collateImage : Info : Keep : Minutes[15.01] FileName[US/WY/RIW/20131206-2000.gif]
collateImage : Info : Keep : Minutes[10.00] FileName[US/WY/RIW/20131206-2005.gif]
collateImage : Info : Keep : Minutes[5.01] FileName[US/WY/RIW/20131206-2010.gif]
collateImage : Info : Keep - New : FileName[US/WY/RIW/20131206-2015.gif]
collateImage : Completed
storeImageMongoDB : Info : Updating Mongo
storeImageMongoDB : Completed
storeImageS3 : Started : Bucket[SRR-Dev] FileName[US/AK/APD/20131206-2015.gif]
storeImageS3 : Info : Putting File Into S3 : FileName[US/AK/APD/20131206-2015.gif]
storeImageMongoDB : Completed

– All Images Complete
15:16:21 radar.go:505: main : updateImages : Completed

This project has taught me a lot about Go. It is exciting to see how fast the Go routines can download the images and perform all the S3 and MongoDB work. Thanks to CGO, I was able to leverage a powerful image processing library and make calls directly from my Go code.

Currently we are porting the web service that powers the mobile application to Go. It is currently written in Ruby. We are using the beego package for our web framework, the goconvey package for our tests and the envconfig package to handle our configuration needs.

Our goal for OutCast is to provide people the ability to know in advance that the weekend is going to be great. We plan on using Go and MongoDB to analyze outdoor condition data with user preferences and experiences to deliver relevant information and forecasting. In the future, users will interact with OutCast by providing an experience review after their outdoor activities have ended.

Currently OutCast is only available in the Apple App Store. We will have our Android version complete in January 2014.

Go Training

We have taught Go to thousands of developers all around the world since 2014. There is no other company that has been doing it longer and our material has proven to help jump start developers 6 to 12 months ahead of their knowledge of Go. We know what knowledge developers need in order to be productive and efficient when writing software in Go.

Our classes are perfect for both experienced and beginning engineers. We start every class from the beginning and get very detailed about the internals, mechanics, specification, guidelines, best practices and design philosophies. We cover a lot about "if performance matters" with a focus on mechanical sympathy, data oriented design, decoupling and writing production software.

Capital One
Red Ventures

Interested in Ultimate Go Corporate Training and special pricing?

Let’s Talk Corporate Training!

Join Our Online
Education Program

Our courses have been designed from training over 30,000 engineers since 2013, and they go beyond just being a language course. Our goal is to challenge every student to think about what they are doing and why.