Ardan Labs on

Should You Learn One Programming Language or Many? A Practical Guide for Software Engineers

Tue, 24 Feb 2026 00:00:00 +0000

Engineers frequently ask whether they should specialize deeply in a single programming language or distribute their effort across several. The question is usually framed as a strategic decision, as if long-term career success depends on selecting the correct option early. That framing is misleading. Career progression is rarely determined by language count; it is determined by how well you understand systems and how effectively you reason about trade-offs within them.

Ardan Labs Connect: São Paulo, Brazil 2026

Thu, 19 Feb 2026 00:00:00 +0000

If you have experience with Go and are interested in future opportunities with Ardan Labs, we encourage you to share your CV with us. We’re always looking for Mid-Level to Senior engineers who are passionate about Go and backend development to join projects across the globe.

Range-Over Functions in Go

Mon, 16 Feb 2026 00:00:00 +0000

Iteration has long been one of the more fragmented areas of Go, with developers relying on ad hoc patterns to traverse custom data structures. This article explores the range-over-functions experiment, a proposed evolution of the language that introduces a standardized iterator model while preserving Go’s familiar for range syntax. Using the new iter package and sequence abstractions, it shows how iteration logic can be expressed more clearly, flexibly, and idiomatically. Originally published in April 2024, the concepts remain highly relevant as Go continues to evolve toward more expressive yet simple language features.

Query Database Using Plain English

Wed, 11 Feb 2026 00:00:00 +0000

Introduction

In this post you’ll see how you can create a system that allows users to query a relational database using plain English. This allows users not familiar with SQL or business intelligence systems to get insights from data.

Setting Up

If you want to follow along, you’ll need to clone the code from the GitHub repo. This will download the code, and the database file containing the data (bikes.ddb)

Note: The data is from the Austin Bike Share dataset.

Miki Tebeka at Golang-Sydney: Go Quiz

Tue, 10 Feb 2026 00:00:00 +0000

Join Miki’s Go Quiz! Tune in via live stream for a night of fun and trivia at the Sydney meet-up.

A Go Quiz

What will happen when you run the following code?

timeout := 3
fmt.Printf("before ")
time.Sleep(timeout * time.Millisecond)
fmt.Println("after")

In this interactive session, Miki will be broadcasting live to the Sydney meet-up to challenge your knowledge of Go.

The format is simple but powerful:

The Challenge: Miki shows a specific snippet of Go code.
The Guess: You’ll have time to analyze the code and commit to an answer.
The Reveal: We run the code live to see the actual output.
The Deep Dive: We discuss why it produced that result.

Research shows that making mistakes is a fantastic way to learn and retain information. We encourage you to think carefully, take a risk with your answer, and enjoy the “aha!” moments that follow.

How Rust, Go, and Blockchain Are Powering Modern Innovation

Wed, 28 Jan 2026 00:00:00 +0000

The Race to Innovate

Speed used to be an advantage. Today, it’s a baseline requirement. Teams are expected to ship quickly, scale without friction, and recover from failure without customers noticing. Whether you’re building financial infrastructure, cloud services, or global networks, the margin for error has narrowed considerably.

At the same time, systems are growing more complex. Distributed architectures are the norm. Data volumes continue to rise. Security threats evolve faster than patch cycles. Innovation, then, is no longer just about velocity. It’s about building software that can hold up under sustained pressure.

Kubernetes Memory Limits and Go

Tue, 27 Jan 2026 00:00:00 +0000

Kubernetes memory limits introduce a subtle but critical interaction with the Go runtime that can determine whether a service runs efficiently or fails under load. This article explores how Go manages memory under normal conditions and what changes when Kubernetes enforces hard memory constraints. Using controlled load testing, it shows why Go is generally excellent at self regulating memory, how OOM events emerge once limits are imposed, and how tools like GOMEMLIMIT can be used to align the runtime with container boundaries. Originally published in July 2024, the guidance remains highly relevant for teams running Go services in containerized environments where stability and throughput are tightly coupled to memory configuration.

Everything You Wanted to Ask About Rust – Answered by Herbert Wolverson

Wed, 07 Jan 2026 00:00:00 +0000

In a recent livestream with JetBrains, Vitaly Bragilevsky sat down with Herbert Wolverson, our Lead Rust Consultant and Instructor here at Ardan Labs, to talk about everything Rust developers – beginners and pros alike – are curious about.

Watch the full livestream replay on the Ardan Labs Channel and check out the GitHub repo with Herbert’s prepared answers and code samples. Below is a short summary of the 23 questions that were covered during the livestream.

Build, Connect, Grow: Ardan Labs’ 2025 in Review

Mon, 29 Dec 2025 00:00:00 +0000

Funny how fast a year moves when you are deep in code. One moment you are shipping a small improvement, the next you are looking back on a year shaped by releases, milestones, and conversations that influenced how developers learned, connected, and grew along the way.

At Ardan Labs, 2025 was defined by momentum. We expanded how engineers learn Go, Rust, and Kubernetes, introduced clearer paths for skill validation, strengthened our live education offerings, and continued building a global community grounded in craftsmanship.

ArdanLabs Connect 2026

Thu, 18 Dec 2025 00:00:00 +0000

Agenda

It is with great pleasure that we announce the 1st edition of Ardan Labs Connect in Brazil, which will take place on March 4th and 5th (Wednesday and Thursday) in São Paulo, SP. This is a two-day, in-person immersion promoted by Ardan Labs, taught entirely by William (Bill) Kennedy, author of the book Go in Action, covering the following topics:

Day 1: Ultimate Software Design (8h)

Learn how to develop production-grade software with Go by applying a domain- and data-oriented architecture designed to run on Kubernetes.

Rust Nation UK Workshop: Async - Fundamentals to Advanced

Thu, 18 Dec 2025 00:00:00 +0000

Everything You Wanted to Ask About Rust

Wed, 10 Dec 2025 00:00:00 +0000

Agenda

In this livestream, Vitaly Bragilevsky (JetBrains) will host a Q&A with Herbert Wolverson from Ardan Labs. They’ll tackle the most trending and curious topics about Rust, sharing comments and ideas throughout the session.

We’ll gather and ask your questions, and of course, include a few of our own. It promises to be an interactive and engaging event to wrap this year up.

About Herbert Wolverson:

Ardan Labs Lead Rust Instructor

Rust is Growing Fast: What Developer Debates Reveal About It

Thu, 04 Dec 2025 00:00:00 +0000

From Systems Obscurity to Mainstream Momentum

Rust didn’t start out as the go-to language for production teams. In its early days, it lived mostly in the world of systems purists, developers who cared deeply about memory, performance, and correctness at the lowest levels. The promise was ambitious: deliver safety and concurrency without sacrificing speed or control.

Fast forward to today, and Rust looks very different. It’s running in production at startups and enterprises alike, powering everything from APIs and infrastructure to blockchain platforms and performance-critical services. What changed wasn’t just the language; it was the confidence developers gained as the ecosystem matured and real-world success stories piled up. Rust stopped being a passion project and became a business-ready tool.

16 Years of Go: A Programming Language Built to Last

Mon, 10 Nov 2025 00:00:00 +0000

Every November, the Go community celebrates another milestone for one of the most quietly transformative programming languages of our time.

Born at Google and released to the world in 2009, Go (or Golang) was created to tackle the complexity of building software at scale, massive codebases, distributed systems, and teams working across continents. Sixteen years later, the same principles that shaped Go’s birth continue to guide its evolution: simplicity, speed, and reliability.

The Future We Build Together: Closing the Gender Gap in Tech

Wed, 22 Oct 2025 00:00:00 +0000

How bias, culture, and systems shape career trajectories—and what leaders can do to change them

The gender gap in tech has persisted for decades. Awareness has grown, but progress often feels slow. In an industry where less than a third of technical roles are held by women, the challenges begin long before the hiring process and extend far beyond it. Women are not failing; the system is.

The narrative often reads like a checklist: women are underrepresented, underpromoted, and leave the field. But what is less discussed is how much of that loss happens before a job offer is even made, and how much is shaped by the culture of belonging, or lack thereof, in the spaces they enter.

Getting Friendly With CPU Caches

Tue, 21 Oct 2025 00:00:00 +0000

Understanding how your data structures interact with hardware is one of the most powerful ways to improve application performance. This blogpost explores how CPU caches influence speed and how thoughtful struct design in Go can yield massive gains. Through a real-world case study, it shows how replacing a large embedded array with a slice improved performance by more than 40 times by reducing cache misses and improving data locality. Originally published in July 2023, its lessons remain highly relevant today for developers optimizing for memory efficiency and cache-aware programming.

Mastering Error Handling in Rust: From Panics to this error & anyhow

Tue, 21 Oct 2025 00:00:00 +0000

When: Wednesday, November 5th, 2025 | 12pm - 1pm EDT

Schedule: 1-Hour Free Webinar

Where: Online - Live Training

Panics, Results, and error crates—oh my! Confused about how to handle errors in Rust? You’re not alone. Rust gives you powerful tools to manage failure, but using them well takes practice.

In this free webinar, Nathan Stocks will demystify when to panic, how to design clear error types, and why crates like thiserror and anyhow can make your code simpler and safer.

Developers Aren’t Machines: How Smart Teams Are Transforming Outsourcing

Tue, 14 Oct 2025 00:00:00 +0000

Outsourcing in software raises strong feelings in the tech world. Some see it as a fast path to scale, tapping into global talent. Others see it as a gamble, one that often costs in quality, clarity, and culture.

If you hang out on Reddit, you’ll hear it all: engineers frustrated by unrealistic timelines, founders worried they’ll lose oversight, and companies scrambling to make promises they can’t keep.

Here at Ardan Labs, we believe the fault lines are rarely about cost. They’re about how outsourcing is executed. And more precisely, how staff augmentation, when done right, solves many of the common mistakes.

How the New $100,000 H-1B Fee Impacts U.S. Companies Hiring Developers and Engineers

Tue, 07 Oct 2025 00:00:00 +0000

The U.S. technology sector has long relied on the H-1B visa program to bring highly skilled engineers, developers, and IT specialists into the country. For many companies, this pathway has been crucial in filling roles in cloud computing, artificial intelligence, DevOps, and cybersecurity fields where domestic talent supply often struggles to meet demand.

But in September 2025, the U.S. Citizenship and Immigration Services (USCIS) announced a dramatic change that is already reshaping hiring strategies. Employers categorized as “H-1B dependent,” those with a significant percentage of H-1B workers in their workforce, will now be required to pay an additional $100,000 fee for each H-1B petition filed.

ARM - Corporate Training

Wed, 24 Sep 2025 00:00:00 +0000

Delver - Corporate Training

Wed, 24 Sep 2025 00:00:00 +0000

Eli Lilly - Corporate Training

Wed, 24 Sep 2025 00:00:00 +0000

Golab.io

Wed, 24 Sep 2025 00:00:00 +0000

Midwest AI Summit

Wed, 24 Sep 2025 00:00:00 +0000

How Important Is Knowing Kubernetes in Today’s Job Market?

Tue, 23 Sep 2025 00:00:00 +0000

From Google Project to Job-Market Staple

Containers are everywhere. Orchestration is no longer optional. But is Kubernetes truly a must-have skill or just another buzzword in the job hunt?

Just as autocomplete reshaped how we code, Kubernetes has reshaped how we deploy. Over the past decade, it has gone from a Google project to the de-facto standard for container orchestration. Recruiters drop the word in job descriptions. Bootcamps promise to teach it in weeks. And engineers wrestle with YAML while wondering if this is really the golden ticket to landing or keeping a job.

Kubernetes CPU Limits and Go

Mon, 22 Sep 2025 00:00:00 +0000

Kubernetes CPU limits can look straightforward on the surface, but their impact on application performance is anything but simple. This article unpacks how Go services interact with Kubernetes CPU throttling and why a seemingly harmless configuration such as setting a limit of 250m can dramatically constrain performance in production. First published in February 2024, its insights remain just as relevant today for anyone running Go applications in containerized environments and for developers who want to avoid costly slowdowns.

What's a Kubernetes Development Environment & why you might need one

Fri, 19 Sep 2025 00:00:00 +0000

When: Wednesday, October 1st, 2025 | 12pm - 1pm EDT

Schedule: 1-Hour Free Webinar

Where: Online - Live Training

To work with Kubernetes, we need a bunch of different tools. Almost everyone uses at least kubectl; and most teams use some combination of helm, kustomize, stern, k9s, skopeo, jq, gron, opentofu… And of course we’ll want something to edit all that YAML, as well as our scripts, HCL files, and whatnot.

New Instructor Announcement - Nathan Stocks

Wed, 03 Sep 2025 00:00:00 +0000

Meet Our New Rust Instructor — Nathan Stocks

We’re excited to welcome Nathan Stocks to the Ardan Labs team as our newest Rust Instructor, joining our Rust program led by Herbert Wolverson. Nathan brings a wealth of experience and a contagious enthusiasm for Rust that inspires developers to push their skills further.

With over 25 years of software development experience, he is recognized as one of the leading Rust instructors globally. He’s created multiple top-rated, best-selling courses that have reached more than 75,000 students worldwide and is also the creator of Rusty Engine, an open-source game engine built in Rust.

Building and Running AI, ML, and Data Science Workloads in Containers

Fri, 29 Aug 2025 00:00:00 +0000

When: Wednesday, September 10th, 2025 | 12pm - 1pm EDT

Schedule: 1-Hour Free Webinar

Where: Online - Live Training

If we want to run AI, ML, or Data Science workloads in containers (for local development or production on Kubernetes), we need to build container images. Doing that with a Dockerfile is fairly straightforward, but is it the best method?

In this talk, we’ll take a well-known speech-to-text model (Whisper) and show how to install its code and dependencies in a container. We’ll test various methods and compare the outcomes in terms of image size and build time. We’ll also show how to switch versions DRY-style (without maintaining multiple Dockerfiles!), how to leverage newer techniques like BuildKit cache mounts, and we’ll discuss other issues like the use of Alpine with Python, progressive image loading, and model loading.

Garbage Collection In Go : Part III - GC Pacing

Tue, 26 Aug 2025 00:00:00 +0000

Go’s garbage collector is designed not only to manage memory safely but also to pace itself intelligently, striking a balance between low latency and high throughput. This blogpost explores how the GC adapts its pace to workload demands, demonstrated through both sequential and concurrent program examples, and why reducing allocations per unit of work is the most effective way to lighten its load. Originally published in 2019, its core principles remain just as relevant today, offering Go developers a deeper understanding of the runtime’s adaptive behavior and confidence that the GC can find the right rhythm without extensive manual tuning.

Corporate Training - Custom Rust Training

Thu, 21 Aug 2025 00:00:00 +0000

Corporate Training - Ultimate Go

Thu, 21 Aug 2025 00:00:00 +0000

RustConf Workshop: Async Fundamentals

Thu, 21 Aug 2025 00:00:00 +0000

Ardan Labs x Deliveroo: Go, AI, & Community

Mon, 28 Jul 2025 00:00:00 +0000

When: Tuesday, August 12th, 2025 | 5:30pm - 9:00pm EST

Schedule: Networking, 2x Talks, Food & Drinks

Where: In-Person | Deliveroo HQ London

Ardan Labs partnered with Deliveroo invites you to take the conversation further at our Go, AI, Community event designed to bring developers together for an evening of insight, inspiration, and connection.

Each advisor used a different CRM or split leads across several. As a result, no single source of truth existed, and the lead data diverged quickly. Syncing systems manually or building per-CRM scripts proved brittle and time-consuming. Data quality suffered, workflows were delayed, and firms struggled to generate accurate reports or provide consistent client service.

The Goal: Deliver Real-Time Synchronization Without Disruption

The company needed a robust and extensible system that replicated lead and client records between CRMs. It had to be real-time, handle multiple data sources, and conform to the business rules of onboarding, contact assignment, and field mapping. Importantly, it had to work within the current toolset, allowing advisors to keep using their CRM of choice without a forced migration.

How Rust and .NET Came Together to Power AI at Scale

Mon, 21 Jul 2025 00:00:00 +0000

Overview

The Challenge: Bringing Rust Performance to a C# Ecosystem

LanceDb is engineered for speed, concurrency, and efficient semantic search—thanks to its Rust foundation and async-first architecture. However, its lack of a network service mode and primary focus on Python made it difficult to adopt for teams outside the Python ecosystem.

Our client’s AI infrastructure was built in C#. Replacing their stack wasn’t an option. They needed a way to access LanceDb’s performance while keeping their .NET development model intact.

Garbage Collection In Go : Part II - GC Traces

Mon, 14 Jul 2025 00:00:00 +0000

Originally published in 2019, this article is part two of a three-part series exploring Go’s garbage collector. Though the Go runtime has continued to evolve, the performance principles covered here remain highly relevant today. This installment focuses on practical techniques for analyzing and reducing garbage collection (GC) overhead in real-world Go applications. It walks through how to interpret GC traces using GODEBUG=gctrace=1 and uncover allocation hotspots with pprof, with a clear message: reducing unnecessary allocations, especially in tight loops, has a direct and measurable impact on latency and throughput. Whether you’re optimizing high-performance services or just becoming GC-aware, the insights here are as applicable now as ever.

GopherCon US Workshop: AI-Powered Systems in Go

Thu, 10 Jul 2025 00:00:00 +0000

GopherCon US Workshop: Ultimate Go

Thu, 10 Jul 2025 00:00:00 +0000

GopherCon US Workshop: Ultimate Software Design and Engineering

Thu, 10 Jul 2025 00:00:00 +0000

Garbage Collection In Go : Part I - Semantics

Tue, 10 Jun 2025 00:00:00 +0000

This article was originally published in 2018, yet its core insights into Go’s garbage collection model remain highly relevant for developers today. While some implementation details of Go’s runtime have evolved, the foundational concepts explored here—such as the semantics of the tri-color mark and sweep algorithm, Stop The World (STW) events, and GC trace interpretation—are still essential to understanding how Go manages memory. Whether you’re optimizing performance or deepening your knowledge of Go internals, this post continues to offer a clear and practical guide to working with the garbage collector, not against it.

Scaling Secure AI Infrastructure with Go and Kubernetes: A Case Study

Fri, 30 May 2025 00:00:00 +0000

Executive Summary

An AI startup in a highly regulated industry needed to scale fast—without compromising compliance or performance. With growing demands across cloud and airgapped environments, their infrastructure was struggling to keep up.

Ardan Labs partnered with their engineering team to architect and implement a secure, scalable, and compliant deployment foundation using Go and Kubernetes.

The result: a flexible infrastructure stack that enables them to confidently deploy their AI platform anywhere.

JSON - The Fine Print: Part 3 - Zero vs Missing Values

Mon, 19 May 2025 00:00:00 +0000

Introduction

In part 1 we took a higher level view on serialization in general and JSON in specific. In part 2 we looked at emitting JSON.

In part 3, we’ll look at an issue you might encounter when consuming JSON, Zero vs NULL field values. To clarify the definition of NULL, this means the absence of value.

So here is the question:

Given a field in a Go struct set to its zero value, how do you know that zero value was set by the user or it’s zero because it was never provided?

Scheduling In Go : Part III - Concurrency

Mon, 12 May 2025 00:00:00 +0000

Although originally written in 2018, the following concepts remain essential for developers working with concurrency. This blogpost focuses on concurrency, distinguishing it from parallelism by defining it as “out of order” execution. It emphasizes the importance of understanding workload types—CPU bound (e.g., summing, sorting) and IO bound (e.g., file reading)—to assess when concurrency is appropriate. Through practical examples and benchmarks, it shows that parallelism boosts performance for CPU bound tasks, while concurrency alone benefits IO bound workloads. The post underscores that identifying workload type is key to applying concurrency effectively without added complexity.

Corporate Training - Ultimate Software Design and Engineering

Fri, 09 May 2025 00:00:00 +0000

Corporate Training - Ultimate Software Design and Engineering

Fri, 09 May 2025 00:00:00 +0000

Corporate Training - Ultimate Software Design and Engineering

Fri, 09 May 2025 00:00:00 +0000

Ardan Labs Launches Go and Rust Certification Exams

Tue, 29 Apr 2025 00:00:00 +0000

Miami, FL – Ardan Labs, a premier provider of technical training, has announced the official launch of its Go and Rust Certification Exams — a new industry benchmark designed to validate real-world programming skills and help developers stand out in a competitive tech landscape.

Created in collaboration with leading technical experts, these new certifications provide a structured and respected way for developers to demonstrate expertise in two of the fastest-growing programming languages: Go and Rust. Designed with working developers in mind, the exams are proctored online and assess a developer’s ability to write clean, efficient, and production-ready code.

GopherCon EU Workshop: Practical Go for Developers

Mon, 21 Apr 2025 00:00:00 +0000

GopherCon EU Workshop: Ultimate Software Design with Kubernetes

Mon, 21 Apr 2025 00:00:00 +0000

GopherCon UK: Ultimate Go: Software Design with Kubernetes

Mon, 21 Apr 2025 00:00:00 +0000

GopherCon UK: AI-Powered Systems in Go

Mon, 21 Apr 2025 00:00:00 +0000

Scheduling In Go : Part II - Go Scheduler

Mon, 14 Apr 2025 00:00:00 +0000

This blogpost is the second installment in a three-part series exploring the mechanics and semantics of the Go scheduler. Despite being published in 2018, the content remains relevant today, as the Go scheduler’s design continues to influence the development of efficient and scalable concurrent systems. In this post, we will go into the inner workings of the Go scheduler, discussing its components, such as the Global Run Queue (GRQ) and Local Run Queue (LRQ), and its behavior, including context switching, work stealing, and the handling of synchronous and asynchronous system calls. By understanding these concepts, developers can make informed decisions about concurrency and optimization in their Go applications.

GopherCon LATAM - Workshop: Ultimate Software Design & Engineering Workshop

Tue, 01 Apr 2025 00:00:00 +0000

Scheduling In Go : Part I - OS Scheduler

Tue, 04 Mar 2025 00:00:00 +0000

Although this blogpost was originally published in 2018, the concepts and principles discussed remain crucial for building efficient and performant multithreaded applications in Go now in 2025 (Go 1.24.0). As the Go ecosystem continues to evolve, understanding how the Go scheduler interacts with the operating system scheduler is more important than ever. This three-part series provides a comprehensive overview of the mechanics and semantics behind Go’s scheduling, starting with the fundamentals of the operating system scheduler.

Intro to Rust

Tue, 11 Feb 2025 00:00:00 +0000

When: Thursday, February 27th, 2025 | 11am - 12pm EST

Schedule: 1-Hour Free Webinar

Where: Online - Live Training

Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.

This free webinar is a fast-paced introduction to Rust concepts, features, community, and language fundamentals for professional software engineers.

We’ll go over what makes Rust so awesome, preview some of its best features, and get you up and running fast.

Context Package Semantics In Go

Mon, 10 Feb 2025 00:00:00 +0000

Although first introduced in 2014, the Context package remains a crucial component of Go programming, enabling efficient management of request-scoped data, deadlines, and cancellation signals. As the Go ecosystem continues to evolve, understanding the Context package’s semantics is vital for developing reliable and maintainable software. This blogpost provides an in-depth exploration of the Context package’s semantics, highlighting best practices and common pitfalls to help developers effectively leverage this powerful tool.

JSON - The Fine Print: Part 2 - Emitting JSON

Tue, 04 Feb 2025 00:00:00 +0000

Introduction

In part 1 we took a high-level view on serialization and JSON. In this part, we’ll roll our sleeves and start working with JSON, focused on emitting JSON. You might think this is a basic topic, but there is much more to it than just calling json.Marshal.

`json.Marshal` vs `json.Encoder`

The encoding/json package has two main APIs: Marshal and NewEncoder. The Marshal function returns a []byte while the NewEncoder function will write to an io.Writer. The question is: When should you use one API over the other?

Rust, C and C++ Interoperability

Mon, 13 Jan 2025 00:00:00 +0000

Corporate Training - Ultimate Go: Software Design with K8s

Wed, 01 Jan 2025 00:00:00 +0000

Corporate Training - Ultimate Software Design and Engineering

Wed, 01 Jan 2025 00:00:00 +0000

Optimizing Databases on Kubernetes: Kubernetes Backup and Recovery with CNPG and ZFS Snapshots

Mon, 30 Dec 2024 00:00:00 +0000

Introduction:

In the final episode of the Optimizing Databases on Kubernetes series, Jérôme Petazzoni dives into advanced backup and recovery techniques for PostgreSQL, showcasing how CNPG (Cloud Native PostgreSQL) and ZFS snapshots ensure durability and fast recovery in production environments. This episode focuses on safeguarding data with comprehensive backup strategies, point-in-time recovery, and rapid database cloning, while leveraging ZFS’s efficiency for optimized storage performance.

Comprehensive Backup Strategies: Exploring backups with CNPG, including write-ahead logs (WAL) and volume snapshots.

Optimizing Databases on Kubernetes Ep.4: Kubernetes Storage: Benchmarking ZFS, Cloud Disks, and Local Paths

Mon, 23 Dec 2024 00:00:00 +0000

Introduction:

Scalable JSON Streaming with HTTP and Go - Ep.5

Mon, 25 Nov 2024 00:00:00 +0000

Introduction:

Welcome to the final episode of the JSON for Engineers series! In this concluding session, we tackle the challenges of working with large JSON datasets, exploring efficient strategies for streaming data while minimizing memory usage. These techniques enable developers to handle massive payloads without overburdening system resources, ensuring scalable and cost-effective applications.

JSON Streaming: Using JSON Lines for memory-efficient data transmission.
HTTP Chunked Encoding: Leveraging HTTP/1.1 chunked transfer encoding for streaming large datasets.
Practical Error Handling: Logging and managing errors in streaming JSON responses.

This episode starts by addressing the inefficiencies of constructing large JSON objects in memory when working with massive datasets, such as database query results. Instead of consuming significant memory to create one monolithic JSON object, the recommended approach involves using JSON Lines (NDJSON), a format where each line represents a separate JSON object. This method reduces memory requirements by transmitting data incrementally. Using Go’s encoding/json package, developers can easily encode and stream multiple JSON objects, as it automatically appends newlines between objects. On the receiving end, decoding JSON streams requires careful looping to handle incoming data dynamically while avoiding memory reuse issues, which could lead to errors or stale data.

Fearless Concurrency Ep.6: Understanding Rust ARC for Efficient Multithreading

Thu, 21 Nov 2024 00:00:00 +0000

Introduction:

Welcome to Episode 6 of the Fearless Concurrency in Rust series! In this episode, we explore the powerful concept of ARC (Atomic Reference Counting) and its critical role in managing shared resources in multithreaded Rust programs. By leveraging ARC, developers can safely share ownership of data across threads without relying on a traditional garbage collector, enabling efficient, flexible, and scalable concurrency.

What is ARC?: Understanding Atomic Reference Counting as a lightweight, manual garbage collection system.

JSON Handling and Type Management with Large Datasets in Go - Ep.4

Thu, 14 Nov 2024 00:00:00 +0000

Introduction:

Welcome to Episode 4 of the JSON for Engineers series! In this episode, we tackle the complexities of working with JSON data, especially when dealing with extensive datasets and optimizing type management. Here, Miki introduces key strategies for managing large JSON documents, avoiding unnecessary type definitions, and using Go’s encoding/json package to handle custom serialization needs effectively.

Type Pollution Avoidance: Minimizing type declarations by using selective decoding and anonymous structures.
Custom JSON Serialization: Implementing Go’s Marshaler and Unmarshaler interfaces for precise JSON formatting.
Working with Mixed-Type Arrays: Handling JSON arrays containing diverse types in Go

Miki begins by addressing the challenge of “type pollution” when working with large JSON responses. When dealing with massive JSON datasets, like data from social trading platforms, defining a separate type for every possible field can quickly lead to an unwieldy codebase. To streamline this, Miki demonstrates how to leverage anonymous structures and selective decoding in Go. By defining only the relevant fields within a function, we can avoid creating additional struct types, making the code more compact and efficient. This technique is especially useful when working with JSON data containing nested fields that may not all be necessary for your application. Through this approach, Go’s encoding/json package will simply ignore any unreferenced fields, allowing you to handle only the data that matters to your project.

Unlock LATAM’s Potential: The Ultimate Outsourcing Product Development Guide

Tue, 12 Nov 2024 00:00:00 +0000

Let’s face it, you have to be a little crazy to go on your own and start a business; long hours, limited resources, moments full of doubt, a challenge to find key talent and a mine field on a daily basis. While some of these you’re just going to have to face the music, finding talent is one of those things that you can expedite and be cost-effective in the process. Nearshoring in LATAM (Latin America) offers a strategic solution, allowing startups to access skilled talent, reduce costs, and accelerate their time to market. According to a report by Deloitte, 62% of companies believe that nearshoring allows them to focus on core business activities while benefiting from a skilled workforce (Deloitte, 2021).

Fearless Concurrency Ep.5: Ensuring Memory Safety with Mutexes and RwLocks in Rust

Fri, 08 Nov 2024 00:00:00 +0000

Introduction:

Welcome to Episode 5 of Fearless Concurrency in Rust! In this episode, Herbert Wolverson dives into advanced concurrency tools, focusing on RwLock for managing simultaneous reads and writes and the powerful concept of interior mutability. These tools provide developers with greater control over data sharing in multithreaded environments, enabling them to optimize for both performance and code simplicity in Rust.

Read-Write Locks (RwLock): Manage concurrent read and write access efficiently.

2 for 1 Ticket: Workshops with Miki Tebeka

When: Wednesday, November 13th | 12pm - 1pm EST

Schedule: 1-Hour Free Webinar

Where: Online - Live Stream

Join our FREE 1-hour live webinar to unveil and explore Ardan Labs’ innovative AI-powered Connect 4 game! Learn about its architecture and how we used Prediction Guard, MongoDB, Google Storage, and GCP to enable real-time learning and strategy.

We will cover:

The game’s architecture and AI-driven technology
How MongoDB’s Vector database and Google Storage enhance performance
Insights into cloud-native deployment on Google Cloud Platform (GCP)
Real-time Q&A session

About William Kennedy:

Managing JSON Field Tags and Value Handling in Go - Ep.3

Thu, 31 Oct 2024 00:00:00 +0000

Introduction:

Welcome to Episode 3 of JSON for Engineers! In this episode, Miki Tebeka explores the intricacies of JSON field mapping and value handling, especially when working in strongly typed languages like Go. This session covers the practical challenges of matching JSON fields to Go struct fields, distinguishing between missing and zero values, and using default values to ensure robust JSON parsing. These strategies are crucial for engineers managing flexible JSON data structures in a language with strict type requirements.

Transitioning from C and C++ to Rust: A Beginner’s Guide

Tue, 29 Oct 2024 00:00:00 +0000

Introduction:

As you embark on the journey from C or C++ to Rust, you’ll discover a world of exciting possibilities. Rust’s emphasis on safety, concurrency, and performance can significantly enhance your programming toolkit. This beginner’s guide on transitioning from C and C++ to Rust will provide a structured approach to making that transition, addressing essential concepts and practical applications. Let’s dive in!

Step 1: Understanding Rust’s Ownership Model

Concept Overview: Rust’s ownership model is its most distinctive feature. Unlike C and C++, where you have pointers and manual memory management, Rust uses a system of ownership with rules that the compiler checks at compile time.

Fearless Concurrency Ep.4: Understanding Mutexes and Thread Safety in Rust

Fri, 25 Oct 2024 00:00:00 +0000

Introduction:

Welcome to Episode 4 of Fearless Concurrency in Rust! In this episode, we explore how Rust’s ownership and borrowing rules impact multithreaded programming, focusing on the complexities of managing memory in a concurrent environment. We’ll break down how Rust’s strict borrowing and ownership model ensures safety and prevents common pitfalls like data races and undefined behavior, making it an ideal choice for building reliable multithreaded applications.

Thread Safety and Ownership: Managing variables across threads using Rust’s ownership rules.

JSON - The Fine Print: Part 1

Wed, 23 Oct 2024 00:00:00 +0000

Introduction

Everybody knows JSON, it’s a simple serialization format and the default format for REST APIs. Like many other topics, there are fine points you should know in order to work with JSON more effectively and avoid common mistakes.

In this article we’ll explore some big picture aspects and some low level details of using JSON.

Serialization

Introduction:

In Episode 1 of the Fearless Concurrency in Rust series, Herbert Wolverson introduces the foundational concept of threads in programming, tracing their origins and evolution while demonstrating how Rust handles concurrency in a safer, more manageable way. He explores the history of threads and their role in enabling programs to perform multiple tasks simultaneously, comparing Rust’s threading model with traditional approaches in C++ and Go.

Thread Basics and History: Understanding how threads function within operating systems and their development over time.

Mid-Level / Senior Backend Engineers with Rust Experience

Thu, 12 Sep 2024 00:00:00 +0000

Ardan Labs is a consulting firm that helps our clients develop software solutions and applications. We work on software projects and teams of different sizes. We are currently looking for Mid-level and Senior Backend Engineers with at least 1 year of Rust experience.

Requirements

In-depth knowledge of Rust, including idioms, guidelines, and popular libraries (e.g. Serde).
For Senior roles, expertise in Foreign Function Interface development is essential.
Experience with latency-sensitive systems, algorithms, multi-threaded code, and scalable distributed systems whilst delivering high-quality code from concept to production with a strong focus on security, reliability, performance and observability.

Ep. 6: AI Techniques: Real-Time Web Parsing and Contextual Conversations

Thu, 05 Sep 2024 00:00:00 +0000

Introduction:

In Episode 6 of the Intro to Generative A.I. series, Daniel shifts the focus from basic search techniques to more dynamic, on-the-fly AI applications. He demonstrates how to enhance AI-driven interactions by integrating real-time data retrieval and multi-turn conversations, pushing beyond static data sources to create more responsive and context-aware systems.

Implementing real-time parsing and AI search of live websites.
Enhancing chatbots with the ability to handle and respond to ongoing conversations.
Creating responsive AI systems that generate answers based on freshly retrieved data.

Ep. 5: Key Techniques for Accurate AI-Driven Information Retrieval

Thu, 22 Aug 2024 00:00:00 +0000

Introduction:

Welcome to Episode 5 of our Intro to Generative AI series! In this episode, Daniel explores practical techniques for enhancing AI models’ ability to handle large volumes of text data effectively. He addresses the challenges developers face when working with extensive content, such as entire web pages or internal documents, and provides actionable strategies to optimize the retrieval and processing of relevant information.

Context Handling: Splitting large text into manageable chunks while preserving context.
Vectorization Techniques: Converting text chunks into vector representations for semantic search.
Semantic Search: Implementing cosine similarity to retrieve relevant information efficiently.

The { Programming } Olympics

Tue, 13 Aug 2024 00:00:00 +0000

We asked members of our team…

What programming challenges should be considered an Olympic sport?

Imagine if programming had its own Olympic Games—what events would make the cut? We had a blast asking our team this fun question, and their answers were anything but ordinary. From the heart-pounding excitement of Debugging Marathons to the nail-biting drama of Cache Invalidation, the suggestions ranged from the incredibly intense to the downright quirky.

Ep. 4: Streamlining Prompt Engineering and Context Handling in Generative AI

Thu, 08 Aug 2024 00:00:00 +0000

Introduction:

Welcome to Episode 4 of our Intro to Generative AI series! In this episode, Daniel dives into the essential technique of prompt engineering, focusing on creating dynamic and interactive prompts to enhance the capabilities of AI models.

Prompt Templating: Techniques for creating and using dynamic prompt templates to enhance AI interactions.
Context Management: Strategies for integrating and switching between multiple contexts in AI applications.
Interactive Systems: Building AI systems that respond to user inputs in real-time, using terminal input loops and command-line arguments.

Categorizing Data with Large Language Models in Rust

Tue, 06 Aug 2024 00:00:00 +0000

Introduction

LibreQoS is an open source project for monitoring and providing quality-of-experience for Internet Service Providers (ISPs) and large networks. It runs as a “middle-box”, monitoring traffic that passes through it. It recently gained the ability to track individual data flows - connections between two endpoints. It’s also my favorite Open Source project, to which I contribute regularly.

Public Internet IP addresses belong to an ASN - an Autonomous System Number. An ASN is a unique number that identifies a network on the Internet. Tracking flows allows you to see which ASNs your users are connecting to, how much data is flowing, and monitor the quality of the connection.

Ep. 12: Implementing Robust Transaction Handling in Go

Mon, 05 Aug 2024 00:00:00 +0000

Introduction:

Welcome to Episode 12 of our Ultimate Software Design series! In this final installment, Bill guides us through the advanced implementation of transaction management in Go, ensuring robust and consistent data operations within applications. By focusing on middleware integration, Bill provides a comprehensive strategy for handling database transactions, committing successful operations, and rolling back when errors occur.

Learn to configure and log transaction middleware for tracking commits and rollbacks.’

Optimize Your Databases on Kubernetes With This One Weird Filesystem Trick

Mon, 05 Aug 2024 00:00:00 +0000

When: Thursday, October 17th | 12pm - 1pm EST

Schedule: 1-Hour Free Webinar

Where: Online - Zoom Meeting

Welcome to Episode 2 of our Intro to Generative AI series! In this segment, Daniel dives into the practical aspects of working with large language models (LLMs) using the Go programming language and the Prediction Guard API.

Accessing LLMs: Learn how to set up and connect to hosted models using the Go client for Prediction Guard.
Prompt Engineering: Discover how to create effective prompts and configure parameters like max tokens and temperature.
Output Variability: Understand how to manage and utilize variability in AI responses for different results.

Ardan Labs Launches Training Sessions and Expands Services in Latin America

Fri, 05 Jul 2024 00:00:00 +0000

Chicago, IL at GopherCon U.S 2024 | July 8th, 2024
Ardan Labs, a leading provider of technical training and consulting services, announces the launch of its new “Short and Advanced Go Training” sessions. These intensive 4-hour live Instructor-led workshops are tailored for experienced Go developers aiming to deepen their expertise and productivity in the Go programming language.

Short and Advanced Go Training Sessions:

The specialized training sessions will cover a range of advanced topics crucial for developers looking to enhance their skills efficiently. Key areas of focus include:

Ep. 8: Enhancing Go Application Security with JWT and OPA

Tue, 02 Jul 2024 00:00:00 +0000

Introduction:

In this segment, Bill delves into the fundamental aspects of authentication and authorization, equipping Go developers with essential knowledge and advanced tools to enhance the security of their applications. Through practical examples and detailed explanations, he unpacks the intricacies of these concepts, demonstrating their crucial role in protecting and managing access to your software systems.

Learn the distinct roles of verifying user identity and determining access levels.
Discover how to use JWTs for secure token generation, validation, and expiration management.

GopherCon U.K

Fri, 28 Jun 2024 00:00:00 +0000

GopherCon U.K - Workshop

Fri, 28 Jun 2024 00:00:00 +0000

Fearless Concurrency with Rust

Thu, 27 Jun 2024 00:00:00 +0000

When: Tuesday, August 27 · 11am - 12pm EST

Schedule: 1-Hour Free Webinar

In this installment, Bill delves into the concept of load shedding in Go, explaining its importance in managing GoRoutines and ensuring clean shutdowns.

How to manage GoRoutines using a parent-child relationship model to prevent orphan GoRoutines.
The role of load shedding in maintaining clean and orderly shutdowns, particularly using the HTTP package in Go.
Strategies for implementing GoRoutines in Go to handle concurrent tasks efficiently without risking data corruption during shutdowns.

Domain-Driven Data-Oriented Design

Mon, 03 Jun 2024 00:00:00 +0000

When: Thursday, June 27 · 11am - 12pm EST

Schedule: 1 Hours Free Webinar

Where: Online - Zoom Meeting

In this talk, Bill will discuss the principles of Domain-Driven Data Oriented Design, emphasizing its significance in software architecture for maintainability and complexity reduction. He will highlight the necessity of package design and layering, stressing the importance of clear architecture to maintain mental models of the codebase. The presentation will cover software architecture layers, data models, and package-oriented design, emphasizing domain-driven design principles and practical implementation strategies. Kennedy will also provide resources for further exploration.

Ep. 4: Mastering Kubernetes Networking: Essential Insights for Efficient Deployment

Mon, 03 Jun 2024 00:00:00 +0000

Introduction:

In this detailed discussion, Bill delves into the critical aspects of networking within Kubernetes clusters, emphasizing the necessity of properly defining services to manage internal and external communication effectively.

Learn the critical role of service definitions in Kubernetes for managing internal and external communication, ensuring your Go applications are accessible and networked correctly within the cluster.
Gain insights into configuring Kubernetes environments to handle traffic routing and service discovery, which is essential for deploying scalable and maintainable Go applications.

Domain-Driven Data Oriented Design

Wed, 29 May 2024 00:00:00 +0000

Ep. 3: Mastering Kubernetes: Setting Up Environments with Kind

Tue, 28 May 2024 00:00:00 +0000

Introduction:

In this installment of the Ultimate Software Design series, Bill guides us through setting up a Kubernetes environment using Kubernetes inside Docker (kind). He breaks down the structure of Kubernetes and offers insights into effective cluster management, ensuring a robust and efficient development environment.

Effective Cluster Management: Learn the basics of setting up and managing a Kubernetes cluster, essential for maintaining scalable and resilient Go applications.
Pod and Node Configuration: Understand how to configure pods and nodes to optimize compute resources and ensure seamless application deployment.

Go’s Trace Tooling and Concurrency

Tue, 28 May 2024 00:00:00 +0000

When: Tuesday, June 25 · 11am - 12pm EDT

Schedule: 1 Hours Free Webinar

Where: Online - Zoom Meeting

Join us for an insightful talk with Bill, where he will demonstrate the use of Go’s trace tooling to analyze a Go program’s performance. During the session, Bill will live-code a Go program, incrementally adding concurrency to observe performance variations.

Key topics include:

Utilizing Go’s trace tooling.
Understanding Goroutines and concurrency semantics.
Exploring performance metrics and optimization techniques.
Don’t miss this opportunity to enhance your Go programming skills and deepen your understanding of concurrency and performance analysis.

About William Kennedy:

3-Day Rust Training

Thu, 23 May 2024 00:00:00 +0000

3-Day Rust Training

Thu, 23 May 2024 00:00:00 +0000

The Guide to Rust Training: Essential Features and Effective Methods

Wed, 22 May 2024 00:00:00 +0000

Are you considering enrolling in a Rust training program but are unsure what to look for? Here at Ardan Labs, we have solidified what makes a good and effective training program for our students of all levels. We would like to share our industry insights with you so that you can use them to your advantage. We’ll show you the essential features every Rust training should offer, along with effective methods for mastering this programming language.

Ep. 2: Elevating Software Design in Go: Package Architecture in Modern Development

Tue, 21 May 2024 00:00:00 +0000

Introduction:

In this insightful episode, Bill dives deep into the realm of package design, shedding light on its crucial role in software development. Here are three key takeaways for Go developers:

Code Modularity: Learn how Go’s packaging system creates essential firewalls between program components, enhancing maintainability.
Focused Functionality: Discover the importance of designing packages that provide specific functionality, fostering code clarity.
Avoiding Pitfalls: Bill warns against common package design mistakes like centralized “common” packages, promoting a layered approach to maintainable code.

Ep. 6: Exploring Concurrency Pitfalls: Rust vs. C++ and Go

Mon, 20 May 2024 00:00:00 +0000

Introduction:

Join Herbert in an insightful discussion on concurrency, where he tackles the complexities of data races and how various programming languages address them:

Understand the crucial role of recognizing data races in concurrent programming and their potential impacts.
See practical examples illustrating the chaos caused by data races and learn how Rust’s compiler prevents such issues, ensuring program stability.
Explore Rust’s unique features like null pointer avoidance and reliable error handling, which enhance safety and reliability in concurrent programming tasks.

Ep. 1: Mastering Software Design: Architectural Layers and Coding Modes

Tue, 14 May 2024 00:00:00 +0000

Introduction:

Embark on a journey into the heart of software design with the Ultimate Software Design series, where Bill navigates through the intricacies of crafting robust and maintainable code in Go.

Understand the importance of prioritizing clarity over brevity in code, fostering better understanding and collaboration among Go developers.
Learn how to ensure precision in every aspect of Go development, emphasizing the necessity for precise semantic meanings in components, functions, and variables.

Ep. 5: Rust Safe at any Speed: Clearly Indicating Intent

Mon, 13 May 2024 00:00:00 +0000

Introduction:

Delving into the intricate realm of type coercion, Herbert’s insights offer invaluable guidance for Rust developers grappling with the complexities of data type conversions:

Learn the art of clearly indicating intent in code, a crucial skill for enhancing readability and maintainability in Rust projects.
Explore Rust’s meticulous approach to data mutability, leveraging features like &mut references to uphold program integrity and prevent unintended modifications.
Discover how embracing Rust’s principles of clarity and safety empowers developers to craft robust and reliable codebases, ensuring long-term success in software development endeavors.

Introduction to Generative AI with Go

Thu, 09 May 2024 00:00:00 +0000

Engineers are scrambling to figure out what they can (or should) do with this latest wave of generative AI technologies, and organizations are struggling to bring the technology into enterprise environments. Leaving this webinar, you will be equipped with the practical, hands-on expertise to start integrating generative AI in your Go applications.

Participants will learn the essential AI engineering skills of prompting, data augmentation, chaining, etc., and they will get hands-on with the latest generative AI models (Mistral, Llama 3, deepseek, etc.).

Rust at Google, Outperforming C++ and Matching Go

Thu, 09 May 2024 00:00:00 +0000

At Rust Nation UK, Lars Bergstrom – director of engineering at Google – announced the results of Google’s foray into using Rust. Rust was a great success – and the study’s key points underline Rust’s remarkable efficiency in real-world projects.

Google tasked different teams with rewriting identical services. The Go and Rust teams finished at the same speed – and the C++ team required twice as long to complete the assigned task. This represents considerable progress for Rust: Rust manages to equal the incredibly productive Go language in development speed.

Ultimate Software Design with Encore

Thu, 09 May 2024 00:00:00 +0000

When: June 10th - 14th, 2024

Schedule: 5 Days/ 5 Hours, from 11am to 4pm EST (GTM - 5)

Where: Online - Zoom Meeting

We have a special offer running for this training: GET 2 TICKETS for the PRICE OF 1

Ticket Prices:

2 FOR 1 - Pay $600 and Get 2 Seats: $300 Each
Individual Ticket Holder - $595

NOTE: If you want a 2x1 ticket but don’t have someone, send us an email, and we will pair you for the reduced price.

Ep. 4: Type Coercion in Rust: Ensuring Safety and Preventing Data Loss

Fri, 03 May 2024 00:00:00 +0000

Introduction:

Delving into the intricate realm of type coercion, Herbert’s insights offer invaluable guidance for Rust developers grappling with the complexities of data type conversions:

Gain insights into the nuances of type coercion, a fundamental aspect of programming languages, and its potential pitfalls, drawing parallels with numerical computation surprises.
Explore how Go and Rust tackle inadvertent type coercion, prioritizing program integrity by making it challenging to assign values between incompatible types.

Ep. 3: Ensuring Data Integrity in Rust: Battling Overflow and Underflow

Mon, 29 Apr 2024 00:00:00 +0000

Introduction:

Delving into the critical issues of overflow and underflow, Herbert’s insights offer valuable takeaways for Rust developers:

Rust’s static analysis capabilities enable early detection and prevention of overflow and underflow errors, allowing developers to address potential issues before they manifest in runtime environments.
Through Rust’s emphasis on safety and correctness, developers can leverage built-in mechanisms to handle overflow and underflow situations gracefully, reducing the likelihood of software crashes and unexpected behavior.

Learning Rust: A Guide for Effective Rust Training.

Fri, 26 Apr 2024 00:00:00 +0000

Effective Training Methods

Are you eager to explore Rust programming but unsure where to begin? Explore practical applications and effective training methods for your journey of learning Rust.

Rust, renowned for its safety, performance, and concurrency features, is increasingly becoming the language of choice for a wide range of applications. From system-level programming to web development and beyond, Rust offers unparalleled advantages that make it a valuable asset for developers seeking efficiency and reliability in their code.

Ep. 2: Mastering Memory Safety: Rust's Defense Against Use After Free & Use After Move

Mon, 22 Apr 2024 00:00:00 +0000

Introduction:

Exploring the critical issues of “use after free” and “use after move,” Herbert’s insights offer a valuable understanding to Rust developers:

Rust’s ownership system acts as a powerful safeguard against memory safety issues like “use after free” and “use after move,” ensuring code correctness at compile time.
By understanding the risks associated with these memory management pitfalls, Rust developers gain deeper insights into crafting more secure and reliable software, leveraging Rust’s robust memory handling mechanisms.

Learning Rust: Why it’s becoming the next trending language

Tue, 16 Apr 2024 00:00:00 +0000

Rust’s Standout Features

Are you considering learning a new programming language but unsure where to start? Look no further than Rust.

In this article, we’ll delve into the compelling reasons why Rust is worth your investment of time and effort. From its robust safety features to its exceptional performance advantages, Rust stands out as a language that empowers developers to write reliable and efficient code.

Rust’s Safety Features

First and foremost, Rust prioritizes safety. Unlike other languages where memory-related bugs can lead to catastrophic failures, Rust’s ownership system ensures memory safety without the need for a garbage collector. This feature alone makes Rust a game-changer in building secure and reliable software, reducing the risk of common bugs that plague other languages.

Ep. 1: Defending Against Memory Breaches: Exploring Rust and Go's Safety Mechanisms

Mon, 15 Apr 2024 00:00:00 +0000

Introduction:

Rust’s memory safety features are advantageous to Rust developers because they:

Ensure robust protection against buffer overflows and underflows, enhancing the reliability and security of software.
Empower developers to manage type coercion effectively, promoting code reliability and safety while facilitating safe type conversions.
Provide confidence in writing concurrent code by enforcing strict mutability rules and synchronization, thereby preventing data race issues and ensuring code correctness and efficiency.

With these memory safety features deeply integrated into Rust’s design, developers can build more reliable, secure, and efficient software, spanning various aspects of memory management, type handling, and concurrent programming paradigms.

Recruiting Tech Talent from Europe: A Strong Ally in Software Engineering

Wed, 03 Apr 2024 00:00:00 +0000

In our series of articles on recruitment, we have talked about engineers in Latin America such as Brazil, Mexico and Argentina, we also talked about the skills of engineers who come from the USA and Canada. Now it’s Europe’s turn.

The software engineering landscape is global, but Europe has emerged as a powerhouse brimming with talent and innovation. Here, we explore the rich resources Europe offers and why recruiting from this region can be highly beneficial.

95% Faster Cold Start times - For any AI/ML/Large Container

Tue, 26 Mar 2024 00:00:00 +0000

Managing software distribution at scale is a significant challenge for enterprises. Specifically, applications cannot commence until they are deployed and installed on a node. Additionally, when deploying large numbers of applications to thousands of machines locally, globally, or into HPC compute clusters, network load issues arise. For example, deploying a 1GB application to 1000 nodes requires transmitting 1TB over your network! Considering that AI/ML applications typically are larger than 4GB and can easily exceed 10GB this can be a considerable improvement.

For Loops and More in Go

Tue, 12 Mar 2024 00:00:00 +0000

Introduction

Looping seems like a basic topic: Write a for loop with a termination condition, and you’re done. However there’s a lot of ways you can write a for loop in Go. Knowing more about the different versions of for will help you choose the best option to accomplish your tasks and it will help you prevent some bugs.

Some Assembly Required

What kind of code is generated by the compiler for a for loop? To keep things simple, I will produce the assembly for an empty loop:

Advantages of Hiring Top-Tier Engineering Talent in the US and Canada

Thu, 07 Mar 2024 00:00:00 +0000

For numerous organizations, acquiring engineering talent can present significant challenges, particularly when relying on conventional sourcing methods. Scouring job websites or relying on recruiters who may not fully understand your technology stack or specific requirements can be time-consuming and often leads to suboptimal results. And even if you do come across a highly skilled engineer, their rates might be beyond your budget.

So, what sets Ardan Labs apart in addressing these challenges? Over the past 12 years, we’ve diligently cultivated a global network of talented engineers who trust us and are eager to collaborate with Ardan Labs. This extensive network allows us to offer you a diverse pool of engineers with varying levels of experience, while ensuring that they align with your budgetary constraints without compromising on quality.

PDB, Evictions, Priority Classes: Kubernetes During Rush Hour

Tue, 27 Feb 2024 00:00:00 +0000

Gophers, you filled the event in just 24hrs! Upon many requests, we have extended our number of tickets to make more available just for you. Mark your calendars & sign up while there are still seats available!

Learn how to keep your Kubernetes clusters healthy and your applications happy when their resource usage increases to critical levels.

Discover the difference between “Eviction” and “Preemption”, and how to leverage advanced concepts like “Pod Disruption Budgets”. Glean tidbits of information along the way (like the meaning of that “NOMINATED NODE” column in “kubectl get pods -o wide”).

Latin America Rising: Enhance Your Engineering Teams

Tue, 20 Feb 2024 00:00:00 +0000

You need engineering resources and you’re back to the same old ways you’ve always used to find someone. Maybe you’re using a job website or working with the same old recruiter who has never understood your tech stack or real needs. In any case, you’re spending time you don’t have and our guess is you’re not finding the actual talent you need. If you do find a solid engineer, you can’t afford them.

Rust: Safe at any Speed

Wed, 14 Feb 2024 00:00:00 +0000

GopherCon Europe 2024 Winter Edition with Ardan Labs

Fri, 02 Feb 2024 00:00:00 +0000

About GopherCon Europe

As Go’s popularity soared, GopherCon went global and GopherCon Europe (GCEU) kicked off its inaugural conference in 2018 in Reykjavik, Iceland. After that, GCEU had plans to rotate the conference every year in different countries and in 2019 the conference was held in Tenerife, Spain. However, due to the impact of COVID-19 it became important to transition the conference to an online format.

From that point on, Berlin has served as the primary base for the conference hosting both the online and in-person events. The Ardan Labs team and our very own Bill Kennedy have been supporters of GCEU since their first year. Today, GCEU has become an institution across the globe — a platform covering the entire Go spectrum for knowledge sharing, community building, and inspiration.

Recruitment: The Role of Qualified Engineers

Wed, 24 Jan 2024 00:00:00 +0000

The Role of Qualified Engineers in Shaping Future of Modern Enterprises

The engineers - the superheroes with a knack for problem-solving, creative thinking, and adaptability fuel businesses to stay ahead in the fast-paced tech landscape.

Investing in top-tier engineering talent isn’t just a strategy; it’s the secret sauce to superior product quality, heightened efficiency, and a distinct edge in the market.

Let’s explore some real-time success stories of onboarding quality engineers and finding spectacular improvements in products and services.

Staff Augmentation: Power Up Your Team for Business Success

Thu, 18 Jan 2024 00:00:00 +0000

In the fast-paced world of corporate dynamism, every company seeks the perfect moves to amp up efficiency, slash costs, and emerge as the undisputed champion. Enter the secret weapon: Staff Augmentation.

What’s the Buzz About Staff Augmentation?

It’s the art of roping in short-term warriors, the freelance superheroes, to swoop in and save the day. Think of it as assembling your Avengers team but for your workforce.

Who’s Riding the Staff Augmentation Wave?

Everyone! From the agile startups to the colossal titans like Google, Apple, Amazon, Microsoft, and IBM. They’ve all got their Avengers assembled.

Ultimate Go Tour

Mon, 15 Jan 2024 00:00:00 +0000

I have always appreciated the Go Team investing time on providing the community with the Go Tour. This website is designed to help developers get started in learning the Go programming language. The nice part of the website is that it provides an interactive environment where one can read, write, and run Go examples.

Since the first time I’ve navigated this tour, I’ve had a few problems. First, this tour is not comprehensive both in the number of examples and the content explaining the examples. Second, the content goes from beginner to expert almost instantly. I’ve always wanted to extend and improve the tour, but since this is a Google property, no one except Google employees can work on it.

A Case Study on Exploring Nearshoring Potential: A Roadmap for Global Dominance

Wed, 10 Jan 2024 00:00:00 +0000

Nearshoring success? How do we make it happen? For many companies, choosing between nearshoring and offshoring becomes tricky. Nevertheless, there are numerous success stories associated with nearshoring.

This case study explores the successes of a few industry leaders like HubSpot, Whirlpool, Duolingo, and Shopify embracing nearshoring as a transformative strategy.

Nearshoring vs. offshoring

While we talk about nearshoring, let’s explore offshoring a little as well. Offshoring entails outsourcing to a distant location, frequently in a different continent, whereas nearshoring involves outsourcing to a geographically closer location, often within the same region or continent. The decision between nearshoring and offshoring hinges on factors like cost considerations, necessary expertise, cultural compatibility, and logistical preferences.

GopherCon Europe: Winter In Athens

Wed, 06 Dec 2023 00:00:00 +0000

In Partnership with Ardan Labs

When: Feb 6th - 8th, 2024

Where: Amalia Hotel - Athens, Greece

Ticket Options

Our Self-Paid Conference Only Ticket is an in-person conference admission which does NOT include entry to workshops.

The Late Gopher Ticket & Corporate Ticket is an in-person full conference admission, including a full-day workshop.

Workshops:

February 7th 2024 at 9:00am.

Ultimate Go: Software Design with Kubernetes with Bill Kennedy
Ultimate Go with Miki Tebeka

Full-Day Workshop: Ultimate Go: Software Design with Kubernetes

Instructor: Bill Kennedy

Ultimate Go with Miki Tebeka

Tue, 05 Dec 2023 00:00:00 +0000

When: February 19th - 22nd, 2024

Schedule: 4 Days/ 4 Hours, from 8am to 12pm EST (GTM - 5)

Where: Online - Zoom Meeting

Description: The workshop is composed of four 4-hour sessions. It is aimed toward developers who have experience with Go and would like to understand how to write more efficient code and understand data semantics and concurrency patterns. We’ll cover many coding best practices that help write precise code.

Rust Nation: UK

Sun, 03 Dec 2023 00:00:00 +0000

Visualizing Map Data with Go and Leaflet JS

Thu, 23 Nov 2023 00:00:00 +0000

Introduction

This year I set a personal goal of walking for a total of 1,000 kilometers and I’m proud to say I’m close to hitting that goal. I’ve been tracking all the different routes I take in an app named Strava. One nice feature of Strava is that the app provides access to the raw data in a format called GPX. Starva can visualize the route in their app, but I wanted to try and perform my own visualization. That brought me to this blog post about using Leaflet JS to plot one of my walks on a map.

Fintech Consulting Success Story

Thu, 26 Oct 2023 00:00:00 +0000

Go-Powered Transformation: Ardan Labs Fintech Consulting Success Story

In today’s fast-paced and highly competitive financial services industry, streamlined software development can be a game-changer for companies aiming to stay ahead of the curve. With the rise of digital financial solutions, fintech companies and traditional financial institutions are increasingly relying on robust services to facilitate real-time secure transactions with seamless customer experiences.

One programming language that has been gaining significant traction in finance is Go. In this post, we will explore how Go is transforming software development for financial companies while making the process more efficient, secure, and scalable.

Implementing Enumerations In Go

Thu, 21 Sep 2023 00:00:00 +0000

Introduction

Prior to coding in Go, I was writing software in C#. In C# enumerations can be declared and the associated type can be used in functions and as fields in a struct. The compiler won’t allow a value of the enumerated type to be passed or set that doesn’t belong to the defined set. This is something that I have missed since coding in Go.

Go doesn’t have enumerations and it can be a problem at times when you want type safety for a well-defined set of values. A common way people try to have enumerations in Go really doesn’t work for me because of the way constants behave. That being said, I have developed a pattern that gets me as close as possible to what I had in C# though it’s not perfect and I will explain.

Slices Package: Contains, Delete, and Equal

Fri, 08 Sep 2023 00:00:00 +0000

Series

Here are all the posts in this series about the slices package.

Introduction

In the last post of this series, I discussed the Compare API from the slices package. In this post, I will share how the Contains, Delete, and Equal APIs work.

Contains

with Cheikh Seck

Introduction

Introduction

Go interfaces are beneficial to Go developers because they:

Allow interfaces to separate mechanism from behavior.
Increase flexibility of function parameters.
Enable mocking of function parameters.

With the addition of generics in Go 1.18, interfaces can be used to constrain the types of values a generic function will accept. To dive deeper into Go interfaces, we invite you to join Miki in his latest series dedicated to Go interfaces.

Ultimate Go: Advanced Engineering Episode 20

Fri, 27 Jan 2023 00:00:00 +0000

Introduction

In episode 19, Bill designed and implemented the data structure for an account on his blockchain. This type will have a nonce field to ensure incoming transactions are valid and performed in order. Since the database will be stored in memory and not on disk, the balances in these accounts are reconstructed with information from previous transactions and the genesis block.

In this video, Bill takes all the theories discussed in this series and applies them by launching his first blockchain node. Bill starts by defining a database type that will house all the account information on his blockchain with the initial values populated by the genesis record. Once he is finished, Bill will add a new package named state and, as the name implies, it will manage the state on his blockchain. Watch and learn how to launch a blockchain node and interface with it over HTTP.

Ultimate Go: Advanced Engineering Episode 18

Tue, 24 Jan 2023 00:00:00 +0000

Introduction

In episode 17, Bill began to design an in-memory accounting database that will store the account balances on his blockchain. To build this database, Bill will add a memory pool on each node that stores a list of public addresses with their respective balances. The balances on these addresses are reconstructed by reading the previous transaction records found on his blockchain.

In episode 9, Bill introduced the idea of the genesis record and its role in his blockchain. As a recap, the genesis record will be used to customize the settings of his blockchain. The approach Bill takes here is similar to how Ethereum operates because he is storing the record in JSON format.

In the first part of the series, Bill designed a dependency management system. The dependency manager needed to be distributed, transparent, cryptographically auditable and scalable. To meet these requirements, Bill borrowed concepts from blockchain. He did this to illustrate how the blockchain can be repurposed and used to accomplish the mundane task of dependency management. While doing so, Bill indirectly highlighted the problems a blockchain solves.

In the next segment of the series, Bill will build a semantically correct blockchain. He plans to perform this task entirely with Go and he’lll re-use the ideas described in the previous segment to further facilitate your understanding of what a blockchain is. Early on Bill admits he is no economist, and for the sake of the series he will use a hypothetical pot of gold as the starting equity for his blockchain. Follow along and learn how assets are “on-ramped” into the blockchain.

Generics vs. Interfaces

Fri, 16 Dec 2022 00:00:00 +0000

Introduction

In this video, Paulo shows the benefit of leveraging generic functions to write less code that does more. He starts by walking you through an API server he initially wrote that makes use of Go’s default HTTP handlers with a sprinkle of GORM to manage database transactions. He then refactors that code by introducing the concept of adapter functions.

In episode 1, Bill finished by describing the dependency management conundrum Go faced in its early days. Prior to the Go team providing the module system, developers were on their own to find a solution. Engineers in the Go community did propose different solutions, but there was no general consensus on which tool to use for dependency management.

In this video, Bill will begin to architect a solution to the reproducible build problem. By doing so, you will learn how blockchain hashing works and then Bill will share how centralizing this dependency database will cause other problems. One of these problems is people eventually losing trust in the database because only one stakeholder, Bill, can make changes to it without a means for users to detect the change. He closes this segment with an important question, “how do we establish trust for this CLI’s database?”

Ultimate Go: Advanced Engineering Episode 1

Mon, 12 Dec 2022 00:00:00 +0000

Introduction

In this video, Bill will introduce the concepts of what a blockchain is, the benefits of a blockchain, and the network environment it operates in. Bill describes a blockchain as a single, append-only, transparent, publicly available and cryptographically auditable database. Blockchains operate using a peer to peer (p2p) network with no single node designated as the central authority.

To better understand what all that means, Bill begins to work through a familiar problem of creating reproducible, durable, and secure builds that software teams face. It’s a real problem the Go team eventually solved with the new module system. As Bill uncovers different problems that present themselves, he introduces different blockchain concepts that can help provide a solution.

Practical Uses Of Blockchain Technology

Mon, 06 Jun 2022 00:00:00 +0000

Introduction

A blockchain is an integrated solution of different computer science problems in the form of a single, append-only, publicly available, transparent, and cryptographically auditable database that runs in a distributed and decentralized environment.

I’ve heard many times that blockchain is a technology looking for a problem to solve. I disagree with that assessment because the tech and computer science behind blockchain has practical uses in everyday engineering problems. One use of this technology that comes to mind is a dependency management verification system.

Blockchain In Go: Part IV: Fraud Detection

Thu, 05 May 2022 00:00:00 +0000

Introduction

In the first three posts, I explained there were four aspects of a blockchain that this series would explore with a backing implementation provided by the Ardan blockchain project.

The first post focused on how the Ardan blockchain provides support for digital accounts, signatures, and verification. The second post focused on transaction distribution and synchronization between different computers. The third post, focused on how the Ardan blockchain handles consensus between different computers which results in the redundant storage of the blockchain database. In this fourth post, I will focus on how the Ardan blockchain can detect forgery or changes to the underlying blocks and transaction data.

Blockchain In Go: Part III: Redundant Storage And Consensus

Tue, 19 Apr 2022 00:00:00 +0000

Introduction

In the first two posts, I explained there were four aspects of a blockchain that this series would explore with a backing implementation provided by the Ardan blockchain project.

The first post focused on how the Ardan blockchain provides support for digital accounts, signatures, and verification. The second post focused on transaction distribution and synchronization between different computers. In this third post, I will focus on how the Ardan blockchain handles consensus between different computers which results in the redundant storage of the blockchain database.

Senior Golang Engineer [Australia] | Ardan Labs

Mon, 11 Apr 2022 00:00:00 +0000

Ardan Labs is a consulting firm that helps our clients develop software solutions and applications. We work on software projects and teams of different sizes. We are currently looking for Senior Backend Engineers with at least 2 years of Go experience.

Requirements

Develop, design and review new features in high-throughput fault-tolerant microservices for an IoT-style real-time monitoring system used by thousands of payment terminals.
Collaborate with your fellow highly-skilled engineers and testers in a fast-paced, constructive nibble environment.

Mon, 01 Nov 2021 00:00:00 +0000

Introduction

You are jogging and want to show off your route to your friends. Let’s imagine the data you have for your route is a CSV file in the following format:

Listing 1: track.csv

time,lat,lng,height
2015-08-20 03:48:07.235,32.519585,35.015021,136.1999969482422
2015-08-20 03:48:24.734,32.519606,35.014954,126.5999984741211
2015-08-20 03:48:25.660,32.519612,35.014871,123.0
2015-08-20 03:48:26.819,32.519654,35.014824,120.5
2015-08-20 03:48:27.828,32.519689,35.014776,118.9000015258789
2015-08-20 03:48:29.720,32.519691,35.014704,119.9000015258789
2015-08-20 03:48:30.669,32.519734,35.014657,120.9000015258789

3+ years of experience as a Software Engineer
Strong SQL skills are required
Experience with core AWS services
Data processing tools like Stitch, DBT, Spark
Data warehousing tools like Redshift, Snowflake

Extract, Transform, and Load in Go

Mon, 13 Sep 2021 00:00:00 +0000

Introduction

You are about to visit Boston, and would like to taste some good food. You ask your friend who lives there what are good places to eat. They reply with “Everything is good, you can’t go wrong”. Which makes you think, maybe I should check where not to eat.

The data geek in you arises, and you find out that the city of Boson has a dataset of food violations. You download it and decide to have a look.

Mid-level / Senior Full Stack Engineer

Sun, 12 Sep 2021 00:00:00 +0000

Ardan Labs is a consulting firm that helps our clients develop software solutions and applications. We work on software projects and teams of different sizes. We are currently looking for a Mid-level / Senior Full Stack Engineer with experience in front-end javascript technologies.

Requirements

5+ years of experience writing software professionally.
Strong proficiency in modern Typescript / Javascript both in browser and server contexts.
Significant experience building, testing and maintaining internal and public-facing RESTful or GraphQL APIs.

DevOps Engineer

Sat, 11 Sep 2021 00:00:00 +0000

Ardan Labs is a high-performance software consulting firm that helps to develop reliable solutions and applications.

One of our clients needs help in solving interesting and large-scale problems through Devops best practices.

The Role

Automation (Orchestrate systems provisioning, processes)
Keep the lights on (day-to-day administration)
Nurture DevOps best practices

Must-Have Skills

Proven track record as a DevOps engineer in a fast-paced environment
Experience with AWS infrastructure (VPC, Security Groups, ECS, S3)

Mid-level and Senior Backend Engineers with Go experience

Fri, 10 Sep 2021 00:00:00 +0000

Requirements

Can demonstrate a good working knowledge of Go, standard library, idioms and common guidelines.
Can demonstrate the ability to collaborate with team members, flushing out designs to finalizing solutions.

Mid-level to Senior VUE.JS Engineer

Thu, 09 Sep 2021 00:00:00 +0000

Hiring 100% Remote (USA, Canada & LATAM only).

Looking for a Vue.JS (Nuxt, Vueify) engineer with 2+ years of experience to work on a challenging project. The project will require the following additional know how:

Requirements

CSS and HTML
GraphQL
Knowledge of Finance / Payment gateways might be good but not necessary.

Benefits

As a W2 employee, we offer family medical coverage, life insurance, paid vacation, and technical training.

Mid-level to Senior React Native Engineer

Wed, 08 Sep 2021 00:00:00 +0000

We have a great opportunity that will be dedicated to mobile app development in a large enterprise. We are looking for engineers with the following experience:

Requirements

2-4 years experience with React Native (required)
1-2 years GraphQL or Go (optional)

The opportunity could also include some API development with our existing engineers, so GraphQL or Go experience is optional. If you are ready to take on a new challenge, please apply!

Data Science in Go: How Much To Tip

Tue, 27 Jul 2021 00:00:00 +0000

The Question

When you work on data science problems, you always start with a question you’re trying to answer. This question will affect the data you pick, your exploration process, and how you interpret the results.

The question for this article is: How much (in percentage) should you tip your taxi driver?

To answer the question, we’ll use a portion of the NYC Taxi dataset. The data file we’ll be using is taxi-01-2020-sample.csv.bz2

Using Bitmasks In Go

Fri, 09 Apr 2021 00:00:00 +0000

Introduction

You write a server for a massively multiplayer online role-playing game (MMORPG).

In the game, players collect keys and you want to design how to store the set of keys each player has.

As an example, imagine the set of keys are copper, jade and crystal. You consider the following options for storing a player key sets:

[]string
map[string]bool

Both options will work, but did you consider a third option of using a bitmask? Using a bitmask will make storing and processing keys more efficient. Once you learn the mechanics, it will be readable and maintainable as well.

Writing Simulations Using Go

Wed, 13 Jan 2021 00:00:00 +0000

Introduction

If you can write a for-loop, you can do statistics. - Jake Vanderplas

A lot of developers shy away from problems which involve statistics or probability. Which is shameful since in today’s data-rich environment, you can gain a lot of insights from data.

In this blog post, I’ll show you how to write a simulation tool which requires no knowledge in statistics or probability. Simulations are easy to write and can be a very effective tool in research. You only need some basic programming skills and a random number generator.

Working with SQLite using Go and Python

Mon, 30 Nov 2020 00:00:00 +0000

Introduction

I prefer to use relational (SQL) databases in general since they provide several features that are very useful when working with data. SQLite is a great choice since the database is a single file, which makes it easier to share data. Even though it’s a single file, SQLite can handle up to 281 terabytes of data. SQLite also comes with a command line client called sqlite3 which is great for quick prototyping.

Python and Pandas : First Contact with Data

Mon, 02 Nov 2020 00:00:00 +0000

Introduction

Every single company I’ve worked at and talked to has the same problem without a single exception so far - poor data quality, especially tracking data. Either there’s incomplete data, missing tracking data, duplicative tracking data. - DJ Patil

I spend a lot of my time digging into data at various companies. Most of the time I’m surprised by what I see and so are the engineers and analysts that work at these companies. I’ve seen missing data, bad data, data nobody knows anything about, and many other oddities.

Generics Part 03: Struct Types and Data Semantics

Tue, 29 Sep 2020 00:00:00 +0000

Series Index

Generics Part 01: Basic Syntax
Generics Part 02: Underlying Types
Generics Part 03: Struct Types and Data Semantics

Introduction

In the previous post, I showed you how to declare a user-defined type, based on an underlying type. I did this through the progression of writing different versions of the same type using concrete types, the empty interface and then finally, generics. I also provided information on how the compiler was limited in its ability to infer the substitution for the generic type during zero value construction, but it could with initialized construction.

Python and Go : Part IV - Using Python in Memory

Tue, 15 Sep 2020 00:00:00 +0000

Series Index

Python and Go: Part I - gRPC
Python and Go: Part II - Extending Python With Go
Python and Go: Part III - Packaging Python Code
Python and Go: Part IV - Using Python in Memory

Introduction

In a previous post we used gRPC to call Python code from Go. gRPC is a great framework, but there is a performance cost to it. Every function call needs to marshal the arguments using protobuf, make a network call over HTTP/2, and then un-marshal the result using protobuf.

Caddy Server Acquired By Apilayer

Thu, 27 Aug 2020 00:00:00 +0000

Ardan Labs partnership with Caddy Server has led to the acquisition of the popular free open-source web server platform by Apilayer.

http://www.prweb.com/releases/caddy_server_acquired_by_apilayer/prweb17367009.htm

Ardan Labs announces they have secured the acquisition of Caddy Server with Apilayer. Caddy is a leading open-source extensible web server platform (written in the Go programming language) that has secured trillions of network connections around the globe. Apilayer a leading provider of cloud-based API and Software-as-a-Service (SaaS) brands.

Matt Holt started building Caddy in 2014 as a way to provide automatic and default HTTPS support for websites. Since then, Matt and the open-source community has built Caddy to provide easy, powerful, and dynamic configuration, with an extensible framework that provides high levels of resilience. “I’m pleased to look back now and see that over 300 people have collaborated to make privacy the default for the web,” Matt explained. “We’ve raised the bar for similar products and made TLS more accessible and reliable than ever. Working with Ardan Labs and now Apilayer to continue the project has been a great boon to the open-source community.”

Bill Kennedy, the Managing Partner at Ardan Labs had this to say: “I am very happy for Matt, Caddy, and the Go community. I feel blessed that Ardan Labs was able to support Matt and the project at a time when it was needed most. We at Ardan always believed in Matt and the full potential of the product, and this acquisition by Apilayer shows that supporting free open-source software can result in positive outcomes. I am most thankful to Apilayer because now Caddy’s future is secure and companies can feel confident that the product will be available for many years to come.”

“As a developer-centric company, our leading API solutions and products are used to power automation processes for hundreds of thousands of customers, ranging from small websites, SMBs, all the way to Fortune 500 enterprises. The acquisition of Caddy brings us one step closer to the developer community, and gives us the opportunity to continue building on the innovative platform that the Caddy team has developed," said Julian Zehetmayr, Founder & CEO of Apilayer.

Julian also said, “Caddy’s innovative technology and thriving open-source community were one of the most attractive elements in this partnership. Caddy will remain a separate and independent open-source brand and will continue to be directed by its original author, Matt Holt. Apilayer is committed to continued innovation, increased investment, and support for the project over the years to come.”

Generics Part 02: Underlying Types

Tue, 18 Aug 2020 00:00:00 +0000

Changes

The draft is a living document which means these posts will need to change over time. This section documents when changes have taken place to this post.

21/08/20 : Moving forward with the generics design draft

Series Index

Generics Part 01: Basic Syntax
Generics Part 02: Underlying Types
Generics Part 03: Struct Types and Data Semantics

Introduction

In the previous post, I showed you how to write a generic function in Go using the proposed syntax from the generics draft document. I did this through the progression of writing different versions of the same function using concrete types, the empty interface, and then finally generics. I also provided information on why the new syntax was needed and how the compiler can simplify the calling of generic functions by inferring type information at the call site.

Python and Go : Part III - Packaging Python Code

Tue, 11 Aug 2020 00:00:00 +0000

Series Index

Python and Go: Part I - gRPC
Python and Go: Part II - Extending Python With Go
Python and Go: Part III - Packaging Python Code
Python and Go: Part IV - Using Python in Memory

Introduction

In the previous post we compiled Go code to a shared library and used it from the Python interactive shell. In this post we’re going to finish the development process by writing a Python module that hides the low level details of working with a shared library and then package this code as a Python package.

Generics Part 01: Basic Syntax

Thu, 23 Jul 2020 00:00:00 +0000

Changes

The draft is a living document which means these posts will need to change over time. This section documents when changes have taken place to this post.

Fri, 15 May 2020 00:00:00 +0000

Introduction

In most of the reviews for this post, I was asked why choose a graph database over something else? This is a hard question to answer since my experience right now is limited on the graph database side. My guess is you’re wondering the same thing, so this is my best answer to date.

At this point in my career, I would only choose a relational database if I was writing financial software. Relational databases are very rigid so you need to really make sure you understand your data upfront. Changing the database is a big effort, but the database when designed correctly will give you very high levels of integrity and you can get good performance. These databases are hard to distribute and scale so you tend to end up with single instances that are very large and require replication for backup. The cloud providers have relational databases today that are supposed to scale, but I have no experience with them.

Getting Started With Dgraph and GraphQL+-

Thu, 16 Apr 2020 00:00:00 +0000

Introduction

I have spent a career building business applications that work with databases, starting with SQL and then moving to No-SQL. I’ve been curious how Graph databases work and how to build applications with them. It seems the sweet spot for a Graph database is when your application needs to model the interrelationships of complex and different datasets. I’ve been using No-SQL databases for these types of applications so I am super interested in knowing how Graph databases differ. After some research, I decided to learn Dgraph and their modified version of GraphQL called GraphQL+-.

Modules Part 06: Vendoring

Mon, 13 Apr 2020 00:00:00 +0000

Series Index

Why and What
Projects, Dependencies and Gopls
Minimal Version Selection
Mirrors, Checksums and Athens
Gopls Improvements
Vendoring

Introduction

It’s no secret that I am a fan of vendoring when it’s reasonable and practical to use it for your application projects. I believe vendoring gives your application projects the most durability since the project owns every line of source code it needs to build the applications. If you want a reproducible build without needing to rely on external services (like module mirrors) and being connected to the network, vendoring is the solution.

Modules Part 05: Gopls Improvements

Wed, 08 Apr 2020 00:00:00 +0000

Series Index

Why and What
Projects, Dependencies and Gopls
Minimal Version Selection
Mirrors, Checksums and Athens
Gopls Improvements
Vendoring

Prelude

Ability to work with Go code outside of the GOPATH workspace.
Ability to version a dependency and identify the most compatible version to use.
Ability to manage dependencies natively using the Go tooling.

With the release of Go 1.13, these three problems are a thing of the past. It has taken a lot of engineering effort from the Go team over the past 2 years to get everyone here. In this post, I will focus on the transition from GOPATH to modules and the problems modules are solving. Along the way, I will provide just enough of the semantics so you can have a better understanding of how modules work at a high level. Maybe more importantly, why they work the way they do.

Integration Testing in Go: Part II - Set-up and Writing Tests

Thu, 03 Oct 2019 00:00:00 +0000

Prelude

This post is the 2nd installment of a 2 part series about integration testing. You can read the first installment, which is about executing tests in a restricted environment using Docker. The example repository that this post draws it examples from can be found here.

Introduction

”More than the act of testing, the act of designing tests is one of the best bug preventers known.” - Boris Beizer

Before an integration test can be executed, the external systems that the test will be touching must be properly configured. If not, the test results will not be valid or reliable. For example, a database needs to be loaded with well defined data that is correct for the behavior being tested. Data updated during a test needs to be validated, especially if that updated data is required to be accurate for a subsequent test.

Why is DevOps Important?

Mon, 26 Aug 2019 00:00:00 +0000

1. What are four keys for great DevOps?

Collaboration: DevOps requires collaboration, both within teams and between teams. Good communication between teams implies to break down silos, which can be harder than it sounds in organizations where a leader and their team are rewarded for individual achievements, discouraging them from collaborating and sharing information with others, and often encouraging politics at the expense of the company’s success.

Blameless Culture: It also requires to internalize that we learn from our failures. Instead of promoting a culture where mistakes can seriously hinder one’s career and are therefore often shoved under the rug, we need to embrace a blameless culture where each failure becomes an opportunity to improve, and improve continuously.

Caddy Partnership With Light Code Labs

Mon, 22 Jul 2019 00:00:00 +0000

Ardan Labs (a leader in building Go enterprise software) is excited to announce that we have partnered with Light Code Labs and Matt Holt to bring Caddy into the Ardan Labs family of products and services. Caddy is a lightweight ingress service written in Go that provides automated security features and a plugin environment for extensibility. With the service being free open source (FOSS), small and large Go engineering teams finally have the ability to own and maintain their ingress and load balancing software with ease. We believe the Caddy project is as important as Docker and Kubernetes in maintaining a solid and scalable production environment.

An Open Letter To The Go Team About Try

Mon, 01 Jul 2019 00:00:00 +0000

“Once a language gets complex enough, programming in it is more like carving out a subset from an infinite sea of features, most of which we’ll never learn. Once a language feels infinite, the costs of adding further features to it are no longer apparent". - [Mark Miller] (https://medium.com/@erights/the-tragedy-of-the-common-lisp-why-large-languages-explode-4e83096239b9)

The new try proposal is an addition to the language that introduces a second mechanic to handle errors. It is being introduced in response to data gathered from the 2018 Go Survey and a review of proposals that were submitted during the Go 2 proposal process. The conclusion the Go team made from these data gathering exercises was that Go developers wanted better error handling.

Ardan Labs Partners With the Popular Open-Source Project, Caddy Web Server.

Tue, 28 May 2019 00:00:00 +0000

Ardan Labs will work with the Caddy team on bringing a highly functional enterprise version to market

Caddy Enterprise We have recently agreed on terms for a partnership which will allow us to develop Caddy side-by-side with the founding team. Caddy is an incredible open-source project that has become the premier automatic HTTPS web server on the market.

Over the past few years, the project has grown tremendously and even attracted some large enterprise users. This has prompted Caddy to expand their capabilities and partner with Ardan Labs to help develop and support Caddy Enterprise. Some of the features we are working on are:

Ardan Labs Announces New Division Focused Solely on Kubernetes

Mon, 27 May 2019 00:00:00 +0000

The new line of services will include Kubernetes Training Workshops and Kubernetes Consulting Services.

Ardan Labs has partnered with Jérôme Petazzoni to lead our Kubernetes training and consulting division. Jérôme was part of the team that built, scaled, and operated the dotCloud PAAS, before that company became Docker.

Here are some things to know about Jérôme:

He has spent seven years working at Docker, where he’s worn countless hats and ran containers in production before it was cool.
He loves to share his knowledge, which has led him to become a speaker that has now given hundreds of talks and demos on containers, Docker, and Kubernetes.
He has trained thousands of people to deploy their apps with confidence on these platforms and continues to do so as an independent consultant.

We are extremely excited about this partnership because it will allow our customers to access the same top-notch training they are used to in GO for Kubernetes.

Concurrency Trap #2: Incomplete Work

Thu, 18 Apr 2019 00:00:00 +0000

Introduction

In my first post on Goroutine Leaks, I mentioned that concurrency is a useful tool but it comes with certain traps that don’t exist in synchronous programs. To continue with this theme, I will introduce a new trap called incomplete work. Incomplete work occurs when a program terminates before outstanding Goroutines (non-main goroutines) complete. Depending on the nature of the Goroutine that is being terminated forcefully, this may be a serious problem.

Integration Testing in Go: Part I - Executing Tests with Docker

Mon, 18 Mar 2019 00:00:00 +0000

Introduction

“Testing leads to failure, and failure leads to understanding.” - Burt Rutan

Burt Rutan is an aerospace engineer who designed Voyager, the first plane to fly around the world without stopping or refueling. Although Rutan was not a software engineer, his words speak volumes to the importance of testing, even testing software. Testing software in all forms is extremely important, whether it be unit, integration, system, or acceptance testing. However, depending on the project, one form of testing can be more valuable than the others. In other words, sometimes one form of testing can lead to better understanding about the health and integrity of the software than the other forms.

Goroutine Leaks - The Abandoned Receivers

Wed, 19 Dec 2018 00:00:00 +0000

Introduction

Goroutine Leaks are a common cause of memory leaks in Go programs. In my previous post, I presented an introduction to Goroutine leaks and provided one example of a common mistake that many Go developers make. Continuing that work, this post presents another scenario on how Goroutines could be leaked.

Leak: The Abandoned Receivers

For this leak example you will see multiple Goroutines blocked waiting to receive values that will never be sent.

Goroutine Leaks - The Forgotten Sender

Mon, 12 Nov 2018 00:00:00 +0000

Introduction

Concurrent programming allows developers to solve problems using more than one path of execution and is often used in an attempt to improve performance. Concurrency doesn’t mean these multiple paths are executing in parallel; it means these paths are executing out-of-order instead of sequentially. Historically, this type of programming is facilitated using libraries that are either provided by a standard library or from 3rd party developers.

In Go, concurrency features like Goroutines and channels are built into the language and runtime to reduce or eliminate the need for libraries. This has created the illusion that writing concurrent programs in Go is easy. You must be cautious when deciding to use concurrency as it comes with some unique side effects or traps if not used correctly. These traps can create complexity and nasty bugs if you’re not careful.

Creating the Art for Gopher Kart

The Behavior Of Channels

Tue, 24 Oct 2017 00:00:00 +0000

Introduction

When I started to work with Go’s channels for the first time, I made the mistake of thinking about channels as a data structure. I saw channels as a queue that provided automatic synchronized access between goroutines. This structural understanding caused me to write a lot of bad and complicated concurrent code.

I learned over time that it’s best to forget about how channels are structured and focus on how they behave. So now when it comes to channels, I think about one thing: signaling. A channel allows one goroutine to signal another goroutine about a particular event. Signaling is at the core of everything you should be doing with channels. Thinking of channels as a signaling mechanism will allow you to write better code with well defined and more precise behavior.

Interface Semantics

Sat, 15 Jul 2017 00:00:00 +0000

Prelude

If you want to put this post in some better context, I suggest reading the following series of posts, which lay out some other fundamental and relevant design principles:

In particular, the idea of value and pointer semantics is everywhere in the Go programming language. As stated in those earlier posts, semantic consistency is critical for integrity and readability. It allows developers to maintain a strong mental model of a code base as it continues to grow. It also helps to minimize mistakes, side effects, and unexpected behavior.

For Range Semantics

Tue, 27 Jun 2017 00:00:00 +0000

Prelude

These are good posts to read first to better understand the material presented in this post:

Index of the four part series:

The idea of value and pointer semantics are everywhere in the Go programming language. As stated before in those earlier posts, semantic consistency is critical for integrity and readability. It allows developers to maintain a strong mental model of a code base as it continues to grow. It also helps to minimize mistakes, side effects, and unknown behavior.

Design Philosophy On Data And Semantics

Thu, 08 Jun 2017 00:00:00 +0000

Prelude

This is the final post in a four part series discussing the mechanics and design behind pointers, stacks, heaps, escape analysis and value/pointer semantics in Go. This post focuses on data and the design philosophies of applying value/pointer semantics in your code.

Index of the four part series:

Design Philosophies

“Value semantics keep values on the stack, which reduces pressure on the Garbage Collector (GC). However, value semantics require various copies of any given value to be stored, tracked and maintained. Pointer semantics place values on the heap, which can put pressure on the GC. However, pointer semantics are efficient because only one value needs to be stored, tracked and maintained.” - Bill Kennedy

Language Mechanics On Memory Profiling

Thu, 01 Jun 2017 00:00:00 +0000

Prelude

Develop Your Design Philosophy

Introduction

Systems cannot be developed assuming that human beings will be able to write millions of lines of code without making mistakes, and debugging alone is not an efficient way to develop reliable systems. - Al Aho (inventor of AWK)

Package Oriented Design

Fri, 24 Feb 2017 00:00:00 +0000

Updated on February 28th, 2017

Prelude

This post is part of a series of posts designed to make you think about your own design philosophy on different topics. If you haven’t read these posts yet, please do so first:

Develop Your Design Philosophy
Design Philosophy On Packaging

Introduction

Package Oriented Design allows a developer to identify where a package belongs inside a Go project and the design guidelines the package must respect. It defines what a Go project is and how a Go project is structured. Finally, it improves communication between team members and promotes clean package design and project architecture that is discussable.

Design Philosophy On Packaging

Mon, 20 Feb 2017 00:00:00 +0000

Prelude

This post is part of a series of posts designed to make you think about your own design philosophy on different topics. If you haven’t read this post yet, please do so first:

Develop Your Design Philosophy

After this post, read this next one:

Package Oriented Design

Introduction

In an interview given to Brian Kernighan by Mihai Budiu in the year 2000, Brian was asked the following question:

“Can you tell us about the worse features of C, from your point of view”?

Design Philosophy On Integrity

Copying Interface Values In Go

Thu, 05 May 2016 00:00:00 +0000

I am constantly thinking about the Go language and how things work. Lately I have been thinking how everything in Go is by value. We see this when we pass values to functions, when we iterate over slices and when we perform type assertions. In every case, copies of the values that are stored in these data structures are returned. When I first started learning Go this threw me off, but I have come to appreciate the reasonability this brings to our code.

Composition with Go

Sun, 13 Sep 2015 00:00:00 +0000

Composition goes beyond the mechanics of type embedding. It’s a paradigm we can leverage to design better APIs and to build larger programs from smaller parts. It all starts from the declaration and implementation of types that have a single purpose. Programs that are architected with composition in mind have a better chance to grow and adapt to changing needs. They are also much easier to read and reason about.

To demonstrate these concepts, we will be reviewing the following program:

Sample Code

This code sample explores the mechanics behind embedding and provides us with an opportunity to discuss how, with composition, we can design for both flexibility and readability. Every identifier that is exported from a package makes up the package’s API. This includes all the constants, variables, types, methods and functions that are exported. Comments are a frequently-overlooked aspect of every package's API, so be very clear and concise when communicating information to the user of the package.

The example is long so let’s break it down in pieces and walk our way through it.

The idea behind this program is we have a contractor that we hired to renovate our house. In particular, there are some boards in the house that have rotted and need to be yanked out, as well as new boards that need to be nailed in. The contractor will be given a supply of nails, boards to work with and tools to perform the work.

Listing 1

// Board represents a surface we can work on.

type Board struct {

    NailsNeeded int

    NailsDriven int

}

In listing 1, we have the declaration of the Board type. A Board has two fields, the number of nails the board needs and the current number of nails driven into the board. Now, let’s look at the interfaces that are declared:

Listing 2

// NailDriver represents behavior to drive nails into a board.

type NailDriver interface {

    DriveNail(nailSupply *int, b *Board)

}

25

// NailPuller represents behavior to remove nails into a board.

type NailPuller interface {

    PullNail(nailSupply *int, b *Board)

}

30

// NailDrivePuller represents behavior to drive/remove nails into a board.

type NailDrivePuller interface {

    NailDriver

    NailPuller

}

Listing 2 shows the interfaces that declare the behavior we need from the tools that may be used by the contractor. The NailDriver interface on line 22 declares behavior to drive a single nail into a board. The method is given a supply of nails and the board to drive the nail into. The NailPuller interface on line 27 declares the opposite behavior. The method is given a supply of nails and the board, but it will pull a nail from the board to place back into the supply.

These two interfaces, NailDriver and NailPuller, each implement a single, well defined behavior. This is what we want. Being able to break down behavior into individual, simple acts, lends itself to composability, flexibility and readability as you will see.

The final interface declared on line 32 is named NailDrivePuller:

Listing 3

    type NailDrivePuller interface {

        NailDriver

        NailPuller

    }

This interface is composed from both the NailDriver and NailPuller interfaces. This is a very common pattern you will find in Go, taking existing interfaces and grouping them into composed behaviors. You will see how this plays into the code later on. For now, any concrete type value that implements both the driver and puller behaviors will also implement the NailDrivePuller interface.

With the behaviors defined, it is time to declare and implement some tools:

Listing 4

// Mallet is a tool that pounds in nails.

type Mallet struct{}

41

// DriveNail pounds a nail into the specified board.

func (Mallet) DriveNail(nailSupply *int, b *Board) {

    // Take a nail out of the supply.

    *nailSupply--

46

    // Pound a nail into the board.

    b.NailsDriven++

49

    fmt.Println("Mallet: pounded nail into the board.")

}

In listing 4 on line 40, we have the declaration of a struct type named Mallet. This type is declared as an empty struct since there is no state that needs to be maintained. We only need to implement behavior.

A mallet is a tool that is used to hammer in nails. So on line 43, the NailDriver interface is implemented via the declaration of the DriveNail method. The implementation of the method is irrelevant, it subtracts a value from the nail supply and adds a value to the board.

Let’s look at the declaration and implementation of our second tool:

Listing 5

// Crowbar is a tool that removes nails.

type Crowbar struct{}

55

// PullNail yanks a nail out of the specified board.

func (Crowbar) PullNail(nailSupply *int, b *Board) {

    // Yank a nail out of the board.

    b.NailsDriven--

60

    // Put that nail back into the supply.

    *nailSupply++

63

    fmt.Println("Crowbar: yanked nail out of the board.")

}

In listing 5, we have the declaration of our second tool on line 54. This type represents a crowbar which can be used to pull nails from a board. On line 57, we have the implementation of the NailPuller interface via the declaration of the PullNail method. The implementation once again is irrelevant but you can see how a nail is subtracted from the board and added to the nail supply.

At this point in the code, we have declared tooling behavior via our interfaces and implemented that behavior via a set of struct types that represent two distinct tools our contractor can use. Now, let’s create a type that represents a contractor who performs the work using these tools:

Listing 6

// Contractor carries out the task of securing boards.

type Contractor struct{}

71

// Fasten will drive nails into a board.

func (Contractor) Fasten(d NailDriver, nailSupply *int, b *Board) {

    for b.NailsDriven < b.NailsNeeded {

        d.DriveNail(nailSupply, b)

    }

}

In listing 6, we have the declaration of the Contractor type on line 70. Again, we don’t need any state so we are using an empty struct. Then on line 73, we see the Fasten method, one of the three methods declared against the Contractor type.

The method Fasten is declared to provide a contractor the behavior to drive the number of nails that are needed into a specified board. The method requires the user to pass as the first parameter a value that implements the NailDriver interface. This value represents the tool the contractor will use to execute this behavior. Using an interface type for the this parameter allows the user of the API to later create and use different tools without the need for the API to change. The user is providing the behavior of the tooling and the Fasten method is providing the workflow for when and how the tool is used.

Notice we are using the NailDriver interface and not the NailDrivePuller interface for the parameter’s type. This is very important since the only behavior that is needed by the Fasten method is the ability to drive nails. By declaring only the behaviors we need, our code is easier to comprehend while also being easier to use due to minimal coupling. Now, let’s look at the Unfasten method:

Listing 7

// Unfasten will remove nails from a board.

func (Contractor) Unfasten(p NailPuller, nailSupply *int, b *Board) {

    for b.NailsDriven > b.NailsNeeded {

        p.PullNail(nailSupply, b)

    }

}

The Unfasten method declared in listing 7 provides the contractor the opposite behavior of the Fasten method. This method will remove as many nails as necessary from the specified board and add those back into the supply. This method accepts only tools that implement the NailPuller interface. This is exactly what we want since this is the only behavior required in the implementation of this method.

The final behavior for a contractor is a method called ProcessBoards which allows the contractor to work on a set of boards at one time:

Listing 8

// ProcessBoards works against boards.

func (c Contractor) ProcessBoards(dp NailDrivePuller, nailSupply *int, boards []Board) {

    for i := range boards {

        b := &boards[i]

90

        fmt.Printf("contractor: examining board #%d: %+v\n", i+1, b)

92

        switch {

        case b.NailsDriven < b.NailsNeeded:

            c.Fasten(dp, nailSupply, b)

96

        case b.NailsDriven > b.NailsNeeded:

            c.Unfasten(dp, nailSupply, b)

        }

    }

}

In listing 8, we see the declaration and implementation of the ProcessBoards method. This method accepts as its first parameter only values that implement both the NailDriver and NailPuller interface:

Listing 9

87 func (c Contractor) ProcessBoards(dp NailDrivePuller, nailSupply *int, boards []Board) {

Listing 9 shows how the method is declaring the exact behavior it needs via the NailDrivePuller interface type. We want our API’s to specify only the behaviors that are required or are being used. The Fasten and Unfasten methods require a value that implement a single act of behavior but ProcessBoards requires a value that implements two behaviors at the same time:

Listing 10

        switch {

        case b.NailsDriven < b.NailsNeeded:

            c.Fasten(dp, nailSupply, b)

96

        case b.NailsDriven > b.NailsNeeded:

            c.Unfasten(dp, nailSupply, b)

        }

Listing 10 shows a piece of the implementation of ProcessBoards and how the value of interface type NailDrivePuller is used in the method calls to Fasten and Unfasten. When the call to Fasten is made on line 95, the NailDrivePuller interface value is passed into the method as an interface value of type NailDriver. Let’s look at the declaration of the Fasten method once again:

Listing 11

73 func (Contractor) Fasten(d NailDriver, nailSupply *int, b *Board) {

Notice the Fasten method requires a value of interface type NailDriver and we are passing a value of interface type NailDrivePuller. This is possible because the compiler knows that any concrete type value that can be stored inside a NailDrivePuller interface value must also implement the NailDriver interface. Therefore, the compiler accepts the method call and the assignment between these two interface type values.

The same is true for the method call to Unfasten:

Listing 12

80 func (Contractor) Unfasten(p NailPuller, nailSupply *int, b *Board) 81 {

The compiler knows that any concrete type value that is stored inside an interface value of type NailDriverPuller also implements the NailPuller interface. Therefore, passing an interface value of type NailDriverPuller is assignable to an interface value of type NailPuller. Because there is a static relationship between these two interfaces, the compiler is able to forego generating a runtime type assertion and instead generate an interface conversion.

With the contractor in place, we can now declare a new type that declares a toolbox for use by a contractor:

Listing 13

// Toolbox can contains any number of tools.

type Toolbox struct {

    NailDriver

    NailPuller

109

    nails int

}

Every good contractor has a toolbox and in listing 13 we have the declaration of a toolbox. The Toolbox is a struct type that embeds on line 107 an interface value of type NailDriver and on line 108 an interface value of type NailPuller. Then on line 110, a supply of nails is declared via the nails field.

When embedding a type inside of another type, it is good to think of the new type as the outer type and the type being embedded as the inner type. This is important because then you can see the relationship that embedding a type creates.

Any type that is embeded always exists as an inner value, inside the outer type value. It never loses its identity and is always there. However, thanks to inner type promotion, everything that is declared within the inner type is promoted to the outer type. This means through a value of the outer type, we can access any field or method associated with the inner type value directly based on the rules of exporting.

Let’s look at an example of this:

Listing 14

package main

02

import "fmt"



// user defines a user in the program.

type user struct {

    name  string

    email string

}

10

// notify implements a method that can be called via

// a pointer of type user.

func (u *user) notify() {

    fmt.Printf("Sending user email To %s<%s>\n",

        u.name,

        u.email)

}

In listing 14, we have the declaration of a type named user. This type contains two string fields and a method named notify. Now, let’s embed this type into another type:

Listing 15

// admin represents an admin user with privileges.

type admin struct {

    user  // Embedded Type

    level string

}

Now we can see the relationship between the inner type and outer type in listing 15. The user type is now an inner type of admin, the outer type. This means that we can call the notify method directly from a value of type admin thanks to inner type promotion. But since the inner type value also exists, in and of itself, we can also call the notify method directly from the inner type value:

Listing 16

// main is the entry point for the application.

func main() {

    // Create an admin user.

    ad := admin{

        user: user{

            name:  "john smith",

            email: "john@yahoo.com",

        },

        level: "super",

    }

35

    // We can access the inner type's method directly.

    ad.user.notify()

38

    // The inner type's method is promoted to the outer type.

    ad.notify()

}

Listing 16 shows how a value of the outer type admin is created using a struct literal on line 28. Inside the struct literal a second struct literal is used to create and initialize a value for the inner type user. With a value of type admin in place, we can call the notify method directly from the inner type value on line 37 or through the outer type value on line 40.

One last thing, this is neither subtyping nor subclassing. Values of type admin can’t be used as values of type user. Values of type admin are only values of type admin and values of type user are only values of type user. Thanks to inner type promotion, the fields and methods of the inner type can be accessed directly through a value of the outer type.

Now back to our toolbox:

Listing 17

// Toolbox can contains any number of tools.

type Toolbox struct {

    NailDriver

    NailPuller

109

    nails int

}

We have not embedded a struct type into our Toolbox but two interface types. This means any concrete type value that implements the NailDriver interface can be assigned as the inner type value for the NailDriver embedded interface type. The same holds true for the embedded NailPuller interface type.

Once a concrete type is assigned, the Toolbox is then guaranteed to implement this behavior. Even more, since the toolbox embeds both a NailDriver and NailPuller interface type, this means a Toolbox also implements the NailDrivePuller interface as well:

Listing 18

    NailDrivePuller interface {

        NailDriver

        NailPuller

    }

In listing 18 we see the declaration of the NailDrivePuller interface again. Embedding interface types takes the idea of inner type promotion and interface compliance to the next level.

Now we are ready to put everything together by implementing the main function:

Listing 19

// main is the entry point for the application.

func main() {

    // Inventory of old boards to remove, and the new boards

    // that will replace them.

    boards := []Board{

        // Rotted boards to be removed.

        {NailsDriven: 3},

        {NailsDriven: 1},

        {NailsDriven: 6},

124

        // Fresh boards to be fastened.

        {NailsNeeded: 6},

        {NailsNeeded: 9},

        {NailsNeeded: 4},

    }

The main function starts out on line 116 in listing 19. Here we create a slice of boards and specify what each board needs in terms of nails. Next, we create our toolbox:

Listing 20

    // Fill a toolbox.

    tb := Toolbox{

        NailDriver: Mallet{},

        NailPuller: Crowbar{},

        nails:      10,

    }

137

    // Display the current state of our toolbox and boards.

    displayState(&tb, boards)

In listing 20, we create our Toolbox value and create a value of the struct type Mallet and assign it to the inner interface type NailDriver. Then we create a value of struct type Crowbar and assign it to the inner interface type NailPuller. Finally, we add 10 nails to the toolbox.

Now let’s create the contractor and process some boards:

Listing 21

    // Hire a contractor and put our contractor to work.

    var c Contractor

    c.ProcessBoards(&tb, &tb.nails, boards)

144

    // Display the new state of our toolbox and boards.

    displayState(&tb, boards)

}

On line 142 in listing 21 we create a value of type Contractor and call the ProcessBoard method against it on line 143. Let’s look at the declaration of the ProcessBoards method once more:

Listing 22

87 func (c Contractor) ProcessBoards(dp NailDrivePuller, nailSupply *int, boards []Board) {

Once again, we see how the ProcessBoard method takes as its first parameter any concrete type value that implements the NailDrivePuller interface. Thanks to the type embedding of the NailDriver and NailPuller interface types inside the Toolbox struct type, a pointer of type Toolbox implements the NailDrivePuller interface and can be passed through.

Note: The Mallet and Crowbar types implement their respective interfaces with value receivers. Based on the rules for Method Sets, both values and pointers then satisfy the interface. This is why we can create and assign values of our concrete type values to the embedded interface type values. The call to ProcessBoards in listing 21 on line 143 is using the address of the Toolbox value because we want to share a single Toolbox. However, a value of type Toolbox also satisfies the interface. If the concrete types used pointer receivers to implement the interfaces, then only pointers would have satisfied the interface.

To be complete, here is the implementation of the displayState function:

Listing 23

// displayState provide information about all the boards.

func displayState(tb *Toolbox, boards []Board) {

    fmt.Printf("Box: %#v\n", tb)

    fmt.Println("Boards:")

153

    for _, b := range boards {

        fmt.Printf("\t%+v\n", b)

    }

157

    fmt.Println()

}

In conclusion, this example has tried to show how composition can be applied and the things to think about when writing your code. How you declare your types and how they can work together is very important. Here is a summary of the things we touch upon in the post:

* Declare the set of behaviors as discrete interface types first. Then think about how they can be composed into a larger set of behaviors.

* Make sure each function or method is very specific about the interface types they accept. Only accept interface types for the behavior you are using in that function or method. This will help dictate the larger interface types that are required.

* Think about embedding as an inner and outer type relationship. Remember that through inner type promotion, everything that is declared in the inner type is promoted to the outer type. However, the inner type value exists in and of itself as is always accessible based on the rules for exporting.

* Type embedding is not subtyping nor subclassing. Concrete type values represent a single type and can’t be assigned based on any embedded relationship.

* The compiler can arrange interface conversions between related interface values. Interface conversion, at compile time, doesn't care about concrete types - it knows what to do merely based on the interface types themselves, not the implementing concrete values they could contain.

I would like to thank Kevin Gillette for working with me on the code and post and Bill Hathaway for helping with reviews and edits to the post.

Object Oriented Programming Mechanics

Sun, 15 Mar 2015 00:00:00 +0000

Go is an object oriented programming language. It may not have inheritance, but in this 20 minute video from the Bangalore meetup, I will show you how object oriented programming practices and techniques can be applied to your Go programs. From an object oriented standpoint, Go does provides the ability to add behavior to your types via methods, allows you to implement polymorphic behavior via interfaces and gives you a way to extend the state and behavior of any existing type via type embedding. Go also provides a form of encapsulation that allows your types, including their fields and methods, to be visible or invisible. Everything you need to write object oriented programs is available in Go.

Example 1 - Methods:
http://play.golang.org/p/hfRkC6nKag

Example 2 - Interfaces:
http://play.golang.org/p/F1UyKlTh3k

Example 3 - Extending Types:
http://play.golang.org/p/JJ811lBwoz

Example 4 - Overriding Inner Types:
http://play.golang.org/p/-xQFBv9_82

Scheduler Tracing In Go

Mon, 23 Feb 2015 00:00:00 +0000

Introduction
One of the things I love about Go is the profiling and debug information you can generate. There is a special environmental variable named GODEBUG that will emit debugging information about the runtime as your program executes. You can request summary and detailed information for both the garbage collector and the scheduler. What’s great is you don’t need to build your program with any special switches for it to work.

Stack Traces In Go

Sun, 11 Jan 2015 00:00:00 +0000

Introduction
Having some basic skills in debugging Go programs can save any programmer a good amount of time trying to identify problems. I believe in logging as much information as you can, but sometimes a panic occurs and what you logged is not enough. Understanding the information in a stack trace can sometimes mean the difference between finding the bug now or needing to add more logging and waiting for it to happen again.

Using Pointers In Go

Tue, 16 Dec 2014 00:00:00 +0000

Introduction
I am asked quite a bit about when and when not to use pointers in Go. The problem most people have, is that they try to make this decision based on what they think the performance tradeoff will be. Hence the problem, don’t make coding decisions based on unfounded thoughts you may have about performance. Make coding decisions based on the code being idiomatic, simple, readable and reasonable.

Error Handling In Go, Part II

Mon, 10 Nov 2014 00:00:00 +0000

Introduction
In part I of this post, we learned about the error interface and how the standard library provides support for creating error interface values via the errors package. We also learned how to work with error interface values and use them to identify when an error has occured. Finally, we saw how some packages in the standard library export error interface variables to help us identify specific errors.

Error Handling In Go, Part I

Mon, 13 Oct 2014 00:00:00 +0000

Introduction
It is idiomatic in Go to use the error interface type as the return type for any error that is going to be returned from a function or method. This interface is used by all the functions and methods in the standard library that return errors. For example, here is the declaration for the Get method from the http package:

Listing 1.1

Go Compiler nil Pointer Checks

Mon, 01 Sep 2014 00:00:00 +0000

Introduction
I was thinking about how the compiler looks to protect the code we write when it can. Invalid memory access checks are one type of safety check the compiler adds to our code. We might think that this "extra code" is hurting our performance and maybe over billions of iterative operations it is. However, these checks can prevent our code from causing damage to the systems we are running on. The compiler is essentially reporting and finding bugs, making the code we write safer to run.

Even with this in mind, Go wants to be fast and if the hardware can catch a problem, it will let it. One of these cases is with the detection of invalid memory access. There are times when the compiler will add a nil pointer check into our code and times when it doesn’t. In this post we will explore one case where the compiler leaves it up to the hardware to detect invalid memory access and one case where the compiler adds a nil pointer check.

Hardware Only Checks
When the compiler can depend on the hardware to check for and report invalid memory access, the compiler can produce less code which helps with performance. If our code attempts to read or write to address 0x0, the hardware will throw an exception that will be caught by the Go runtime and reported back up to our program in the form of a panic. If the panic is not recovered, a stack trace is produced.

Here is an example that attempts to write to memory address 0x0 and the corresponding panic/stack trace:

ALS Ice Bucket Challenge - Go Style

Sat, 30 Aug 2014 00:00:00 +0000

If you have not heard about the ALS Ice Bucket Challenge I would be surprised. It’s everywhere nowadays, being done by celebrities alike. After being challenged by my children there was only one person, or should I say Gopher, I knew I needed to nominate. The Gopher took to the challenge and here it is:

I would like to thank my daughter for spending time today filming and editing the video. She is a senior member of the award winning [ West Broward High School TV Production Club ](/broken-link). They do amazing work so please give them a shot out on twitter at @wbtvclub if you enjoyed the video.

The Gopher character is based on the Go mascot designed by Renée French and copyrighted under the Creative Commons Attribution 3.0 license.

Ice Cream Makers and Data Races Part II

Fri, 27 Jun 2014 00:00:00 +0000

Introduction
Dave Cheney published a post called Ice Cream Makers and Data Races. The post showed an example of an interesting data race that can occur when using an interface typed variable to make a method call. If you have not read the post yet please do. Once you read the post you will discover that the problem lies with the fact that an interface value is implemented internally using a two word header and that the Go memory model states only writes to a single word are atomic.

The program in the post shows a race condition that allows two goroutines to perform a read and write operation at the same time against an interface value. Not synchronizing this read and write allows the read to observe a partial write to the interface value. This allows the method implementation for the Ben type to operate against a value of the Jerry type and visa versa.

In Dave’s example, the layout of the Ben and Jerry structs were identical in memory, so they were in some sense compatible. Dave suggested the chaos that would occur if they had different memory representations. This is because each implementation of the Hello method assumes the code is operating against values of the receiver type. When this bug surfaces, this is no longer the case. To give you a visual view of this suggested chaos, I am going to change the declaration of the Jerry type in two different ways. Both changes will give you some better insight into the interworking of interface types and memory.

First Code Change
Let’s review the code and see the first set of changes. My changes to the original code are in bold:

Actionable Data With MongoDB and Go

Wed, 25 Jun 2014 00:00:00 +0000

Introduction
If you are building any kind of application for a consumer based product, it is common to have large amounts of application data being generated about your users. Running reports is a traditional use of this data, but what if you could make this data actionable? What if you could adapt the user experience by aggregating and testing this data against rules that could dictate actions or special messaging?

In this post I am going to show you a way that could be used to make your application data "actionable". We are going to review a sample program written in Go that leverages MongoDB’s aggregation pipeline to execute rules that aggregate and test data. The rule the program defines and executes determines what messaging we want to show the user based on their current set of financial transactions in the system.

The Data
We have two collections of data we will be using which are being hosted in a public MongoLab database. The first collection contains credit card transactions for users. This data represent application generated data:

Pitfalls With Closures In Go

Fri, 20 Jun 2014 00:00:00 +0000

Introduction
Closures in Go are a very powerful construct but they can also be the cause of bugs if you don’t understand how they work. In this post I am going to pull a small piece of code from Chapter 2 from the Go In Action book that discusses a pitfall you can run into when using closures. The full code example can be found in the Github repository for the book. Chapter 2 discusses this code example in full detail.

The Closure Pitfall
First let’s look at the piece of code:

Methods, Interfaces and Embedded Types in Go

Sat, 03 May 2014 00:00:00 +0000

Introduction
My business partner Ed asked me what would happen if a struct and an embedded field both implemented the same interface. We asked ourselves two questions:

Would the compiler throw an error because we now had two implementations of the interface?

If the compiler accepted the type declaration, how does the compiler determine which implementation to use for interface calls?

We hacked out some code to answer the questions and then I dug into the specification. What we found was really interesting and we believed worth sharing with others who are also learning the language. Once we learned the mechanics behind methods, interfaces, and embedded types, the answers seemed obvious. To get started, let’s talk about what methods are in Go.

Methods
Go has both functions and methods. In Go, a method is a function that is declared with a receiver. A receiver is a value or a pointer of a named or struct type. All the methods for a given type belong to the type’s method set.

Let’s declare a struct type and a method for that type:

type User struct {

    Name string

    Email string

}

func (u User) Notify() error

First we declare a struct type named User and then we declare a method named Notify with a receiver that accepts a value of type User. To call the Notify method we need a value or pointer of type User:

// Value of type User can be used to call the method

// with a value receiver.

bill := User{"Bill", "bill@email.com"}

bill.Notify()

// Pointer of type User can also be used to call a method

// with a value receiver.

jill := &User{"Jill", "jill@email.com"}

jill.Notify()

In the case where we are using a pointer, Go adjusts and dereferences the pointer so the call can be made. Be aware that when the receiver is not a pointer, the method is operating against a copy of the receiver value.

We can change the Notify method to use a pointer for the receiver:

func (u *User) Notify() error

Once again, we can call the Notify method like we did before:

// Value of type User can be used to call the method

// with a pointer receiver. 

bill := User{"Bill", "bill@email.com"}

bill.Notify()

// Pointer of type User can be used to call the method

// with a pointer receiver.

jill := &User{"Jill", "jill@email.com"}

jill.Notify()

If you are unsure about when to use a value or a pointer for the receiver, the Go wiki has a great set of rules that you can follow. The Go wiki also contains a paragraph about the conventions the community follows for naming receivers.

Interfaces
Interfaces in Go are special and provide an incredible amount of flexibility and abstraction for our programs. They are a way of specifying that values and pointers of a particular type can behave in a specific way. From a language perspective, an interface is a type that specifies a method set and all the methods for an interface type are considered to be the interface.

Let’s declare an interface in Go:

type Notifier interface {

    Notify() error

}

Here we declare an interface called Notifier with a single method called Notify. It is a convention in Go to name interfaces with an -er suffix when the interface contains only one method. This is not a hard rule but something we should honor, especially when the interface and method name have the same signature and meaning.

We can specify as many methods as we want for our interfaces. In the standard library you will be hard pressed to find many interfaces with more than two methods.

Implementing Interfaces
Go is unique when it comes to how we implement the interfaces we want our types to support. Go does not require us to explicitly state that our types implement an interface. If every method that belongs to an interface’s method set is implemented by our type, then our type is said to implement the interface.

Let’s continue with our example by creating a function that accepts any value or pointer of a type that implements the Notifier interface:

func SendNotification(notify Notifier) error {

    return notify.Notify()

}

The SendNotification function calls the Notify method that is implemented by the value or pointer that is passed into the function. This function can be used to execute the specific behavior for any value or pointer of a given type that implements the interface.

Let’s implement the Notify method for our User type and call the SendNotification function passing a value of type User:

func (u *User) Notify() error {

    log.Printf("User: Sending User Email To %s<%s>\n",

        u.Name,

        u.Email)

    return nil

}

func main() {

    user := User{

        Name:  "janet jones",

        Email: "janet@email.com",

    }

    SendNotification(user)

}

// Output:

cannot use user (type User) as type Notifier in function argument:

      User does not implement Notifier (Notify method has pointer receiver)

http://play.golang.org/p/VieiPRDGVu

Why does the compiler not consider our value to be of a type that implements the interface? The rules for determining interface compliance are based on the receiver for those methods and how the interface call is being made. Here are the rules in the spec for how the compiler determines if the value or pointer for our type implements the interface:

The method set of the corresponding pointer type *T is the set of all methods with receiver *T or T

This rule is stating that if the interface variable we are using to call a particular interface method contains a pointer, then methods with receivers based on both values and pointers will satisfy the interface. This rule does not apply for our example because we are passing a value to the SendNotification function.

The method set of any other type T consists of all methods with receiver type T.

This rule is stating that if the interface variable we are using to call a particular interface method contains a value, then only methods with receivers based on values will satisfy the interface. This rule does not apply for our example because the receiver for the Notify method accepts a pointer.

Since those are the only two rules in the spec for interface compliance, I have derived this rule that applies to our example:

The method set of the corresponding type T does not consists of any methods with receiver type *T.

This is our case and why we are receiving the compiler error. The Notify method is using a pointer for the receiver and we are using a value to make the interface method call. To fix this we just need to pass the address of the User value to the SendNotification function:

func main() {

    user := &User{

        Name:  "janet jones",

        Email: "janet@email.com",

    }

    SendNotification(user)

}

// Output:

User: Sending User Email To janet jones<janet@email.com>

http://play.golang.org/p/3NNiS4dMrK

Embedded Types
Struct types have the ability to contain anonymous or embedded fields. This is also called embedding a type. When we embed a type into a struct, the name of the type acts as the field name for what is then an embedded field.

Let’s declare a new type and embed our User type into it:

type Admin struct {

    User

    Level string

}

We have declared a new type called Admin and embedded the User type within the struct declaration. This is not inheritance but composition. There is no relationship between the User and the Admin type.

Let’s change main to create a value of the Admin type and pass the address of this value to the SendNotification function:

func main() {

    admin := &Admin{

        User: User{

            Name:  "john smith",

            Email: "john@email.com",

        },

        Level: "super",

    }

    SendNotification(admin)

}

// Output

User: Sending User Email To john smith<john@email.com>

http://play.golang.org/p/2jZMCGEfxW

Sure enough, we are able to call the SendNotification function with a pointer of type Admin. Thanks to composition, the Admin type now implements the interface through the promotion of the methods from the embedded User type.

If the Admin type now contains the fields and methods of the User type, then where are they in relationship to the struct?

"When we embed a type, the methods of that type become methods of the outer type, but when they are invoked, the receiver of the method is the inner type, not the outer one." - Effective Go

Since the name of the embedded type acts as the field name and the embedded type exists as an inner type, we can then make the following method call:

admin.User.Notify()

// Output

User: Sending User Email To john smith<john@email.com>

http://play.golang.org/p/_huNeKVmXS

Here we are accessing the field and method set of the inner type through the use of the type’s name. However, these fields and methods are also promoted to the outer type:

admin.Notify()

// Output

User: Sending User Email To john smith<john@email.com>

http://play.golang.org/p/v4ro-KHiKJ

So calling the Notify method using the outer type, calls the implementation of the inner type’s method.

These are the rules for inner type method set promotion in Go:

Given a struct type S and a type named T, promoted methods are included in the method set of the struct as follows:

If S contains an anonymous field T, the method sets of S and *S both include promoted methods with receiver T.

This rule is stating that when we embed a type, methods for the embedded type with receivers that use a value are promoted and available for calling by values and pointers of the outer type.

The method set of *S also includes promoted methods with receiver *T.

This rule is stating that when we embed a type, methods for the embedded type with receivers that use a pointer are promoted and available for calling by pointers of the outer type.

If S contains an anonymous field *T, the method sets of S and *S both include promoted methods with receiver T or *T.

This rule is stating that when we embed a pointer of the type, methods for the embedded type with receivers that use both values and pointers are promoted and available for calling by values and pointers of the outer type.

Since those are the only three rules in the spec for method promotion, I have derived this rule for one other case:

If S contains an anonymous field T, the method set of S does not include promoted methods with receiver *T.

This rule is stating that when we embed a type, methods for the embedded type with receivers that use a pointer are not promoted for calling by values of the outer type. This is consistent with the rules for interface compliance that we stated above.

Answering The Questions
Now we can finalize the sample program that will provide the answers for the two questions we asked in the beginning of the post. Let’s implement the Notifier interface for the Admin type:

func (a *Admin) Notify() error {

    log.Printf("Admin: Sending Admin Email To %s<%s>\n",

        a.Name,

        a.Email)

    return nil

}

The implementation of the interface by the Admin type displays a message on behalf of an admin. This will help us determine which implementation gets called when we use a pointer of the Admin type to make the function call to SendNotification.

Now let’s create a value of the Admin type and pass the address of that value to the SendNotification function and see what happens:

func main() {

    admin := &Admin{

        User: User{

            Name:  "john smith",

            Email: "john@email.com",

        },

        Level: "super",

    }

    SendNotification(admin)

}

// Output

Admin: Sending Admin Email To john smith<john@email.com>

http://play.golang.org/p/NkDioPJs04

As expected, the Admin type’s implementation of the interface is called by the SendNotification function. So now what happens when we call the Notify method using the outer type:

admin.Notify()

// Output

Admin: Sending Admin Email To john smith<john@email.com>

http://play.golang.org/p/RG50rxC0d7

We get the output for the Admin type’s implementation. The User type’s implementation is no longer promoted to the outer type:

So now we have the knowledge we need to answer the questions:

Would the compiler throw an error because we now had two implementations of the interface?

No, because when we use an embedded type, the unqualified type’s name acts as the field name. This has the effect of fields and methods of the embedded type having a unique name as an inner type of the struct. So we can have an inner and outer implementation of the same interface with each implementation being unique and accessible.

If the compiler accepted the type declaration, how does the compiler determine which implementation to use for interface calls?

If the outer type contains an implementation that satisfies the interface, it will be used. Otherwise, thanks to method promotion, any inner type that implements the interface can be used through the outer type.

Conclusion
The way methods, interfaces, and embedded types work together is something that makes Go very unique. These features help us create powerful constructs to achieve the same ends as object oriented code without all the complexity. With the language features that we talked about in this post, we can build abstracted and scalable frameworks with a minimal amount of code and confusion.

The more I learn about the details of the language and the compiler, the more I come to appreciate how orthogonal the language is. Small features that work together and allow us to be creative and use the language in ways not even the language designers thought or dreamed about. I recommend to take the time to learn the language features so you can do more with less and be both creative and productive at the same time.

Introduction To Numeric Constants In Go

Sun, 06 Apr 2014 00:00:00 +0000

Introduction
One of the more unique features of Go is how the language implements constants. The rules for constants in the language specification are unique to Go. They provide the flexibility Go needs at the compiler level to make the code we write readable and intuitive while still maintaining a type safe language.

This post will attempt to build a foundation for what numeric constants are, how they behave in their simplest form and how best to talk about them. There are a lot of little nuances, terms and concepts that can trip us up. Because of this, the post is going to take things slowly.

So if you are ready to peek under the covers just a bit, roll up your sleeve and let’s get started:

Untyped and Typed Numeric Constants
Constants can be declared with or without a type in Go. When we declare literal values in our code, we are actually declaring constants that are both untyped and unnamed.

The following examples show typed and untyped numeric constants that are both named and unnamed:

Exported/Unexported Identifiers In Go

Sat, 15 Mar 2014 00:00:00 +0000

Introduction
One of the first things I learned about in Go was using an uppercase or lowercase letter as the first letter when naming a type, variable or function. It was explained that when the first letter was capitalized, the identifier was public to any piece of code that wanted to use it. When the first letter was lowercase, the identifier was private and could only be accessed within the package it was declared.

Web Form Validation And Localization In Go

Fri, 07 Mar 2014 00:00:00 +0000

Introduction
As I improve my knowledge and framework for a Go based web service I am building, I continue to go back and enhance my Beego Sample App. Something I just added recently was providing localized messages for validation errors. I was fortunate to find Nick Snyder's go-i18n package. Nick's package made it easy to support multiple languages for the Go web service I am writing.

Abstracting go-i18n
The go-i18n package is simple to use and you can use it to read files or strings that contain all the messages you want to localize. It has some nice features including variable substitution and support for handling plurals for each individual locale. Nick has documentation for his package, so I am going to show you how I abstracted and integrated go-i18n into the Beego sample app.

I decided I didn't want to use files to store the messages, but create raw string literal variables. The less I had to worry about managing external resources the better. With that being said, I built a simple package that abstracted the support I needed. Luckily go-i18n supports passing in a string that can contain the JSON document with the message data:

I am just using simple messages right now, but as you can see, the variable En_US is defined and assigned a JSON document with the messages I need localized. The go-i18n package also lets you define messages like this:

In this sample, the translation has one message for the singular case and one for the plural case. There is also support for using variable substitution thanks to template support.

Here is the localize package that provides support for the web service:

The Init function creates the default locale for the application. Currently the Beego Sample App only supports English for the United States. Eventually, we can add cases for the other locales. Obviously this can all be done through configuration in the future.

The Init function uses the LoadJSON function to load the go-i18n datastore with the internal messages for the default locale. Later on we can use the LoadJSON function again to load more JSON documents for the same or different locales.

The Init function also uses the NewTranslation function to obtain a new i18n.TranslateFunc object for the default locale. This object is used to retrieve messages from the go-i18n datastore. If we have a scenario where the default locale is not valid, we can use the NewTranslation function at any time to obtain an object for the locale we need.

Beego Integration
To see how I integrated the go-i18n package into the sample app, we need to look at the controller:

As discussed in my previous post about the Beego Sample App, we define a struct with tags that are used by the Beego validation module. I added support for defining the error to be returned when validation fails, by providing a new tag called error. In this case the error tag contains the id of the localized message we want to return. The ParseAndValidate function will handle the rest:

When the Beego validation module finds a problem, then the real work begins. The function uses reflection to find the error tag on any of the fields in the param struct. If an error tag is found, the id of the localized message is stored along with the field name. Then the function ranges over all the errors that the Beego validation module found and if an error tag existed for that field, we use the id to retrieve the localized message.

Testing
If we run the run_endpoint_test.sh shell script, which can be found in the zscripts folder, we can see the localized message returned in the last test:

The last test is designed to validate the localized message is returned.

Conclusion
The Beego framework has been great for developing my Go web service. It has the right amount of framework and modules, like the validation module, when you need it. The ability to bring in a package like go-i18n and integrate it so easily is another big win for Beego.

If you are in need for localizing your Go application, take a look at go-i18n and see if it can work for you.

Running MongoDB Queries Concurrently With Go

Tue, 25 Feb 2014 00:00:00 +0000

If you are attending GopherCon 2014 or plan to watch the videos once they are released, this article will prepare you for the talk by Gustavo Niemeyer and Steve Francia. It provides a beginners view for using the Go mgo driver against a MongoDB database.

Introduction
MongoDB supports many different programming languages thanks to a great set of drivers. One such driver is the MongoDB Go driver which is called mgo. This driver has been externally developed by Gustavo Niemeyer from Canonical, and eventually Steve Francia, the head of the drivers team at MongoDB Inc, took notice and offered support. Both Gustavo and Steve will be talking at GopherCon 2014 in April about "Painless Data Storage With MongoDB and Go ". The talk centers around the mgo driver and how MongoDB and Go really work well together to build highly scalable and concurrent software.

MongoDB and Go let us build scalable software on many different operating systems and architectures, without the need to install any frameworks or runtime environments. Go programs are native binaries and the Go tooling is constantly improving to create binaries that run as fast as equivalent C programs. That wouldn’t mean anything if writing code in Go was complicated and as tedious as writing programs in C. This is where Go really shines because once you get up to speed, writing programs in Go is fast and fun.

In this post I am going to show you how to write a Go program using the mgo driver to connect and run queries concurrently against a MongoDB database. I will break down the sample code and explain a few things that seem to be always be a bit confusing to those new to MongoDB and Go.

Sample Program
The sample program connects to a public MongoDB database I have hosted with MongoLab. If you have Go and Bazaar installed on your machine, you can run the program. The program launches ten goroutines that individually query all the records from the buoy_stations collection inside the goinggo database. The records are unmarshaled into native Go types and each goroutine logs the number of documents returned:

The Nature Of Channels In Go

Mon, 17 Feb 2014 00:00:00 +0000

Introduction
In my last post called Concurrency, Goroutines and GOMAXPROCS, I set the stage for talking about channels. We discussed what concurrency was and how goroutines played a role. With that foundation in hand, we can now understand the nature of channels and how they can be used to synchronize goroutines to share resources in a safe, less error prone and fun way.

What Are Channels
Channels are type safe message queues that have the intelligence to control the behavior of any goroutine attempting to receive or send on it. A channel acts as a conduit between two goroutines and will synchronize the exchange of any resource that is passed through it. It is the channel's ability to control the goroutines interaction that creates the synchronization mechanism. When a channel is created with no capacity, it is called an unbuffered channel. In turn, a channel created with capacity is called a buffered channel.

To understand what the synchronization behavior will be for any goroutine interacting with a channel, we need to know the type and state of the channel. The scenarios are a bit different depending on whether we are using an unbuffered or buffered channel, so let's talk about each one independently.

Unbuffered Channels
Unbuffered channels have no capacity and therefore require both goroutines to be ready to make any exchange. When a goroutine attempts to send a resource to an unbuffered channel and there is no goroutine waiting to receive the resource, the channel will lock the sending goroutine and make it wait. When a goroutine attempts to receive from an unbuffered channel, and there is no goroutine waiting to send a resource, the channel will lock the receiving goroutine and make it wait.

Concurrency, Goroutines and GOMAXPROCS

Wed, 29 Jan 2014 00:00:00 +0000

Introduction
When new people join the Go-Miami group they always write that they want to learn more about Go’s concurrency model. Concurrency seems to be the big buzz word around the language. It was for me when I first started hearing about Go. It was Rob Pike’s Go Concurrency Patterns video that finally convinced me I needed to learn this language.

To understand how Go makes writing concurrent programs easier and less prone to errors, we first need to understand what a concurrent program is and the problems that result from such programs. I will not be talking about CSP (Communicating Sequential Processes) in this post, which is the basis for Go’s implementation of channels. This post will focus on what a concurrent program is, the role that goroutines play and how the GOMAXPROCS environment variable and runtime function affects the behavior of the Go runtime and the programs we write.

Processes and Threads
When we run an application, like the browser I am using to write this post, a process is created by the operating system for the application. The job of the process is to act like a container for all the resources the application uses and maintains as it runs. These resources include things like a memory address space, handles to files, devices and threads.

A thread is a path of execution that is scheduled by the operating system to execute the code we write in our functions against a processor. A process starts out with one thread, the main thread, and when that thread terminates the process terminates. This is because the main thread is the origin for the application. The main thread can then in turn launch more threads and those threads can launch even more threads.

The operating system schedules a thread to run on an available processor regardless of which process the thread belongs to. Each operating system has its own algorithms that make these decisions and it is best for us to write concurrent programs that are not specific to one algorithm or the other. Plus these algorithms change with every new release of an operating system, so it is dangerous game to play.

Goroutines and Parallelism
Any function or method in Go can be created as a goroutine. We can consider that the main function is executing as a goroutine, however the Go runtime does not start that goroutine. Goroutines are considered to be lightweight because they use little memory and resources plus their initial stack size is small. Prior to version 1.2 the stack size started at 4K and now as of version 1.4 it starts at 8K. The stack has the ability to grow as needed.

The operating system schedules threads to run against available processors and the Go runtime schedules goroutines to run within a logical processor that is bound to a single operating system thread. By default, the Go runtime allocates a single logical processor to execute all the goroutines that are created for our program. Even with this single logical processor and operating system thread, hundreds of thousands of goroutines can be scheduled to run concurrently with amazing efficiency and performance. It is not recommended to add more that one logical processor, but if you want to run goroutines in parallel, Go provides the ability to add more via the GOMAXPROCS environment variable or runtime function.

Concurrency is not Parallelism. Parallelism is when two or more threads are executing code simultaneously against different processors. If you configure the runtime to use more than one logical processor, the scheduler will distribute goroutines between these logical processors which will result in goroutines running on different operating system threads. However, to have true parallelism you need to run your program on a machine with multiple physical processors. If not, then the goroutines will be running concurrently against a single physical processor, even though the Go runtime is using multiple logical processors.

Concurrency Example
Let’s build a small program that shows Go running goroutines concurrently. In this example we are running the code with one logical processor:

Decode JSON Documents In Go

Tue, 21 Jan 2014 00:00:00 +0000

Introduction
We are working on a project where we have to make calls into a web service. Many of the web calls return very large documents that contain many sub-documents. The worst part is, we usually only need a handful of the fields for any given document and those fields tend to be scattered all over the place.

Here is a sample of a smaller document:

Be Selected To Attend GopherCon 2014

Thu, 16 Jan 2014 00:00:00 +0000

Gopher Academy announced a great program today.

Today I’d like to announce the GopherCon Scholarship Program. It’s goal is to increase the visibility of women in the Go community through the most direct path I can imagine: sponsoring their attendance to GopherCon 2014.

The program works by you purchasing one of these special tickets. There will be a selection/nomination process and those woman selected will be able to attend GopherCon thanks to you. It is designed to make GopherCon the most inclusive and welcoming place for all people to get together and learn about Go based technologies.

OutCast.io and Ardan Studios recognizes the need to find ways to engage more women and girls to become technology creators. We have daughters that we would love to see have more opportunity. There is a great organization called ChickTech. They are leading the way in creating local programs to get more women and girls involved in technology. ChickTech is dedicated to:

"retaining women in the technology workforce and increasing the number of women and girls pursuing technology-based careers."

If you’re interested, ChickTech is always looking for local mentors and volunteers. They have a way for you to make a difference that fits your schedule and interests. This is a chance for you to give back to the woman and girls in your life, it just takes an email to get started.

We really want to help and do something special. I will be speaking at GopherCon 2014 in Denver this April and speakers get their flight and hotel room for free. We wanted to give that to some deserving woman who is passionate about technology and otherwise would not be able to attend the conference.

If you’re interested in receiving this free trip to GopherCon or want to nominate someone you know, write a post about yourself or that person and why you/they would like to attend GopherCon. You can also write a post about something cool you are doing with Go. Write your post in Google Docs and share it with me at bill@thekennedyclan.net. GoingGo.net will publish your post and we, with the GopherCon team, will select a winner on February 15th.

We hope to get a great response and are very excited that we can be involved in this great scholarship program. It is important to get more women and girls active in technology and this is a great way to start.

Go Package Management For 2014

Thu, 16 Jan 2014 00:00:00 +0000

Introduction
In October 2013 I sent out a call to action to the Go community. I wanted to form a group of Gophers that would come together and help write a specification and build a working implementation of a package management tool. We are not there yet, but the group did accomplish a few things:

We started a mailing list called Go package management [go-pm] where people could discuss ideas and get feedback on existing and new tools.
We wrote a goals document that outlined what we wanted to accomplish along with guidelines and help for package management tool authors.
We identified and categorized the existing set of tools that are available for use.
We developed a set of user stories that describe the different scenarios people were facing.
We started a list of packages that tool developers can use to test their tools against.
We helped some of the tool developers find and fix bugs.

Overall I think the last 3 months have been productive and I am pleased with the results.

Building Tools
I have come to appreciate that there isn't going to be a silver bullet or a one tool fits all solution. I believe a majority of the use-cases that have been defined in the goals document can be solved and building tools to handle these use-cases is worth the time and effort. If you're thinking about building a tool, please consider these guidelines which are outlined in the goals document:

Work with the Go language as defined in the Go 1 spec of March 28, 2012.
Don’t implement solutions that require feature changes or build tools that change the way Go works.

Provide backwards compatibility with go get and the convention of using VCS repositories in the import paths.
The existing set of programs, build processes and workflows can’t break. You must respect the existing environments and allow them to continue to function.

Prevent version skewing.
Don’t build into the solution the potential for version skewing to occur. Such as requiring semantic versioning in the import path urls. Imports should not need to be changed to access the latest or different versions of a package.

Work on all operating systems and architectures that Go currently supports.
One of the great things about Go is that programs can be built on all these different operating systems and architectures. Your tool should not exclude a platform or make use of a specific operating system construct like symlinks.

Interoperate in such a way that ‘go get’ continues to work for package authors who do not wish to participate, or for existing Go source code that cannot be changed. Also, do not force package authors to choose between making their code go getable or using the proposed solution.
No one should be required to use a tool in order to share or import a repository. The tooling must continue to work for everyone.

All these guidelines are important because they will allow others to try and use your tools without the need to refactor any existing code. They also guarantee that existing projects continue to build and install with the Go tooling on all platforms.

Choices
I would love to see the community rally around a few tools and help improve them. There are two classes of tools that I think work well with Go, Vendoring and Revision Locking.

Vendoring takes the 3rd party source code that is referenced in your project and makes a copy of that code inside a new folder within the project. All the code your project needs is inside the one project repository. Vendoring also provides a performance enhancement on getting the code because only one url call is required.

Revision Locking creates a dependency file that references specific commits in the different version control systems the code is located in. Just like vendoring, the RL tool is used to get, build and install your project. One advantage is that your project repository continues to only contain the specific project code.

Choosing how to handle package management it is a matter of personal preference. The Go team recommends vendoring, which can be found in the Go FAQ. They mention Keith Rarick's tool goven as an option. Keith has abandoned** goven for his other tool godep, which provides both vendoring and revision locking.

** After talking with Keith he has stated that he has not totally abandoned goven, it is just "finished". He continues to maintain the package and merge bug fixes when necessary.

New Call To Action
For 2014 I would like to see the Go community play a greater role in helping the package management tool authors. There are several ways I think this can be done:

Participate in the go-pm group. Give advice and help to those who have questions.
Submit packages that the tool authors can use to help test their tools.
Report bugs and feature requests.
Continue to add user stories that are missing in the goals document.
Submit your new tools and ideas to the go-pm group.
Work with the existing tool authors to improve the tools that we have today.

I hope to see all of you participating in the go-pm mailing list this year. I love Go and only want to see it improve for everyone.

Macro View of Map Internals In Go

Tue, 31 Dec 2013 00:00:00 +0000

Introduction
There are lots of posts that talk about the internals of slices, but when it comes to maps, we are left in the dark. I was wondering why and then I found the code for maps and it all made sense.

https://golang.org/src/runtime/hashmap.go

At least for me, this code is complicated. That being said, I think we can create a macro view of how maps are structured and grow. This should explain why they are unordered, efficient and fast.

Creating and Using Maps
Let’s look at how we can use a map literal to create a map and store a few values:

Queue Your Way To Scalability

Fri, 20 Dec 2013 00:00:00 +0000

Introduction
The first thing I did when I started programming in Go was begin porting my Windows utilities classes and service frameworks over to Linux. This is what I did when I moved from C++ to C#. Thank goodness, I soon learned about Iron.IO and the services they offered. Then it hit me, if I wanted true scalability, I needed to start building worker tasks that could be queued to run anywhere at any time. It was not about how many machines I needed, it was about how much compute time I needed.

Three-Index Slices in Go 1.2

Tue, 17 Dec 2013 00:00:00 +0000

With the release of Go 1.2, slices gained the ability to specify the capacity when performing a slicing operation. This doesn’t mean we can use this index to extend the capacity of the underlying array. It means we can create a new slice whose capacity is restricted. Restricting the capacity provides a level of protection to the underlying array and gives us more control over append operations.

Here are the release notes and design document for the feature request:

http://tip.golang.org/doc/go1.2#three_index
https://docs.google.com/document/d/1GKKdiGYAghXRxC2BFrSEbHBZZgAGKQ-yXK-hRKBo0Kk/pub

Let’s write some code to explore using the new capacity index. As with all my slice posts, I am going to use this InspectSlice function:

Sample Web Application Using Beego and Mgo

Wed, 11 Dec 2013 00:00:00 +0000

Introduction
I am very excited about the Beego web framework. I wanted to share with you how I use the framework to build real world web sites and web services. Here is a picture of the sample website the post is going to showcase:

The sample web application:

Implements a traditional grid view of data calling into MongoDB
Provides a modal dialog box to view details using a partial view to generate the HTML
Implements a web service that returns a JSON document
Takes configuration parameters from the environment using envconfig
Implements tests via goconvey
Leverages my logging package

The code for the sample can be found in the GoingGo repository up on Github:
https://github.com/goinggo/beego-mgo

You can bring the code down and run it. It uses a public MongoDB database I created at MongoLab. You will need git and bazaar installed on your system before running go get.

go get github.com/goinggo/beego-mgo

To quickly run or test the web application, use the scripts located in the zscripts folder.

Web Application Code Structure
Let's take a look at the project structure and the different folders that exist:

controllers	Entry point for each Web call. Controllers process the requests.
localize	Provides localization support for different languages and cultures
models	Models are data structures used by the business and service layers
routes	Mappings between URL's and the controller code that handles those calls.
services	Services provide primitive functions for the different services that exist. These could be database or web calls that perform a specific function.
static	Resource files such as scripts, stylesheets and images
test	Tests that can be run through the go test tool.
utilities	Code that supports the web application. Boilerplate and abstraction layers for accessing the database and handling panics.
views	Code related to rendering views
zscripts	Support scripts to help make it easier to build, run and test the web application

Controllers, Models and Services
These layers make up the bulk of the code that implement the web application. The idea behind the framework is to hide and abstract as much boilerplate code as possible. This is accomplished by implementing a base controller package and a base services package.

Base Controller Package
The base controller package uses composition to abstract default controller behavior required by all controllers:

type (

    BaseController struct {

        beego.Controller

        services.Service

    }

)

func (this *BaseController) Prepare() {

    this.UserId = this.GetString("userId")

    if this.UserId == "" {

        this.UserId = this.GetString(":userId")

    }

    err := this.Service.Prepare()

    if err != nil {

        this.ServeError(err)

        return

    }

}

func (this *BaseController) Finish() {

    defer func() {

        if this.MongoSession != nil {

            mongo.CloseSession(this.UserId, this.MongoSession)

            this.MongoSession = nil

        }

    }()

}

A new type called BaseController is declared with the Beego Controller type and the base Service type embedded directly. This composes the fields and methods of these types directly into the BaseController type and makes them directly accessible through an object of the BaseController type.

Beego Controller framework will execute the Prepare and Finish functions on any Controller object that implements these interfaces. The Prepare function is executed prior to the Controller function being called. These functions will belong to every Controller type by default, allowing this boilerplate code to be implemented once.

Services Package
The Service package maintains state and implements boilerplate code required by all services:

type (

    // Services contains common properties

    Service struct {

        MongoSession *mgo.Session

        UserId string

    }

)

func (this *Service) Prepare() (err error) {

    this.MongoSession, err = mongo.CopyMonotonicSession(this.UserId)

    if err != nil {

        return err

    }

    return err

}

func (this *Service) Finish() (err error) {

    defer helper.CatchPanic(&err, this.UserId, "Service.Finish")

    if this.MongoSession != nil {

        mongo.CloseSession(this.UserId, this.MongoSession)

        this.MongoSession = nil

    }

    return err

}

func (this *Service) DBAction(databaseName string, collectionName string, mongoCall mongo.MongoCall) (err error) {

    return mongo.Execute(this.UserId, this.MongoSession, databaseName, collectionName, mongoCall)

}

In the Service type, the Mongo session and the id of the user is maintained. This version of Prepare handles creating a MongoDB session for use. Finish closes the session which releases the underlying connection back into the pool. The function DBAction provides an abstraction layer for running MongoDB commands and queries.

Buoy Service
This Buoy Service package implements the calls to MongoDB. Let's look at the FindStation function that is called by the controller methods:

func FindStation(service *services.Service, stationId string) (buoyStation *buoyModels.BuoyStation, err error) {

    defer helper.CatchPanic(&err, service.UserId, "FindStation")

    queryMap := bson.M{"station_id": stationId}

    buoyStation = &buoyModels.BuoyStation{}

    err = service.DBAction(Config.Database, "buoy_stations",

        func(collection *mgo.Collection) error {

            return collection.Find(queryMap).One(buoyStation)

        })

    if err != nil {

        if strings.Contains(err.Error(), "not found") == false {

            return buoyStation, err

        }

        err = nil

    }

    return buoyStation, err

}

The FindStation function prepares the query and then using the DBAction function to execute the query against MongoDB.

Implementing Web Calls
With the base types, boilerplate code and service functionality in place, we can now implement the web calls.

Buoy Controller
The BuoyController type is composed solely from the BaseController. By composing the BuoyController in this way, it immediately satisfies the Prepare and Finish interfaces and contains all the fields of a Beego Controller.

The controller functions are bound to routes. The routes specify the urls to the different web calls that the application supports. In our sample application we have three routes:

beego.Router("/", &controllers.BuoyController{}, "get:Index")

beego.Router("/buoy/retrievestation", &controllers.BuoyController{}, "post:RetrieveStation")

beego.Router("/buoy/station/:stationId", &controllers.BuoyController{}, "get,post:RetrieveStationJson")

The route specifies a url path, an instance of the controller used to handle the call and the name of the method from the controller to use. A prefix of which verb is accepted can be specified as well.

The Index controller method is used to deliver the initial html to the browser. This will include the javascript, style sheets and anything else needed to get the web application going:

func (this *BuoyController) Index() {

    region := "Gulf Of Mexico"

    buoyStations, err := buoyService.FindRegion(&this.Service, region)

    if err != nil {

        this.ServeError(err)

        return

    }

    this.Data["Stations"] = buoyStations

    this.Layout = "shared/basic-layout.html"

    this.TplNames = "buoy/content.html"

    this.LayoutSections = map[string]string{}

    this.LayoutSections["PageHead"] = "buoy/page-head.html"

    this.LayoutSections["Header"] = "shared/header.html"

    this.LayoutSections["Modal"] = "shared/modal.html"

}

A call is made into the service layer to retrieve the list of regions. Then the slice of stations are passed into the view system. Since this is setting up the initial view of the application, layouts and the template are specified. When the controller method returns, the beego framework will generate the html for the response and deliver it to the browser.

Building A Weather App Using Go

Tue, 10 Dec 2013 00:00:00 +0000

At Ardan Studios we have spent the last 6 months, in our spare time and on weekends, building a consumer based mobile application called OutCast. The mobile application is tailored towards those who like spending time outdoors, whether that be fishing, hunting or any other type of activity.

This first release of OutCast shows the conditions for the buoy stations and marine forecasts areas within the United States. All this information is updated every 10 minutes and there are map views with traditional grids and search.

The backend processing for buoy and marine data is built using Go and running at Iron.IO as a scheduled worker task. The buoy processing downloads a text file from the NOAA website and rips through it, updating MongoDB with any changes. The marine processing is a bit more complicated. This requires pulling down multiple web pages from the NOAA website and parsing out all the text. Go made building and running these tasks a breeze.

Another important aspect of OutCast is real time weather radar for the last 50 minutes. This has been very challenging on multiple levels. Mainly because we needed a real good image library that would run on Linux and could be integrated with Go. We were fortunate to find ImageMagick’s MagickWand C API and their Go package that provides the CGO bindings (https://github.com/gographics/imagick).

Processing images is an intense piece of work. Sometimes it takes 3 seconds to clean a single image. With 155 radar stations that need to be processed every 5 minutes, it took us several refactors to get things working well. The MagickWand library can only handle processing one image at a time. This restriction places a lot of stress on getting things done accurately within an acceptable amount of time.

Here is a sample of a radar image before and after processing:

Write Your Go Programs Using GEdit

Sat, 23 Nov 2013 00:00:00 +0000

This is a guest post from Tad Vizbaras from Etasoft in South Florida.

There are a number of editors and IDEs for Go development. LiteIde, Vim, Emacs and GEdit just to name a few. Each developer has their own favorite editor for each language they work with. Some like full featured IDE environments while others prefer speed over features. My personal favorite editors for Go development at the moment are Vim and GEdit.

GEdit comes as part of many Linux distros. If you use Ubuntu, it is part of the operating system. GEdit has some decent features like:

* Syntax Highlighting
* Split Windows
* Word Wrapping

Advanced features are left to be handled by external plug-ins.

I prefer project-less development. That means there are no formal project files and projects are preserved via a workspace bound to a directory structure. Go has excellent support for project-less development. When building and installing your projects, the Go tooling, in conjunction with the way Go packages code, can minimize the need for external scripts and makefiles.

GEdit is a decent editor but I could not find any good Plug-ins that would allow me to perform a Go build right from the editor. The "External Tools" Plug-in has worked for me. I was able to set up shortcuts and get "go build" to execute. When you click on errors, displayed in the bottom pane of GEdit, the cursor jumps to exact error location.

When I started programming in Go, the "External Tools" Plug-in worked for me for quite some time. But after awhile, I started to wish that "go build" would run similar to how Linters ran. With Linters, you can run a command after the file is saved. Since Go usually takes only few seconds to build, the Plug-in could execute a "go build" on save and then jump to the error location if there were any.

I wrote a GEdit Plug-in that is developed in Python. Depending on the version of Python you have installed, you may require some small adjustments. This is covered in the Known Issues section below.

Ops, I forgot to mention… Go is Awesome. But you probably already know that.

Meet GoBuild for GEdit 3.x
GoBuild - GEdit 3 Plug-in for Go (golang) development.

GoBuild Plug-in version 1.0 for GEdit. Plug-in attaches to the on_save event in GEdit for Go source code files only. It does nothing for any other file type.

It will run "go build" after the file is saved. If the current filename has "_test.go" in the name, then the Plug-in will run "go test" against the current file’s directory. The Plug-in will wait a number of seconds for the build or test to complete. It will timeout and quit the build or test so GEdit will not freeze.

Images
Plug-in captures "go build" errors and shows them in the Gedit status bar. It also jumps to the first error and highlights error line if error is in the current file.

Label Breaks In Go

Thu, 21 Nov 2013 00:00:00 +0000

Have you ever found yourself in this situation. You have a case statement inside of a for loop and you would like to break from both the case and for statements in a single call?

var err error

timeout := time.After(30 * time.Second)

sigChan := make(chan os.Signal, 1)

signal.Notify(sigChan, os.Interrupt)

complete := make(chan error)

go launchProcessor(complete)

for {

    select {

    case <-sigChan:

       atomic.StoreInt32(&shutdownFlag, 1)

       continue

    case <-timeout:

        os.Exit(1)

    case err = <-complete:

        break

    }

    // Break the loop

    break

}

return err

Here I have an endless for loop waiting on three channels using a select statement.

The first case is listening for an operating system Interrupt event. If the operating system requests the program to shutdown, this case will set a package level variable and continue back into the loop.

The second case is listening for a timeout event. If the programs runs for 30 seconds, the timeout event will fire and the program will immediately terminate.

The third case is listening for a complete event. If the Goroutine that is launched prior to entering the loop completes it work, it will notify the code on this channel. In this case we need to break out of both the case and the for loop.

Fortunately there isn't any more logic to process outside of the select statement, so the second break statement works. If there were other cases that broke out of the select statement and did not require the loop to terminate, I would be in trouble. The code would require more logic and flags to determine when to break out of the loop and when to continue iterating.

Go has an answer to this coding delima. You can define a label and break to that label.

var err error

timeout := time.After(30 * time.Second)

sigChan := make(chan os.Signal, 1)

signal.Notify(sigChan, os.Interrupt)

complete := make(chan error)

go launchProcessor(complete)

Loop:

    for {

        select {

        case <-sigChan:

           atomic.StoreInt32(&shutdownFlag, 1)

           continue

        case <-timeout:

            os.Exit(1)

        case err = <-complete:

            break Loop

        }

    }

return err

I have changed the code a bit by declaring a label called Loop just above the for statement. Then in the last case, the break statement is provided the name of that label. This single call to break will jump the execution of the program outside of the for loop and to the next line of code. In this case, the next line of code is the call to return err.

You can also use a label with a continue statement. This is a silly example but it shows you the mechanism:

    guestList := []string{"bill", "jill", "joan"}

    arrived := []string{"sally", "jill", "joan"}

CheckList:

    for _, guest := range guestList {

        for _, person := range arrived {

            fmt.Printf("Guest[%s] Person[%s]\n", guest, person)

            if person == guest {

                fmt.Printf("Let %s In\n", person)

                continue CheckList

            }

        }

    }

Here is the output:

Guest[bill] Person[sally]

Guest[bill] Person[jill]

Guest[bill] Person[joan]

Guest[jill] Person[sally]

Guest[jill] Person[jill]

Let jill In

Guest[joan] Person[sally]

Guest[joan] Person[jill]

Guest[joan] Person[joan]

Let joan in

In this example there are two for loops, one nested inside the other. From the nested for loop, the continue statement uses a label to jump back to the outer for loop. From the output, you can see that the
outer for loop starts its next iteration. Once the outer for loop is complete, the execution of the program continues on.

If you think this is just a fancy goto statement, it really isn't. The label being referenced must enclose the same for, switch or select statement. As you saw, the continue will still begin the next iteration of the for loop.

Using label breaks and continues in these scenario keeps the code clean and precise.

Using The Log Package In Go

Tue, 05 Nov 2013 00:00:00 +0000

Linux is unique to Windows in many ways, and writing programs in Linux is no exception. The use of standard out, standard err and null devices is not only a good idea but it’s the law. If your programs are going to be logging information, it is best to follow the destination conventions. This way your programs will work with all of the Mac/Linux tooling and hosted environments.

Go has a package in the standard library called log and a type called logger. Using the log package will give you everything you need to be a good citizen. You will be able to write to all the standard devices, custom files or any destination that support the io.Writer interface.

I have provided a really simple sample that will get you started with using logger:

Using XSLT With Go

Sun, 03 Nov 2013 00:00:00 +0000

I am working on a project that requires pulling and processing different XML feeds from the web and storing the data into MongoDB as JSON. Since new feeds come up everyday, changing the Go program to process and publish new feeds is out of the question. A second constraint is that processing has to work in Iron.io or any other linux cloud based environment.

What I needed was a Go program that could take an XML document and XSLT stylesheet at runtime, transform the XML into JSON and then store the JSON to MongoDB. I have some specific field names and other requirements for the JSON document that I need to make sure exist. XSLT makes this real easy to support.

At first I looked at the different C libraries that exist. I figured I could integrate a library using CGO but after a few hours I realized this was not going to work. The libraries I found were huge and complex. Then by chance I found a reference about a program called xsltproc. The program exists both for the Mac and Linux operating systems. In fact, it comes pre-installed on the Mac and an apt-get will get you a copy of the program on your linux operating system.

I have built a sample program that shows how to use xsltproc in your Go programs. Before we download the sample code we need to make sure you have xsltproc installed.

If you are running on a Mac, xsltproc should already exist under /usr/bin

Manage Dependencies With GODEP

Sat, 26 Oct 2013 00:00:00 +0000

Introduction

If you are using 3rd party packages, (packages that you don't own or control), you will want a way to create a reproducible build every time you build your projects. If you use 3rd party packages directly and the package authors change things, your projects could break. Even if things don't break, code changes could create inconsistent behavior and bugs.

Keith Rarick's tool godep is a great step in the right direction for managing 3rd party dependencies and creating reproducible builds. The godep tool gives you two options for managing dependencies. The first option creates a dependency file with version control information and then with some godep magic, the code is built against those versions. You can also Vendor your 3rd party packages inside your projects as well. You never need to change a single source code file and everything is accomplished in conjunction with the go tooling.

Downloading Godep

Download godep using go get and make sure your $GOPATH/bin directory is in your PATH.

go get github.com/kr/godep

export PATH=$PATH:$GOPATH/bin

Create A Project

Build your project using the 3rd party packages as you normally would. Since godep does not require you to change any import paths in the code, 'go get' the code you need and import those packages directly. To keep the post simple, I am going to use an existing program called News Search that uses one 3rd party dependency.

export GOPATH=$HOME/example

go get github.com/goinggo/newssearch

My Channel Select Bug

Thu, 17 Oct 2013 00:00:00 +0000

I was testing new functionality on a program that is already running in production when suddenly the code behaved very badly. What I saw shocked me and then it became obvious why it happened. I also have a race condition just waiting to be a problem.

I have tried to provide a simplified version of the code and the two bugs.

package main

import (

    "fmt"

    "os"

    "os/signal"

    "time"

)

var Shutdown bool = false

func main() {

    sigChan := make(chan os.Signal, 1)

    signal.Notify(sigChan, os.Interrupt)

    for {

        select {

        case <-sigChan:

            Shutdown = true

            continue

        case <-func() chan struct{} {

            complete := make(chan struct{})

            go LaunchProcessor(complete)

            return complete

        }():

            return

        }

    }

}

func LaunchProcessor(complete chan struct{}) {

    defer func() {

        close(complete)

    }()

    fmt.Printf("Start Work\n")

    for count := 0; count < 5; count++ {

        fmt.Printf("Doing Work\n")

        time.Sleep(1 * time.Second)

        if Shutdown == true {

            fmt.Printf("Kill Early\n")

            return

        }

    }

    fmt.Printf("End Work\n")

}

The idea behind this code is to run a task and terminate. It allows the operating system to request the program to terminate early. I always like shutting down the program cleanly when possible.

The sample code creates a channel that is bound to an operating system signal and looks for <ctrl> C from the terminal window. If <ctrl> C is issued, the Shutdown flag is set to true and the program continues back into the select statement. The code also spawns a Go routine that performs the work. That routine checks the Shutdown flag to determine if the program needs to terminate early.

Bug Number 1

Take a look at this part of the code:

case <-func() chan struct{} {

    complete := make(chan struct{})

    go LaunchProcessor(complete)

    return complete

}():

I thought I was being so clever when I wrote this code. I thought it would be cool to execute a function on the fly to spawn the Go routine. It returns a channel that the select waits on to be told the work is complete. When the Go routine is done it closes the channel and the program terminates.

Let's run the program:

Start Work

Doing Work

Doing Work

Doing Work

Doing Work

Doing Work

End Work

As expected the program starts and spawns the Go routine. Once the Go routine is complete the program terminates.

This time I will hit <ctlr> C while the program is running:

Start Work

Doing Work

^CStart Work

Doing Work

Kill Early

Kill Early

When I hit <ctrl> C the program launched the Go routine again!!

I thought that the function associated with the case would only be executed once. Then the select would just wait on the channel moving forward. I had no idea that the function would be executed every time the loop iterated back to the select statement.

To fix the code I needed to remove the function out of the select statement and spawn the Go routine outside of the loop:

func main() {

    sigChan := make(chan os.Signal, 1)

    signal.Notify(sigChan, os.Interrupt)

    complete := make(chan struct{})

    go LaunchProcessor(complete)

    for {

        select {

        case <-sigChan:

            Shutdown = true

            continue

        case <-complete:

            return

        }

    }

}

Now when we run the program we get a better result:

Start Work

Doing Work

Doing Work

^CKill Early

This time when I hit <ctrl> C the program terminate early and doesn't spawn another Go routine again.

Bug Number 2

There is a second less obvious bug lurking in the code as well. Take a look at these pieces of code:

var Shutdown bool = false

if whatSig == syscall.SIGINT {

    Shutdown = true

}

if Shutdown == true {

    fmt.Printf("Kill Early\n")

    return

}

The code is using a package level variable to signal the running Go routine to shut down when <ctrl> C is hit. The code is working every time I hit <ctrl> C so why is there a bug?

First let's run the race detector against the code:

go build -race

./test

While it is running I hit <ctrl> C again:

Start Work

Doing Work

^C==================

WARNING: DATA RACE

Read by goroutine 5:

    main.LaunchProcessor()

        /Users/bill/Spaces/Test/src/test/main.go:46 +0x10b

    gosched0()

        /Users/bill/go/src/pkg/runtime/proc.c:1218 +0x9f

Previous write by goroutine 1:

    main.main()

        /Users/bill/Spaces/Test/src/test/main.go:25 +0x136

    runtime.main()

        /Users/bill/go/src/pkg/runtime/proc.c:182 +0x91

Goroutine 5 (running) created at:

    main.main()

        /Users/bill/Spaces/Test/src/test/main.go:18 +0x8f

    runtime.main()

        /Users/bill/go/src/pkg/runtime/proc.c:182 +0x91

Goroutine 1 (running) created at:

    _rt0_amd64()

        /Users/bill/go/src/pkg/runtime/asm_amd64.s:87 +0x106

==================

Kill Early

Found 1 data race(s)

My use of the Shutdown flag comes up on the race detector. This is because I have two Go routines trying to access the variable in an unsafe way.

My initial reason for not securing access to the variable was practical but wrong. I figured that since the variable is only used to shutdown the program when it becomes necessary, I didn't care about a dirty read. If by chance, within the microsecond of glare there was between writing to the variable and reading the variable, if a dirty read occurred, I would catch it again on the next loop. No harm done, right? Why add complicated channel or locking code for something like this?

Well, there is a little thing called the Go Memory Model.

http://golang.org/ref/mem

The Go Memory Model does not guarantee that the Go routine reading the Shutdown variable will ever see the write by the main routine. It is valid for the write to the Shutdown variable by the main routine to never be written back to main memory. This is because the main routine never reads the Shutdown variable.

This is not happening today but as the Go compiler becomes more sophisticated it could decide to eliminate the write to the Shutdown variable altogether. This behavior is allowed by the Go Memory Model. Also, we don't want code that can't pass the race detector, it is just bad practice, even for practical reasons.

Here is a final version of the code with all bugs fixed:

package main

import (

    "fmt"

    "os"

    "os/signal"

    "sync/atomic"

    "time"

)

var Shutdown int32 = 0

func main() {

    sigChan := make(chan os.Signal, 1)

    signal.Notify(sigChan, os.Interrupt)

    complete := make(chan struct{})

    go LaunchProcessor(complete)

    for {

        select {

        case <-sigChan:

            atomic.StoreInt32(&Shutdown, 1)

            continue

        case <-complete:

            return

        }

    }

}

func LaunchProcessor(complete chan struct{}) {

    defer func() {

        close(complete)

    }()

    fmt.Printf("Start Work\n")

    for count := 0; count < 5; count++ {

        fmt.Printf("Doing Work\n")

        time.Sleep(1 * time.Second)

        if atomic.LoadInt32(&Shutdown) == 1 {

            fmt.Printf("Kill Early\n")

            return

        }

    }

    fmt.Printf("End Work\n")

}

I prefer to use an if statement to check if the Shutdown flag is set so I can sprinkle that code as needed. This solution changes the Shutdown flag from a boolean value to an int32 and uses the atomic functions Store and Load.

In the main routine if a <ctrl> C is detected, the Shutdown flag is safely changed from 0 to 1. In the LaunchProcessor Go routine, the value of the Shutdown flag is compared to 1. If that condition is true the Go routine returns.

It's amazing sometimes how a simple program like this can contain a few gotchas. Things you may have never thought about or realized when you started. Especially when the code always seems to work.

Functions and Naked Returns In Go

Thu, 10 Oct 2013 00:00:00 +0000

In Go values that are returned from functions are passed by value. Go gives you some nice flexibility when it comes to returning values from a function.

Here is a simple example of returning two values from a function:

package main

import (

   "fmt"

)

func main() {

   id, err := ReturnId()

   if err != nil {

      fmt.Printf("ERROR: %s", err)

      return

   }

   fmt.Printf("Id: %d\n", id)

}

func ReturnId() (int, error) {

   id := 10

   return id, nil

}

The function ReturnId returns a value of type integer and of type error. This is something very common that is done in Go. Error handling is performed by returning a value of type error from your functions and the calling function evaluating that value before continuing.

If you don't care about the error for some reason after a function call returns, you can do something like this:

   id, _ := ReturnId()

This time I used an underscore to represent the return value for the second return argument, which was the error. This is really nice because I don't need to declare a variable to hold the value being passed in, I can simply ignore it.

You also have the option to name your return arguments:

func ReturnId() (id int, err error) {

   id = 10

   return id, err

}

If you name your return arguments you are creating local variables just like with your function parameters. This time when I set the id variable, I remove the colon (:) from the short variable declaration and convert it to an assignment operation. Then in the return I specify the return variables as normal.

Naming your return arguments is a nice way to document what you are returning. There is also something else that you can do with your named arguments, or not do:

func ReturnId() (id int, err error) {

   id = 10

   return

}

This is what is called a naked return. I have removed the arguments from the return statement. The Go compiler automatically returns the current values in the return arguments local variables. Though this is really cool you need to watch for shadowing:

func ReturnId() (id int, err error) {

   id = 10

   if id == 10 {

      err := fmt.Errorf("Invalid Id\n")

      return

   }

   return

}

If you try to compile this you will get the following compiler error:

err is shadowed during return

To understand why this error exists you need to understand what curly bracket do inside of a function. Each set of curly brackets define a new level of scope. Take this code for example:

func main() {

   id := 10

   id := 20

   fmt.Printf("Id: %d\n", id)

}

If you try to compile this code you get the following error:

no new variables on left side of :=

This makes sense because you are trying to declare the same variable name twice. The error goes away if we change the code to look like this:

func main() {

   id := 10

   {

       id := 20

       fmt.Printf("Id: %d\n", id)

   }

   fmt.Printf("Id: %d\n", id)

}

The curly brackets define a new stack frame and therefore a new level of scope. The variable name can be reused inside the new set of curly brackets. When the code reaches the closing curly bracket that small piece of the stack is popped.

Look again at the code that caused the shadowing error:

func ReturnId() (id int, err error) {

   id = 10

   if id == 10 {

      err := fmt.Errorf("Invalid Id\n")

      return

   }

   return

}

Inside the if statement we are creating a new variable called err. We are not using the err variable declared as the function return argument. The compiler recognizes this and produces the error. If the compiler did not report this error, you would never see the error that occured inside the if statement. The return err variable is what is passed by default

Naming your return arguments come in real handy when using a defer statement:

func ReturnId() (id int, err error) {

   defer func() {

      if id == 10 {

         err = fmt.Errorf("Invalid Id\n")

      }

   }()

   id = 10

   return

}

Because the return arguments are named, you can reference them in the defer function. You can even change the value of the return arguments inside the defer call and the calling function will see the new values. This version will display the error message.

You need to be aware that the defer statement is evaluated inline with the rest of the code:

func ReturnId() (id int, err error) {

   defer func(id int) {

      if id == 10 {

         err = fmt.Errorf("Invalid Id\n")

      }

   }(id)

   id = 10

   return

}

This version does not display the error message. The value of id is not 10 until after the defer statement is evaluated.

Sometimes it makes sense to use named return arguments, such when using a defer statement at the top of your function. If you are passing raw values out of your function then something like this does not make sense:

package main

import (

   "fmt"

)

func main() {

   ans := AddNumbers(10, 12)

   fmt.Printf("Answer: %d\n", ans)

}

func AddNumbers(a int, b int) (result int) {

   return a + b

}

The return argument is named for the AddNumbers function but never used. Instead we return the answer of the operation directly out of the return. This shows you how you can still return any value you want even if you name the return arguments.

I asked the Go community for their opinions about using named arguments and naked returns:

https://plus.google.com/107537752159279043170/posts/8hMjHhmyNk2

I got a very good mix of opinions that I think everyone should read. Go gives you a lot of flexibility and choice when building your functions. Don't look for a single common practice for everything. Take each function individually and implement the best construct for that use case.

Cross Compile Your Go Programs

Wed, 02 Oct 2013 00:00:00 +0000

Introduction
In my post about building and running programs in Iron.Io, I needed to switched over to my Ubuntu VM to build linux versions of my test programs locally. I love the ability to have Ubuntu available to me for building and testing my code. However, if I can stay on the Mac side it is better.

I have wanted to learn how to cross compile my Go programs for the two platforms I use, darwin/amd64 and linux/amd64. This way I could create final builds of my programs and publish everything from my Mac OS. After a couple of hours I am finally able to do this.

If you don’t have the need to cross compile your code then I recommend you stick with the traditional distribution packages and installs for Go. If this is something you need, then it all starts with downloading the current release of the Go source code.

Installing Git
The Go source code is stored in a DVCS called Github and is located on github.com/golang/go. The first thing you need to do is install Git if you don’t already have it.

Go to the download page on the Git website: http://git-scm.com/downloads

Go Package Management Call To Action

Tue, 01 Oct 2013 00:00:00 +0000

Nathan Youngman, with the help of others, has produced this document outlining months of research and discovery. I would appreciate everyone to honestly read it before continuing with my post.

http://nathany.com/go-packages/

Mitchell Hashimoto also published this post on go-nuts and everyone should read this as well.

https://groups.google.com/forum/#!msg/golang-nuts/BMZDD6FM-QE/LX4JSs4NVLIJ

These two documents outline the current capabilities and issues surrounding the Go tooling as it relates to package management. This has been discussed and discussed at length yet we still don't have a consensus on what the community will rally around.

I am willing to perform all the administrative work involved in putting together a specification and working reference implementation. If we can accomplish this, then we have something that can be used by the community to build a package management tool and we all win.

I have created a Google Group and I ask those of you who are interested to join.

https://groups.google.com/forum/#!forum/go-package-management

The group is public. If you are interested in being a part of the working group please submit your request at the group.

Our goal is to present our specification and working reference implementation to the community at GopherCon in April 2014.

The first step will be to define a set of milestones that allow us to meet our goals by April next year. Then we need to organize the working group into teams with team leaders and members to help do the research and make recommendations.

I hope some of you consider joining the group and help the community solve this problem.

Detecting Race Conditions With Go

Thu, 26 Sep 2013 00:00:00 +0000

I always find it interesting when I realize that something I have been practicing or dealing with for a long time has a name. This time it happens to be race conditions. This is something you can’t avoid thinking about as soon as you have more than one routine sharing any kind of resource. If you’re not thinking about race conditions in your code, now is the time.

A race condition is when two or more routines have access to the same resource, such as a variable or data structure and attempt to read and write to that resource without any regard to the other routines. This type of code can create the craziest and most random bugs you have ever seen. It usually takes a tremendous amount of logging and luck to find these types of bugs. Over the years I have really perfected my logging skills to help identify these problems when they occur.

Back in June with Go version 1.1, the Go tooling introduced a race detector. The race detector is code that is built into your program during the build process. Then once your program is running, it is able to detect and report any race conditions it finds. It is seriously cool and does an incredible job in identifying the code that is the culprit.

Let’s take a very simple program that contains a race condition and build the code with the race detector.

Recursion And Tail Calls In Go

Thu, 26 Sep 2013 00:00:00 +0000

This article was written for and published by Gopher Academy

I was looking at a code sample that showed a recursive function in Go and the writer was very quick to state how Go does not optimize for recursion, even if tail calls are explicit. I had no idea what a tail call was and I really wanted to understand what he meant by Go was not optimized for recursion. I didn't know recursion could be optimized.

For those who don't know what recursion is, put simply, it is when a function calls itself. Why would we ever write a function that would call itself? Recursion is great for algorithms that perform operations on data that can benefit from using a stack, FILO (First In Last Out). It can be faster than using loops and can make your code much simpler.

Performing math operations where the result of a calculation is used in the next calculation is a classic example where recursion shines. As with all recursion, you must have an anchor that eventually causes the function to stop calling itself and return. If not, you have an endless loop that eventually will cause a panic because you will run out of memory.

Why would you run out of memory? In a traditional C program, stack memory is used to handle all the coming and going of function calls. The stack is pre-allocated memory and very fast to use. Look at the following diagram:

Iterating Over Slices In Go

Mon, 23 Sep 2013 00:00:00 +0000

Slices are used everywhere in my code. If I am working with data from MongoDB, it is stored in a slice. If I need to keep track of a collection of problems after running an operation, it is stored in a slice. If you don’t understand how slices work yet or have been avoiding them like I did when I started, read these two posts to learn more.

https://www.ardanlabs.com/blog/2013/08/understanding-slices-in-go-programming.html
https://www.ardanlabs.com/blog/2013/08/collections-of-unknown-length-in-go.html

Pool Go Routines To Process Task Oriented Work

Sat, 14 Sep 2013 00:00:00 +0000

After working in Go for some time now, I learned how to use an unbuffered channel to build a pool of goroutines. I like this implementation better than what is implemented in this post. That being said, this post still has value in what it describes.

https://github.com/goinggo/work

On more than one occasion I have been asked why I use the Work Pool pattern. Why not just start as many Go routines as needed at any given time to get the work done? My answer is always the same. Depending on the type of work, the computing resources you have available and the constraints that exist within the platform, blindly throwing Go routines to perform work could make things slower and hurt overall system performance and responsiveness.

Every application, system and platform has a breaking point. Resources are not unlimited, whether that is memory, CPU, storage, bandwidth, etc. The ability for our applications to reduce and reuse resources is important. Work pools provide a pattern that can help applications manage resources and provide performance tuning options.

Here is the pattern behind the work pool:

Slices of Slices of Slices in Go

Tue, 10 Sep 2013 00:00:00 +0000

I am working on building code to load polygons for the different Marine Forecast areas in the United States. These polygons need to be stored in MongoDB and there is a special way that needs to be done. It would not have been a big deal if it wasn’t for this fact. There isn’t just one polygon for each area. There is an external polygon and then zero to many interior polygons that need to be stored in relationship.

After staring at the problem for a bit I realized that I needed to create a slice of Marine Forecast areas, each of which contained a slice of polygons. To store each polygon ring I needed a slice of geographic coordinates. Finally each coordinate needed to be stored in a two dimensional array of floats.

A picture is worth a thousand words:

Running Go Programs In IronWorker

Sat, 07 Sep 2013 00:00:00 +0000

Introduction
Iron.io has a product called IronWorker which provides a task oriented Linux container that you can run your programs inside. If you are not sure what I mean, think of this as having a temporary Linux virtual machine instantly available for your personal but short term use. IronWorker allows you to load your binaries, code files, support files, shells scripts and just about anything else you may need to run your program in the container. You specify a single task to execute, such as running a shell script or a binary and IronWorker will perform that task when requested. Once the task is complete, IronWorker will tear down the container as if it never existed.

If you are developing in Windows or on the Mac and plan to load and run pre-built binaries in IronWorker, they must be built for Linux. If that could be a problem don’t despair, you have options.

You can create an Ubuntu VM so you can build the Linux binaries you need. This is what I do. I don’t develop in my Ubuntu VM, I use it as a testing and staging area. Your second option is to build your program inside of the IronWorker container and then execute it. You have a lot of flexibility to do what you need with IronWorker.

Timer Routines And Graceful Shutdowns In Go

Wed, 04 Sep 2013 00:00:00 +0000

In my Outcast data server I have several data retrieval jobs that run using different go routines. Each routine wakes up on a set interval. The most complex job is the downloading of radar images. What makes this complex is that there are 155 radar stations throughout the United States that take a new picture every 120 seconds. All these radar images can be put together to create a mosaic. When the go routine wakes up to pull down the new images, it must do this as quickly as possible for all 155 stations. If it doesn’t, the mosaics will be out of sync and any overlays across station boundaries will look off.

Using CGO with Pkg-Config And Custom Dynamic Library Locations

Sat, 31 Aug 2013 00:00:00 +0000

Earlier in the month I wrote a post about using C Dynamic Libraries in Go Programs. The article built a dynamic library in C and created a Go program that used it. The program worked but only if the dynamic library was in the same folder as the program.

This constraint does not allow for the use of the go get command to download, build and install a working version of the program. I did not want to have any requirements to pre-install dependencies or run extra scripts or commands after the call to go get. The Go tool was not going to copy the dynamic library into the bin folder and therefore I would not be able to run the program once the go get command was complete. This was simply unacceptable and there had to be a way to make this work.

The solution to this problem was twofold. First, I needed to use a package configuration file to specify the compiler and linker options to CGO. Second, I needed to set an environment variable so the operating system could find the dynamic library without needing to copy it to the bin folder.

If you look, you will see that some of the standard libraries provide a package configuration (.pc) file. A special program called pkg-config is used by the build tools, such as gcc, to retrieve information from these files.

Organizing Code to Support Go Get

Wed, 28 Aug 2013 00:00:00 +0000

For those of you who are like me, trying to learn the Mac and Linux operating systems, Golang programming and deployment constructs all at the same time, I feel your pain. I have been building a Go application for a couple of months on my Mac and it was time to deploy the code on a local Ubuntu server. I was having a really tough time and it was turning into a disaster. Like always, I kept telling myself, I must be doing something wrong.

Well, I was, big-time. Let me put it this way. After spending 2 days reorganizing repositories and code, I finally figured out how the Go tool works. Now I can update my development environment, deploy, build, install and update my Go applications on any machine with one simple call to go get.

I am sure there are several ways you can organize code that will work with the Go tool. What I am going to present is working for me and I wish I knew all of this when I started. There are a 1000 ways to skin a cat and we all have our favorite way, this one is now mine.

Everything we do in Go should be based on what the Go tool can do. Going off the reservation will get you in trouble, I am living proof. One special command we are going to focus on is get. The get command will download code, build and install packages and produce an executable binary if you are building a program. It is very smart and can read your code to find dependences that it will download, built and install as well. That is, if everything is structured and set correctly.

This document explains all the things the Go tool can do:

http://golang.org/cmd/go/

But I want to concentrate on this section:

http://golang.org/cmd/go/#hdr-Remote_import_path_syntax

Even as I read this document now I have a hard time understanding what it is trying to tell me.

I use Github but you don’t have to. The Go tool supports Github, Mercurial, Subversion and Bazaar out of the box. If you are not using any of these systems for version control, there are ways to give the Go tool the information it needs to support your version control system.

When I say the Go tool supports these version control systems it’s a bit vague right now, so let’s jump into this. Everything starts with a repository, so let’s look at the ones I have for my project. I have two accounts in Github. One is called goinggo which contains all my shared and reusable repositories. The other is ArdanStudios which contains my private repositories.

Here is the goinggo repository:

Collections Of Unknown Length in Go

Mon, 26 Aug 2013 00:00:00 +0000

If you are coming to Go after using a programming language like C# or Java, the first thing you will discover is that there are no traditional collection types like List and Dictionary. That really threw me off for months. I found a package called container/list and gravitated to using it for almost everything.

Something in the back of my head kept nagging me. It didn’t make any sense that the language designers would not directly support managing a collection of unknown length. Everyone talks about how slices are widely used in the language and here I am only using them when I have a well defined capacity or they are returned to me by some function. Something is wrong!!

So I wrote an article earlier in the month that took the covers off of slices in a hope that I would find some magic that I was missing. I now know how slices work but at the end of the day, I still had an array that would have to grow. I was taught in school that linked lists were more efficient and gave you a better way to store large collections of data. Especially when the number of items you need to collect is unknown. It made sense to me.

When I thought about using an empty slice, I had this very WRONG picture in my head:

Using C Dynamic Libraries In Go Programs

Tue, 20 Aug 2013 00:00:00 +0000

My son and I were having fun last weekend building a console based game in Go. I was recreating a game from my youth, back when I was programming on a Kaypro II.

I loved this computer. I would write games in BASIC on it all day and night. Did I mention it was portable. The keyboard would strap in and you could carry it around. LOL.

But I digress, back to my Go program. I figured out a way to use the VT100 escape character codes to draw out a simple screen and started programming some of the logic.

Then something horrible happened and I had a major flashback. I could not get input from stdin without hitting the enter key. Ahhhhh I spent all weekend reading up on how to make this happen. I even found two Go libraries that had support for this but they didn't work. I realized that if I was going to make this happen I needed to build the functionality in C and link that to my Go program.

After a 4 hour coding session at the local Irish pub, I figured it out. I would like to thank Guinness for the inspiration and encouragement I needed. Understand that for the past 10 years I have been writing windows services in C#. For 10 years before that I was writing C/C++ but on the Microsoft stack. Everything I was reading: gcc, gco, static and shared libraries on the Mac and Linux, etc, was foreign to me. I had a lot to learn and still do.

After all my research it became clear I needed to use the ncurses dynamic library. I decided to write a simple program in C using the library. If I could make it work in a compiled C program, I was sure I could get it to work in Go.

Understanding Slices in Go Programming

Thu, 15 Aug 2013 00:00:00 +0000

Since I started programming in Go the concept and use of slices has been confusing. This is something completely new to me. They look like an array, and feel like an array, but they are much more than an array. I am constantly reading how slices are used quite a bit by Go programmers and I think it is finally time for me to understand what slices are all about.

Using Time, Timezones and Location in Go

Thu, 08 Aug 2013 00:00:00 +0000

I ran into a problem today. I was building code to consume NOAA’s tide station XML document and quickly realized I was in trouble. Here is a small piece of that XML document:

<timezone>LST/LDT</timezone>

<item>

<date>2013/01/01</date>

<day>Tue</day>

<time>02:06 AM</time>

<predictions_in_ft>19.7</predictions_in_ft>

<predictions_in_cm>600</predictions_in_cm>

<highlow>H</highlow>

</item>

If you notice the timezone tag, it states the time is in Local Standard Time / Local Daylight Time. This is a real problem because I need to store this data in UTC. Without a proper timezone I am lost. After scratching my head for a bit my business partner showed me two API's that take a latitude and longitude position and return timezone information. Luckily for me I have a latitude and longitude position for each tide station.

If you open this web page you can read the documentation for Google's Timezone API:

https://developers.google.com/maps/documentation/timezone/

The API is fairly simple. It requires a location, timestamps and a flag to identify if the requesting application is using a sensor, like a GPS device, to determine the location.

Here is a sample call to the Google API and response:

https://maps.googleapis.com/maps/api/timezone/json?location=38.85682,-92.991714&sensor=false&timestamp=1331766000

{

    "dstOffset" : 3600.0,

    "rawOffset" : -21600.0,

    "status" : "OK",

    "timeZoneId" : "America/Chicago",

    "timeZoneName" : "Central Daylight Time"

}

There is a limit of 2,500 calls a day. For my initial load of the tide stations, I knew I was going to hit that limit and I didn't want to wait several days to load all the data. So my business partner found the timezone API from GeoNames.

If you open this web page you can read the documentation for GeoNames's Timezone API:

[http://www.geonames.org/export/web-services.html#timezone](/broken-link)

The API requires a free account which is real quick to setup. Once you activate your account you need to find the account page and activate your username for use with the API.

Here is a sample call to the GeoNames API and response:

http://api.geonames.org/timezoneJSON?lat=47.01&lng=10.2&username=demo

{

    "time":"2013-08-09 00:54",

    "countryName":"Austria",

    "sunset":"2013-08-09 20:40",

    "rawOffset":1,

    "dstOffset":2,

    "countryCode":"AT",

    "gmtOffset":1,

    "lng":10.2,

    "sunrise":"2013-08-09 06:07",

    "timezoneId":"Europe/Vienna",

    "lat":47.01

}

This API returns a bit more information. There is no limit to the number of calls you can make but the response times are not guaranteed. I used it for several thousand calls and had no problems.

So now we have two different web calls we can use to get the timezone information. Let's look at how we can use Go to make the Google web call and get an object back that we can use in our program.

First, we need to define a new type that can contain the information we will get back from the API.

package main

import (

    "encoding/json"

    "fmt"

    "io/ioutil"

    "net/http"

    "time"

)

const googleURI = "[https://maps.googleapis.com/maps/api/timezone/json?location=%f,%f&timestamp=%d&sensor=false](/broken-link) "

type GoogleTimezone struct {

    DstOffset    float64 `bson:"dstOffset"`

    RawOffset    float64 `bson:"rawOffset"`

    Status       string  `bson:"status"`

    TimezoneID   string  `bson:"timeZoneId"`

    TimezoneName string  `bson:"timeZoneName"`

}

Go has awesome support for JSON and XML. If you look at the GoogleTimezone struct you will see that each field contains a "tag". A tag is extra data attached to each field that can later be retrieved by our program using reflection. To learn more about tags read this document:

http://golang.org/pkg/reflect/#StructTag

The encoding/json package has defined a set of tags it looks for to help with marshaling and unmarshaling JSON data. To learn more about the JSON support in Go read these documents:

http://golang.org/doc/articles/json_and_go.html
http://golang.org/pkg/encoding/json/

If you make the field names in your struct the same as the field names in the JSON document, you don't need to use the tags. I didn't do that so the tags are there to tell the Unmarshal function how to map the data.

Let's look at a function that can make the API call to Google and unmarshal the JSON document to our new type:

func RetrieveGoogleTimezone(latitude float64, longitude float64) (googleTimezone *GoogleTimezone, err error) {

    defer func() {

        if r := recover(); r != nil {

            err = fmt.Errorf("%v", r)

        }

    }()

    uri := fmt.Sprintf(googleURI, latitude, longitude, time.Now().UTC().Unix())

    resp, err := http.Get(uri)

    if err != nil {

        return googleTimezone, err

    }

    defer resp.Body.Close()

    // Convert the response to a byte array

    rawDocument, err = ioutil.ReadAll(resp.Body)

    if err != nil {

        return googleTimezone, err

    }

    // Unmarshal the response to a GoogleTimezone object

    googleTimezone = new(GoogleTimezone)

    if err = json.Unmarshal(rawDocument, googleTimezone); err != nil {

        return googleTimezone, err

    }

    if googleTimezone.Status != "OK" {

        err = fmt.Errorf("Error : Google Status : %s", googleTimezone.Status)

        return googleTimezone, err

    }

    if len(googleTimezone.TimezoneId) == 0 {

        err = fmt.Errorf("Error : No Timezone Id Provided")

        return googleTimezone, err

    }

    return googleTimezone, err

}

The web call and error handling is fairly boilerplate so let's just talk briefly about the Unmarshal call.

    rawDocument, err = ioutil.ReadAll(resp.Body)

    err = json.Unmarshal(rawDocument, googleTimezone)

When the web call returns, we take the response and store it in a byte array. Then we call the json Unmarshal function, passing the byte array and a reference to our return type pointer variable. The Unmarshal call creates an object of type GoogleTimezone, extracts and copies the data from the returned JSON document and sets the value of our pointer variable. It's really brilliant. If any fields can't be mapped they are simply ignored. The Unmarshal call will return an error if there are casting issues.

So this is great, we can get the timezone data and unmarshal it to an object with three lines of code. Now the only problem is, how the heck do we use the timezoneid to set our location?

Here is the problem again. We have to take the local time from the feed document, apply the timezone information and then convert everything UTC.

Let's look at the feed document again:

<timezone>LST/LDT</timezone>

<item>

<date>2013/01/01</date>

<day>Tue</day>

<time>02:06 AM</time>

<predictions_in_ft>19.7</predictions_in_ft>

<predictions_in_cm>600</predictions_in_cm>

<highlow>H</highlow>

</item>

Assuming we have extracted the data from this document, how can we use the timezoneid to get us out of this jam? Look at the code I wrote in the main function. It uses the time.LoadLocation function and the timezone id we get from the API call to solve the problem:

func main() {

    // Call to get the timezone for this lat and lng position

    googleTimezone, err := RetrieveGoogleTimezone(38.85682, -92.991714)

    if err != nil {

        fmt.Printf("ERROR : %s", err)

        return

    }

    // Pretend this is the date and time we extracted

    year := 2013

    month := 1

    day := 1

    hour := 2

    minute := 6

    // Capture the location based on the timezone id from Google

    location, err := time.LoadLocation(googleTimezone.TimezoneId)

    if err != nil {

        fmt.Printf("ERROR : %s", err)

        return

    }

    // Capture the local and UTC time based on timezone

    localTime := time.Date(year, time.Month(month), day, hour, minute, 0, 0, location)

    utcTime := localTime.UTC()

    // Display the results

    fmt.Printf("Timezone:\t%s\n", googleTimezone.TimezoneId)

    fmt.Printf("Local Time: %v\n", localTime)

    fmt.Printf("UTC Time: %v\n", utcTime)

}

Here is the output:

Timezone:   America/Chicago

Local Time: 2013-01-01 02:06:00 -0600 CST

Time:       2013-01-01 08:06:00 +0000 UTC

Everything worked like a champ. Our localTime variable is set to CST or Central Standard Time, which is where Chicago is located. The Google API provided the correct timezone for the latitude and longitude because that location falls within Missouri.

https://maps.google.com/maps?q=39.232253,-92.991714&z=6

The last question we have to ask is how did the LoadLocation function take that timezone id string and make this work. The timezone id contains both a country and city (America/Chicago). There must be thousands of these timezone ids.

If we take a look at the time package documentation for LoadLocation, we will find the answer:

http://golang.org/pkg/time/#LoadLocation

Here is the documentation for LoadLocation:

LoadLocation returns the Location with the given name.

If the name is "" or "UTC", LoadLocation returns UTC. If the name is "Local", LoadLocation returns Local.

Otherwise, the name is taken to be a location name corresponding to a file in the IANA Time Zone database, such as "America/New_York".

The time zone database needed by LoadLocation may not be present on all systems, especially non-Unix systems. LoadLocation looks in the directory or uncompressed zip file named by the ZONEINFO environment variable, if any, then looks in known installation locations on Unix systems, and finally looks in $GOROOT/lib/time/zoneinfo.zip.

If you read the last paragraph you will see that the LoadLocation function is reading a database file to get the information. I didn't download any database, nor did I set an environment variable called ZONEINFO. The only answer is that this zoneinfo.zip file exists in GOROOT. Let's take a look:

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-08-08+at+8.06.04+PM.png)

Sure enough there is a zoneinfo.zip file located in the lib/time directory where Go was installed. Very Cool !!

There you have it. Now you know how to use the time.LoadLocation function to help make sure your time values are always in the correct timezone. If you have a latitude and longitude, you can use either API to get that timezone id.

I have added a new package called timezone to the GoingGo repository in Github if you want a reusable copy of the code with both API calls. Here is the entire working sample program:

package main

import (

    "encoding/json"

    "fmt"

    "io/ioutil"

    "net/http"

    "time"

)

const (

    googleURI = "[https://maps.googleapis.com/maps/api/timezone/json?location=%f,%f&timestamp=%d&sensor=false](/broken-link)"

)

type GoogleTimezone struct {

    DstOffset    float64 `bson:"dstOffset"`

    RawOffset    float64 `bson:"rawOffset"`

    Status       string  `bson:"status"`

    TimezoneID   string  `bson:"timeZoneId"`

    TimezoneName string  `bson:"timeZoneName"`

}

func main() {

    // Call to get the timezone for this lat and lng position

    googleTimezone, err := RetrieveGoogleTimezone(38.85682, -92.991714)

    if err != nil {

        fmt.Printf("ERROR : %s", err)

        return

    }

    // Pretend this is the date and time we extracted

    year := 2013

    month := 1

    day := 1

    hour := 2

    minute := 6

    // Capture the location based on the timezone id from Google

    location, err := time.LoadLocation(googleTimezone.TimezoneID)

    if err != nil {

        fmt.Printf("ERROR : %s", err)

        return

    }

    // Capture the local and UTC time based on timezone

    localTime := time.Date(year, time.Month(month), day, hour, minute, 0, 0, location)

    utcTime := localTime.UTC()

    // Display the results

    fmt.Printf("Timezone:\t%s\n", googleTimezone.TimezoneID)

    fmt.Printf("Local Time: %v\n", localTime)

    fmt.Printf("UTC Time: %v\n", utcTime)

}

func RetrieveGoogleTimezone(latitude float64, longitude float64) (googleTimezone *GoogleTimezone, err error) {

    defer func() {

        if r := recover(); r != nil {

            err = fmt.Errorf("%v", r)

        }

    }()

    uri := fmt.Sprintf(googleURI, latitude, longitude, time.Now().UTC().Unix())

    resp, err := http.Get(uri)

    if err != nil {

        return googleTimezone, err

    }

    defer resp.Body.Close()

    // Convert the response to a byte array

    rawDocument, err := ioutil.ReadAll(resp.Body)

    if err != nil {

        return googleTimezone, err

    }

    // Unmarshal the response to a GoogleTimezone object

    googleTimezone = new(GoogleTimezone)

    if err = json.Unmarshal(rawDocument, &googleTimezone); err != nil {

        return googleTimezone, err

    }

    if googleTimezone.Status != "OK" {

        err = fmt.Errorf("Error : Google Status : %s", googleTimezone.Status)

        return googleTimezone, err

    }

    if len(googleTimezone.TimezoneID) == 0 {

        err = fmt.Errorf("Error : No Timezone Id Provided")

        return googleTimezone, err

    }

    return googleTimezone, err

}

Gustavo's IEEE-754 Brain Teaser

Mon, 05 Aug 2013 00:00:00 +0000

Back in June, Gustavo Niemeyer posted the following question on his Labix.org blog:

Assume uf is an unsigned integer with 64 bits that holds the IEEE-754 representation for a binary floating point number of that size.

How can you tell if uf represents an integer number?

I can't talk for you, but I write business applications. I just don't have the background to quickly knock out an answer for a question like this. Fortunately Gustavo posted the logic for the answer. I thought it would be fun to better understand the question and break down his answer. I am going to work with a 32 bit number just to make everything easier.

How does the IEEE-754 standard store a floating point number in a binary format? To start I found these two pages:

http://steve.hollasch.net/cgindex/coding/ieeefloat.html
http://class.ece.iastate.edu/arun/CprE281_F05/ieee754/ie5.html

The IEEE-754 specification represents a floating point number in base 2 scientific notation using a special binary format. If you don't know what I mean by base 2 then look at my post on Understanding Type In Go (https://www.ardanlabs.com/blog/2013/07/understanding-type-in-go.html).

Scientific notation is an efficient way of writing very large or small numbers. It works by using a decimal format with a multiplier. Here are some examples:

Base 10 Number	Scientific Notation	Calculation	Coefficient	Base	Exponent	Mantissa
700	7e+2	7 * 10²	7	10	2	0
4,900,000,000	4.9e+9	4.9 * 10⁹	4.9	10	9	.9
5362.63	5.36263e+3	5.36263 * 10³	5.36263	10	3	.36263
-0.00345	3.45e-3	3.45 * 10^-3	3.45	10	-3	.45
0.085	1.36e-4	1.36 * 2^-4	1.36	2	-4	.36

In normal scientific notation form there is always just one digit on the left side of the decimal point. For base 10 numbers that digit must be between 1 through 9 and for base 2 numbers that digit can only be 1.

The entire number in scientific notation is called the Coefficient. The Mantissa is all the numbers to the right of the decimal point. These terms are important so take the time to study and understand the chart above.

How we move the decimal point to that first position determines the value of the Exponent. If we have to move the decimal point to the left, the Exponent is a positive number, to the right, it is a negative number. Look at the chart above and see the Exponent value for each example.

The Base and the Exponent work together in the notation. The exponent determines the "Power Of" calculation we need to perform on the base. In the first example the number 7 is multiplied by 10 (The Base) to the power of 2 (The Exponent) to get back to the original base 10 number 700. We moved the decimal point to the left two places to convert 700 to 7.00, which made the Exponent +2 and created the notation of 7e+2.

The IEEE-754 standard does not store base 10 scientific notation numbers but base 2 scientific notation numbers. The last example in the chart above represents the base 10 number 0.085 in base 2 scientific notation. Let's learn how that notation is calculated.

Base 10 Number	Scientific Notation	Calculation	Coefficient	Base	Exponent	Mantissa
0.085	1.36e-4	1.36 * 2^-4	1.36	2	-4	.36

We need to divide the base 10 number (0.085) by some power of two so we get a 1 + Fraction value. What do I mean by a 1 + Fraction value? We need a number that looks like the Coefficient in the example, 1 + .36. The IEEE-754 standard requires that we have a "1." in the Coefficient. This allows us to only have to store the Mantissa and give us an extra bit of precision.

If we use brute force you will see when we finally get the 1 + Fraction value for 0.085:

0.085 / 2^-1 = 0.17
0.085 / 2^-2 = 0.34
0.085 / 2^-3 = 0.68
0.085 / 2^-4 = 1.36 ** We found the 1 + Fraction

An exponent of -4 gives us the 1 + Fraction value we need. Now we have everything we need to store the base 10 number 0.085 in IEEE-754 format.

Let's look at how the bits are laid out in the IEEE-754 format.

Precision	Sign	Exponent	Fraction Bits	Bias
Single (32 Bits)	1 [31]	8 [30-23]	23 [22-00]	127
Double (64 Bits)	1 [63]	11 [62-52]	52 [51-00]	1023

The bits are broken into three parts. There is a bit reserved for the sign, bits for the exponent and bits that are called fraction bits. The fractions bits are where we store the mantissa as a binary fraction.

If we store the value of 0.085 using Single Precision (a 32 bit number), the bit pattern in IEEE-754 would look like this:

Sign	Exponent (123)	Fraction Bits (.36)
0	0111 1011	010 1110 0001 0100 0111 1011

The Sign bit, the leftmost bit, determines if the number is positive or negative. If the bit is set to 1 then the number is negative else it is positive.

The next 8 bits represent the Exponent. In our example, the base 10 number 0.085 is converted to 1.36 * 2^-4 using base 2 scientific notation. Therefore the value of the exponent is -4. In order to be able to represent negative numbers, there is a Bias value. The Bias value for our 32 bit representation is 127. To represent the number -4, we need to find the number, that when subtract against the Bias, gives us -4. In our case that number is 123. If you look at the bit pattern for the Exponent you will see it represents the number 123 in binary.

The remaining 23 bits are the Fraction bits. To calculate the bit pattern for the fraction bits, we need to calculate and sum binary fractions until we get the value of the Mantissa, or a value that is as close as possible. Remember, we only need to store the Mantissa because we always assume that the "1." value exists.

To understand how binary fractions are calculated, look at the following chart. Each bit position from left to right represents a fractional value:

Binary	Fraction	Decimal	Power
0.1	1/2	0.5	2^-1
0.01	1/4	0.25	2^-2
0.001	1/8	0.125	2^-3

We need to set the correct number of fraction bits that add up to or gets us close enough to the mantissa. This is why we can sometimes lose precision.

Bit	Value	Fraction	Decimal	Total
010 1110 0001 0100 0111 1011 = (0.36)
2	4	1/4	0.25	0.25
4	16	1/16	0.0625	0.3125
5	32	1/32	0.03125	0.34375
6	64	1/64	0.015625	0.359375
11	2048	1/2048	0.00048828125	0.35986328125
13	8192	1/8192	0.0001220703125	0.3599853515625
17	131072	1/131072	0.00000762939453	0.35999298095703
18	262144	1/262144	0.00000381469727	0.3599967956543
19	524288	1/524288	0.00000190734863	0.35999870300293
20	1048576	1/1048576	0.00000095367432	0.35999965667725
22	4194304	1/4194304	0.00000023841858	0.35999989509583
23	8388608	1/8388608	0.00000011920929	0.36000001430512

You can see that setting these 12 bits get us to the value of 0.36 plus some extra fractions.

Let's sum up what we now know about the IEEE-754 format:

1. Any base 10 number to be stored is converted to base 2 scientific notation.
2. The base 2 scientific notation value we use must follow the 1 + Fraction format.
3. There are three distinct sections in the format.
4. The Sign bit determines if the number is positive or negative.
5. The Exponent bits represent a number that needs to be subtracted against the Bias.
6. The Fraction bits represent the Mantissa using binary fraction summation.

Let's prove that our analysis is correct about the IEEE-754 format. We should be able to store the number 0.85 and display bit patterns and values for each section that match everything we have seen.

The following code stores the number 0.085 and displays the IEEE-754 binary representation:

package main

import (

    "fmt"

    "math"

)

func main() {

    var number float32 = 0.085

    fmt.Printf("Starting Number: %f\n\n", number)

    // Float32bits returns the IEEE 754 binary representation

    bits := math.Float32bits(number)

    binary := fmt.Sprintf("%.32b", bits)

    fmt.Printf("Bit Pattern: %s | %s %s | %s %s %s %s %s %s\n\n",

        binary[0:1],

        binary[1:5], binary[5:9],

        binary[9:12], binary[12:16], binary[16:20],

        binary[20:24], binary[24:28], binary[28:32])

    bias := 127

    sign := bits & (1 << 31)

    exponentRaw := int(bits >> 23)

    exponent := exponentRaw - bias

    var mantissa float64

    for index, bit := range binary[9:32] {

        if bit == 49 {

            position := index + 1

            bitValue := math.Pow(2, float64(position))

            fractional := 1 / bitValue

            mantissa = mantissa + fractional

        }

    }

    value := (1 + mantissa) * math.Pow(2, float64(exponent))

    fmt.Printf("Sign: %d Exponent: %d (%d) Mantissa: %f Value: %f\n\n",

        sign,

        exponentRaw,

        exponent,

        mantissa,

        value)

}

When we run the program we get the following output:

Starting Number: 0.085000

Bit Pattern: 0 | 0111 1011 | 010 1110 0001 0100 0111 1011

Sign: 0 Exponent: 123 (-4) Mantissa: 0.360000 Value: 0.085000

If you compare the displayed bit pattern with our example above, you will see that it matches. Everything we learned about IEEE-754 is true.

Now we should be able to answer Gustavo's question. How can we tell if the value being stored is an integer? Here is a function, thanks to Gustavo's code, that tests if the IEEE-754 stored value is an integer:

func IsInt(bits uint32, bias int) {

    exponent := int(bits >> 23) - bias - 23

    coefficient := (bits & ((1 << 23) - 1)) | (1 << 23)

    intTest := (coefficient & (1 << uint32(-exponent) - 1))

    fmt.Printf("\nExponent: %d Coefficient: %d IntTest: %d\n",

        exponent,

        coefficient,

        intTest)

    if exponent < -23 {

        fmt.Printf("NOT INTEGER\n")

        return

    }

    if exponent < 0 && intTest != 0 {

        fmt.Printf("NOT INTEGER\n")

        return

    }

    fmt.Printf("INTEGER\n")

}

So how does this function work?

Let's start with the first condition which tests if the Exponent is less than -23. If we use the number 1 as our test number, the exponent will be 127 which is the same as the Bias. This means when we subtract the Exponent against the Bias we will get zero.

Starting Number: 1.000000

Bit Pattern: 0 | 0111 1111 | 000 0000 0000 0000 0000 0000

Sign: 0 Exponent: 127 (0) Mantissa: 0.000000 Value: 1.000000

Exponent: -23 Coefficient: 8388608 IntTest: 0

INTEGER

The test function adds an extra subtraction of 23, which represents the starting bit position for the Exponent in the IEEE-754 format. That is why you see -23 for the Exponent value coming from the test function.

Precision	Sign	Exponent	Fraction Bits	Bias
Single (32 Bits)	1 [31]	8 [30-23]	23 [22-00]	127

This subtraction is required to help with the second test. So any value that is less than -23 must be less than one (1) and therefore not an integer.

To understand how the second test works, let's use an integer value. This time we will set the number to 234523 in the code and run the program again.

Starting Number: 234523.000000

Bit Pattern: 0 | 1001 0000 | 110 0101 0000 0110 1100 0000

Sign: 0 Exponent: 144 (17) Mantissa: 0.789268 Value: 234523.000000

Exponent: -6 Coefficient: 15009472 IntTest: 0

INTEGER

The second test looks for two conditions to identify if the number is not an integer. This requires the use of bitwise mathematics. Let's look at the math we are performing in the function:

    exponent := int(bits >> 23) - bias - 23

    coefficient := (bits & ((1 << 23) - 1)) | (1 << 23)

    intTest := coefficient & ((1 << uint32(-exponent)) - 1)

The coefficient calculation is adding the 1 + to the Mantissa so we have the base 2 Coefficient value.

When we look at the first part of the coefficient calculation we see the following bit patterns:

coefficient := (bits & ((1 << 23) - 1)) | (1 << 23)

Bits:                   01001000011001010000011011000000

(1 << 23) - 1:          00000000011111111111111111111111

bits & ((1 << 23) - 1): 00000000011001010000011011000000

The first part of the coefficient calculation removes the bits for the Sign and Exponent from the entire IEEE-754 bit pattern.

The second part of the coefficient calculation adds the "1 +" into the binary bit pattern:

coefficient := (bits & ((1 << 23) - 1)) | (1 << 23)

bits & ((1 << 23) - 1): 00000000011001010000011011000000

(1 << 23):              00000000100000000000000000000000

coefficient:            00000000111001010000011011000000

Now that the coefficient bit pattern is set, we can calculate the intTest value:

exponent := int(bits >> 23) - bias - 23

intTest := (coefficient & ((1 << uint32(-exponent)) - 1))

exponent:                     (144 - 127 - 23) = -6

1 << uint32(-exponent):       000000

(1 << uint32(-exponent)) - 1: 111111

coefficient:                 00000000111001010000011011000000

1 << uint32(-exponent)) - 1: 00000000000000000000000000111111

intTest:                     00000000000000000000000000000000

The value of the exponent we calculate in the test function is used to determine the number of bits we will compare against the Coefficient. In this case the exponent value is -6. That is calculated by subtracting the stored Exponent value (144) against the Bias (127) and then against the starting bit position for the Exponent (23). This gives us a bit pattern of 6 ones (1's). The final operation takes those 6 bits and AND's them against the rightmost 6 bits of the Coefficient to get the intTest value.

The second test is looking for an exponent value that is less than zero (0) and an intTest value that is NOT zero (0). This would indicate the number being stored is not an integer. In our example with 234523, the Exponent is less than zero (0), but the value of intTest is zero (0). We have an integer.

I have included the sample code in the Go playground so you can play with it.

[http://play.golang.org/p/3xraud43pi](/broken-link)

If it wasn't for Gustavo's code I could never have identified the solution. Here is a link to his solution:

http://bazaar.launchpad.net/~niemeyer/strepr/trunk/view/6/strepr.go#L160

Here is a copy of the code that you can copy and paste:

package main

import (

    "fmt"

    "math"

)

func main() {

    var number float32 = 234523

    fmt.Printf("Starting Number: %f\n\n", number)

    // Float32bits returns the IEEE 754 binary representation

    bits := math.Float32bits(number)

    binary := fmt.Sprintf("%.32b", bits)

    fmt.Printf("Bit Pattern: %s | %s %s | %s %s %s %s %s %s\n\n",

        binary[0:1],

        binary[1:5], binary[5:9],

        binary[9:12], binary[12:16], binary[16:20],

        binary[20:24], binary[24:28], binary[28:32])

    bias := 127

    sign := bits & (1 << 31)

    exponentRaw := int(bits >> 23)

    exponent := exponentRaw - bias

    var mantissa float64

    for index, bit := range binary[9:32] {

        if bit == 49 {

            position := index + 1

            bitValue := math.Pow(2, float64(position))

            fractional := 1 / bitValue

            mantissa = mantissa + fractional

        }

    }

    value := (1 + mantissa) * math.Pow(2, float64(exponent))

    fmt.Printf("Sign: %d Exponent: %d (%d) Mantissa: %f Value: %f\n\n",

        sign,

        exponentRaw,

        exponent,

        mantissa,

        value)

    IsInt(bits, bias)

}

func IsInt(bits uint32, bias int) {

    exponent := int(bits>>23) - bias - 23

    coefficient := (bits & ((1 << 23) - 1)) | (1 << 23)

    intTest := coefficient & ((1 << uint32(-exponent)) - 1)

    ShowBits(bits, bias, exponent)

    fmt.Printf("\nExp: %d Frac: %d IntTest: %d\n",

        exponent,

        coefficient,

        intTest)

    if exponent < -23 {

        fmt.Printf("NOT INTEGER\n")

        return

    }

    if exponent < 0 && intTest != 0 {

        fmt.Printf("NOT INTEGER\n")

        return

    }

    fmt.Printf("INTEGER\n")

}

func ShowBits(bits uint32, bias int, exponent int) {

    value := (1 << 23) - 1

    value2 := (bits & ((1 << 23) - 1))

    value3 := (1 << 23)

    coefficient := (bits & ((1 << 23) - 1)) | (1 << 23)

    fmt.Printf("Bits:\t\t\t%.32b\n", bits)

    fmt.Printf("(1 << 23) - 1:\t\t%.32b\n", value)

    fmt.Printf("bits & ((1 << 23) - 1):\t\t%.32b\n\n", value2)

    fmt.Printf("bits & ((1 << 23) - 1):\t\t%.32b\n", value2)

    fmt.Printf("(1 << 23):\t\t\t%.32b\n", value3)

    fmt.Printf("coefficient:\t\t\t%.32b\n\n", coefficient)

    value5 := 1 << uint32(-exponent)

    value6 := (1 << uint32(-exponent)) - 1

    inTest := (coefficient & ((1 << uint32(-exponent)) - 1))

    fmt.Printf("1 << uint32(-exponent):\t\t%.32b\n", value5)

    fmt.Printf("(1 << uint32(-exponent)) - 1:\t%.32b\n\n", value6)

    fmt.Printf("coefficient:\t\t\t%.32b\n", coefficient)

    fmt.Printf("(1 << uint32(-exponent)) - 1:\t%.32b\n", value6)

    fmt.Printf("intTest:\t\t\t%.32b\n", inTest)

}

I want to thank Gustavo for posting the question and giving me something to really fight through to understand.

An RSS Feed Searching Framework Using Go

Mon, 29 Jul 2013 00:00:00 +0000

This article was written for and published by Safari Books Online

Back in May, I finally decided that it was time to move away from the Microsoft stack to Linux. The cost of developing, licensing, and purchasing cloud computing on the Microsoft stack is too expensive. My company was approved for BizSpark, which made development possible for a small company like mine, but what am I going to do three years from now? My other problem is how can I ask a client to pay thousands of dollars for Microsoft cloud computing and licensing, especially when the open source technologies that run on Linux only cost hundreds of dollars, and those technologies scale better.

We have been using Ruby for some time now for our web projects. Unfortunately, we have needed to stay on .NET for our server development. I didn’t want to start writing C/C++ again and Ruby, Erlang and every other language I looked at just didn’t feel right for this type of development. Then my partner told me to look at Go.

You can implement very robust code using the Go programming language and paradigms. In a few hours I was able to write a program called NewsSearch. The program downloads a list of RSS feed documents and processes them using different search plugins. The first search plugin is called Expression. It searches through each RSS document looking for matching keywords or regular expressions. It displays any matching documents and where the match was made.

Continue reading →

An Open Source Debate

Sat, 27 Jul 2013 00:00:00 +0000

This article was written for and published by Safari Books Online

If you read my article titled Analyze Data With MongoDB and Go, then you will understand the context of this debate with the idea of open source. That article talks about an application we are building, and it publishes some of the proprietary data we are gathering. It also shows techniques we are using to perform the data analysis. This has caused a passionate debate among the business partners in my company. Does the article provide too much proprietary information that could harm our success with the application?

Let me go back in time for a bit. When Microsoft released .NET to the world back in 2003, I was quick to leave C++ programming behind. I began to port all of my C++ utilities, so I could write Windows services in C#. During this time, I wrote two articles about my thread pooling and socket utilities. They were published and I was excited, but I was quickly told to stop because I was crazy for giving away all of this code.

Understanding Pointers and Memory Allocation

Sat, 27 Jul 2013 00:00:00 +0000

In the documentation provided by the Go language team you will find great information on pointers and memory allocation. Here is a link to that documentation:

http://golang.org/doc/faq#Pointers

We need to start with the understanding that all variables contain a value. Based on the type that variable represents will determine how we can use it to manipulate the memory it contains. Read this post to learn more: Understanding Type In Go

Analyze Data With MongoDB and Go

Wed, 24 Jul 2013 00:00:00 +0000

This article was written for and published by Safari Books Online

My company is building a mobile application called Outcast. The idea behind Outcast is to give people who love the outdoors the ability to get ahead of the weather. By analyzing real time buoy, tide, lunar and solar data with user preferences and experiences, the application can deliver relevant information and forecasts. The user helps with the forecasting by providing an experience review after their outdoor activities have ended. Over time, the application learns what is the best condition for each individual user, their activity and favorite locations.

OSCON 2013 - The Gophers Are Coming

Sat, 20 Jul 2013 00:00:00 +0000

On June 22nd through the 26th, Portland Oregon is hosting OSCON (Open Source Convention). The event is run by O’Reilly. After looking at the speakers list it appears it is going to be a great convention.

There are three Gophers who will be giving talks. Check out the links:

Francesc Campoy Flores
Go Developer Programs Engineer, Google Inc.
http://www.oscon.com/oscon2013/public/schedule/speaker/155088

Johan Euphrosine
Developer Programs Engineer, Google
http://www.oscon.com/oscon2013/public/schedule/speaker/155083

Brad Fitzpatrick
Software Engineer, Google
http://www.oscon.com/oscon2013/public/schedule/speaker/25905

I have also been asked by Backstop Media to write a few article on Go to supplement the conference. Once they are publish I will provide links and a quick summary.

If you are going to the convention have a great time. I can’t wait to see the content and videos that come out of it.

Understanding Type in Go

Fri, 12 Jul 2013 00:00:00 +0000

When I was coding in C/C++ it was imperative to understand type. If you didn’t, you would get into a lot of trouble with both the compiler and running your code. Regardless of the language, type touches every aspect of programming syntax. A good understand of types and pointers is critical to good programming. This post will focus on type.

Take these bytes of memory for starters:

Object Oriented Programming in Go

Wed, 10 Jul 2013 00:00:00 +0000

Someone asked a question on the forum today on how to gain the benefits of inheritance without embedding. It is really important for everyone to think in terms of Go and not the languages they are leaving behind. I can’t tell you much code I removed from my early Go implementations because it wasn’t necessary. The language designers have years of experience and knowledge. Hindsight is helping to create a language that is fast, lean and really fun to code in.

Singleton Design Pattern in Go

Sat, 06 Jul 2013 00:00:00 +0000

Multi-threaded applications are very complicated, especially when your code is not organized and consistent with how resources are accessed, managed and maintained. If you want to minimize bugs you need philosophies and rules to live by. Here are some of mine:

Resource allocation and de-allocation should be abstracted and managed within the same type.
Resource thread safeness should be abstracted and managed within the same type.
A public interface should be the only means to accessing shared resources.
Any thread that allocates a resource should de-allocated the same resource.

In Go we don't have threads but Go Routines. The Go runtime abstracts the threading and task swapping of these routines. Regardless, the same philosophies and rules apply.

One of my favorite design patterns is the Singleton. It provides a great implementation when you only need one instance of a type and that type manages shared resources. A Singleton is a design pattern where the type creates an instance of itself and keeps that reference private. Access to the shared resources managed by that reference is abstracted through a static pubic interface. These static methods also provide thread safeness. The application using the Singleton is responsible for initializing and de-initializing the Singleton but never has direct access to the internals.

It escaped me for some time how to implement a Singleton in Go because Go is not a traditional object oriented programming language and there are no static methods.

I consider Go to be a light object oriented programming language. Yes it does have encapsulation and type member functions but it lacks inheritance and therefore traditional polymorphism. In all of the OOP languages I have ever used, I never used inheritance unless I wanted to implement polymorphism. With the way interfaces are implemented in Go there is no need for inheritance. Go took the best parts of OOP, left out the rest and gave us a better way to write polymorphic code.

In Go we can implement a Singleton by leveraging the scoping and encapsulation rules of packages and types. For this post we will explore my straps package since it will give us a real world example.

The straps package provides a mechanism to store configuration options (straps) in an XML document and read them into memory for use by the application. The name strap comes from the early days of configuring networking equipment. Those settings were called straps and that name has always stuck with me. In the MacOS we have .plist files, in .Net we have app.config files and in Go I have straps.xml files.

Here is a sample straps files for one of my applications:

<straps>

    <!-- Log Settings -->

    <strap key="baseFilePath" value="/Users/bill/Logs/OC-DataServer">

    <strap key="machineName" value="my-machine">

    <strap key="daysToKeep" value="1">

    <!-- ServerManager Settings -->

    <strap key="cpuMultiplier" value="100">

</straps>

The straps package knows how to read this xml file and provide access to the values via a Singleton based public interface. Since these values only need to be read into memory once a Singleton is a great option for this package.

Here is the package and type information for straps:

package straps

import (

    "encoding/xml"

    "io"

    "os"

    "path/filepath"

    "strconv"

)

.

. Types Removed

.

type straps struct {

    StrapMap map[string]string // The map of strap key value pairs

}

var st straps // A reference to the singleton

I am not going to talk about the aspects of reading the XML document. If you are interested please read this blog post https://www.ardanlabs.com/blog/2013/06/reading-xml-documents-in-go.html.

In the code snippet above you will see the package name (straps), the definition of the private type straps and the private package variable st. The st variable will contain the value for the Singleton.

The scoping rules for Go state that types and functions that start with a capital letter are public and accessible outside of the package. Types and functions that start with a lowercase letter are private and not accessible outside of the package.

I name my variables that are defined within the scope of a function with lowercase letters. Variable names defined outside the scope of a function, such as type members and package variables start with a capital letter. This allows me to look at the code and know instantly where memory for any given variable is being referenced.

Both the straps type and the st variable are private and only accessible from within the package.

Look at the Load function which initializes the Singleton for use:

func MustLoad() {

    // Find the location of the straps.xml file

    strapsFilePath, err := filepath.Abs("straps.xml")

    // Open the straps.xml file

    file, err := os.Open(strapsFilePath)

    if err != nil {

        panic(err.Error())

    }

    defer file.Close()

    // Read the straps file

    xmlStraps, err := readStraps(file)

    if err != nil {

        panic(err.Error())

    }

    // Create a straps object

    st = straps{

        StrapMap: make(map[string]string),

    }

    // Store the key/value pairs for each strap

    for _, strap := range xmlStraps {

        st.StrapMap[strap.Key] = strap.Value

    }

}

The Load function is a public function of the package. Applications can access this function through the package name. You can see how I use names that start with a lowercase letter for local variables. At the bottom of the Load function a straps object is created and the reference is set to the st variable. At this point the Singleton exists and straps is ready to use.

Accessing the straps is done with the public function Strap:

func Strap(key string) string {

    return st.StrapMap[key]

}

The public function Strap uses the Singleton reference to access the shared resource. In this case the map of straps. If the map could change during the lifetime of the application, then a mutex or some other synchronization object would need to be used to protect the map. Luckily the straps never change once they are loaded.

Since the resource being managed by straps is just memory there is no need for an Unload or Close method. If we needed a function to close any resources another public function would have to be created.

If private methods are required in the Singleton package to help organize code, I like to use member functions. Since the type is private I can make the member functions public because they won't be accessible. I also think the member functions help make the code more readable. By looking to see if the function is a member function or not I know if the function is private or part of the public interface.

Here is a example of using a member function:

func SomePublicFunction() {

    .

    st.SomePrivateMemberFunction("key")

    .

}

func (straps *straps) SomePrivateMemberFunction(key string) {

    return straps.StrapMap[key]

    .

}

Since the function is a member function we need to use the st variable to make the function call. From within the member function I use the local variable (straps) and not the st variable. The member function is public but the reference is private so only the package can reference the member function. This is just a convention I established for myself.

Here is a sample program that uses the straps package:

package main

import (

    "ArdanStudios/straps"

)

func main() {

    straps.MustLoad()

    cpu := straps.Strap("cpuMultiplier")

}

In main we don't need to allocate any memory or maintain references. Through the package name we call Load to initialize the Singleton. Then through the package name again we access the public interface, in this case the Strap function.

If you have the same need to managed shared resources through a public interface try using a Singleton.

As always I hope this helps you write better and bug less code.

How Packages Work in Go

Fri, 05 Jul 2013 00:00:00 +0000

Since I started writing code in Go it has been a mystery to me how best to organize my code and use the package keyword. The package keyword is similar to using a namespace in C#, however the convention is to tie the package name to the directory structure.

Go has this web page that attempts to explain how to write Go Code.

http://golang.org/doc/code.html

When I started programming in Go this was one of the first documents I read. This went way over my head, mainly because I have been working in Visual Studio and code is packaged for you in Solution and Project files. Working out of a directory on the file system was a crazy thought. Now I love the simplicity of it but it has taken quite a while for it all to make sense.

"How to Write Go Code" starts out with the concept of a Workspace. Think of this as the root directory for your project. If you were working in Visual Studio this is where the solution or project file would be located. Then from inside your Workspace you need to create a single sub-directory called src. This is mandatory if you want the Go tools to work properly. From within the src directory you have the freedom to organize your code the way you want. However you need to understand the conventions set forth by the Go team for packages and source code or you could be refactoring your code down the line.

On my machine I created a Workspace called Test and the required sub-directory called src. This is the first step in creating your project.

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.03.44+AM.png)

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.10.42+AM.png)

Then in LiteIDE open the Test directory (the Workspace), and create the following sub-directories and empty Go source code files.

First we create a sub-directory for the application we are creating. The name of the directory where the main function is located will be the name of the executable. In our case main.go contains the main function and is under the myprogram directory. This means our executable file will be called myprogram.

The other sub-directories inside of src will contain packages for our project. By convention the name of the directory should be the name of the package for those source code files that are located in that directory. In our case the new packages are called samplepkg and subpkg. The name of the source code files can be anything you like.

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.07.09+AM.png)

Create the same package folders and empty Go source code files to follow along.

If you don't add the Workspace folder to the GOPATH we will have problems.

It took me a bit to realize that the Custom Directories window is a Text Box. So you can edit those folders directly. The System GOPATH is read only.

The Go designers have done several things when naming their packages and source code files. All the file names and directories are lowercase and they did not use underscores to break words apart in the package directory names. Also, the package names match the directory names. Code files within a directory belong to a package named after the directory.

Take a look at the Go source code directory for a few of the standard library packages:

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.09.25+AM.png)

The package directories for bufio and builtin are great examples of the directory naming convention. They could have been called buf_io and built_in.

Look again at the Go source code directory and review the names of the source code files.

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.17.49+AM.png)

Notice the use of the underscore in some of the file names. When the file contains test code or is specific to a particular platform, an underscore is used.

The usual convention is to name one of the source code files the same as the package name. In bufio this convention is followed. However, this is a loosely followed convention.

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.20.36+AM.png)

In the fmt package you will notice there is no source code file named fmt.go. I personally like naming my packages and source code files differently.

Last, open the doc.go, format.go, print.go and scan.go files. They are all declared to be in the fmt package.

Let's take a look at the code for sample.go:

package samplepkg

import (

    "fmt"

)

type Sample struct {

    Name string

}

func New(name string) (sample * Sample) {

    return &Sample{

        Name: name,

    }

}

func (sample * Sample) Print() {

    fmt.Println("Sample Name:", sample.Name)

}

The code is useless but it will let us focus on the two important conventions. First, notice the name of the package is the same as the name of the sub-directory. Second, there is a function called New.

The function New is a Go convention for packages that create a core type or different types for use by the application developer. Look at how New is defined and implemented in log.go, bufio.go and cypto.go:

log.go

// New creates a new Logger. The out variable sets the

// destination to which log data will be written.

// The prefix appears at the beginning of each generated log line.

// The flag argument defines the logging properties.

func New(out io.Writer, prefix string, flag int) * Logger {

    return &Logger{out: out, prefix: prefix, flag: flag}

}

bufio.go

// NewReader returns a new Reader whose buffer has the default size.

func NewReader(rd io.Reader) * Reader {

    return NewReaderSize(rd, defaultBufSize)

}

crypto.go

// New returns a new hash.Hash calculating the given hash function. New panics

// if the hash function is not linked into the binary.

func (h Hash) New() hash.Hash {

    if h > 0 && h < maxHash {

        f := hashes[h]

        if f != nil {

            return f()

        }

    }

    panic("crypto: requested hash function is unavailable")

}

Since each package acts as a namespace, every package can have their own version of New. In bufio.go multiple types can be created, so there is no standalone New function. Here you will find functions like NewReader and NewWriter.

Look back at sample.go. In our code the core type is Sample, so our New function returns a reference to a Sample type. Then we added a member function to display the name we provided in New.

Now let's look at the code for sub.go:

package subpkg

import (

    "fmt"

)

type Sub struct {

    Name string

}

func New(name string) (sub * Sub) {

    return &Sub{

        Name: name,

    }

}

func (sub * Sub) Print() {

    fmt.Println("Sub Name:", sub.Name)

}

The code is identical except we named our core type Sub. The package name matches the sub-directory name and New returns a reference to a Sub type.

Now that our packages are properly defined and coded we can use them.

Look at the code for main.go:

package main

import (

    "samplepkg"

    "samplepkg/subpkg"

)

func main() {

    sample := samplepkg.New("Test Sample Package")

    sample.Print()

    sub := subpkg.New("Test Sub Package")

    sub.Print()

}

Since our GOPATH points to the Workspace directory, in my case /Users/bill/Spaces/Test, our import references start from that point. Here we are referencing both packages based on the directory structure.

Next we call the New functions for each respective package and create variables of those core types.

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.23.25+AM.png)

Now build and run the program. You should see that an executable program was created called myprogram.

Once your program is ready for distribution you can run the install command.

![Screen Shot](../../../images/goinggo/Screen+Shot+2013-07-28+at+10.24.16+AM.png)

The install command will create the bin and pkg folders in your Workspace. Notice the final executable was placed under the bin directory.

The compiled packages were placed under the pkg directory. Within that directory a sub-directory that describes the target architecture is created and that mirrors the source directories.

These compiled packages exist so the go tool can avoid recompiling the source code unnecessarily.

The problem with that last statement, found in the "How to Write Go Code" post, is that these .a files are ignore by the Go tool when performing future builds of your code. Without the source code files you can't build your program. I have not found any documentation to really explain how these .a files can be used directly to build Go programs. If anyone can shed some light on this topic it would be greatly appreciated.

At the end of the day it is best to follow the conventions handled down from the Go designers. Looking at the source code they have written provides the best documentation for how to do things. Many of us are writing code for the community. If we all follow the same conventions we can be assured of compatibility and readability. When in doubt open finder to /usr/local/go/src/pkg and start digging.

As always I hope this helps you understand the Go programming language a little better.

Running Go Programs as a Background Process

Mon, 24 Jun 2013 00:00:00 +0000

I have been writing Windows services in C/C++ and then in C# since 1999. Now that I am writing server based software in Go for the Linux OS I am completely lost. What is even more frustrating, is that for the first time the OS I am developing on (Mac OSX) is not the operating system I will be deploying my code on. That will be for another blog post.

Reading XML Documents in Go

Mon, 17 Jun 2013 00:00:00 +0000

I was really surprised how easy it was to read an XML document using the encoding/xml package that comes with the standard library. The package works by defining structs that map the XML document. If you need more flexibility then use Gustavo Niemeyer’s xmlpath package (found here).

Here is the XML document we are going to read and de-serialize:

<straps>

    <strap key="CompanyName" value="NEWCO" />

    <strap key="UseEmail" value="true" />

</straps>

The first thing we need to do is define the structs we will use to map the document:

type XMLStrap struct {

    XMLName  xml.Name ̀ xml:"strap"̀ 

    Key      string   ̀ xml:"key,attr"̀ 

    Value    string   ̀ xml:"value,attr"̀ 

}

type XMLStraps struct {

    XMLName  xml.Name    ̀ xml:"straps"̀ 

    Straps   []XMLStrap ̀ xml:"strap"̀ 

}

There are two structs, one for the entire document (<straps>) and one for each individual child node (<strap>). If you look closely at the structs you may see something new. Each field has a tag associated with it. These tags are bound to each individual field. Go's reflect package allows you to access these tags.

These tag formats are specific to the decoding support inside the encoding/xml package. The tags map the nodes and attributes of the XML document to the struct.

The following code decodes the XML document and returns the array of strap nodes:

func ReadStraps(reader io.Reader) ([]XMLStrap, error) {

    var xmlStraps XMLStraps

    if err := xml.NewDecoder(reader).Decode(&xmlStraps); err != nil {

        return nil, err

    }

    return xmlStraps.Straps, nil

}

The function takes an io.Reader. We will be passing a os.File variable into this method. The function returns an array of pointers for each strap we read from the file.

First we create a XMLStraps variable and get its address. Next we create a decoder using the xml.NewDecoder method passing the io.Reader object. Then we call Decode which reads the file and de-serializes the file into the XMLStraps variable. Then we just return the array of strap values.

The following completes the sample code:

/*

straps.xml should be located in the default working directory

<straps>

    <strap key="CompanyName" value="NEWCO" />

    <strap key="UseEmail" value="true" />

</straps>

*/

package main

import (

    "encoding/xml"

    "fmt"

    "io"

    "os"

    "path/filepath"

)

type XMLStrap struct {

    XMLName  xml.Name ̀ xml:"strap"̀ 

    Key      string   ̀ xml:"key,attr"̀ 

    Value    string   ̀ xml:"value,attr"̀ 

}

type XMLStraps struct {

    XMLName  xml.Name    ̀ xml:"straps"̀ 

    Straps   []XMLStrap ̀ xml:"strap"̀ 

}

func ReadStraps(reader io.Reader) ([]XMLStrap, error) {

    var xmlStraps XMLStraps

    if err := xml.NewDecoder(reader).Decode(&xmlStraps); err != nil {

        return nil, err

    }

    return xmlStraps.Straps, nil

}

func main() {

    // Build the location of the straps.xml file

    // filepath.Abs appends the file name to the default working directly

    strapsFilePath, err := filepath.Abs("straps.xml")

    if err != nil {

        fmt.Println(err)

        os.Exit(1)

    }

    // Open the straps.xml file

    file, err := os.Open(strapsFilePath)

    if err != nil {

        fmt.Println(err)

        os.Exit(1)

    }

    defer file.Close()

    // Read the straps file

    xmlStraps, err := ReadStraps(file)

    if err != nil {

        fmt.Println(err)

        os.Exit(1)

    }

    // Display The first strap

    fmt.Printf("Key: %s  Value: %s", xmlStraps[0].Key, xmlStraps[0].Value)

}

I hope this sample gets you started with reading XML documents for your Go applications.

Send an email in Go with smtp.SendMail

Thu, 13 Jun 2013 00:00:00 +0000

I wanted to send an email from my TraceLog package when a critical exception occurred. Fortunately Go’s standard library has a package called smpt which can be found inside the net package. When you look at the documentation you are left wanting.

I spent 20 minutes researching how to use this package. After fighting through the parameters and bugs, I came up with this sample code:

Go's time.Duration Type Unravelled

Tue, 11 Jun 2013 00:00:00 +0000

I have been struggling with using the Time package that comes in the Go standard library. My struggles have come from two pieces of functionality. First, trying to capture the number of milliseconds between two different time periods. Second, comparing that duration in milliseconds against a pre-defined time span. It sounds like a no brainier but like I said, I have been struggling.

In the Time package there is a custom type called Duration and a set of helper constants:

Understanding Defer, Panic and Recover

Sat, 08 Jun 2013 00:00:00 +0000

I am building my TraceLog package and it is really important that the package logs any internal exceptions and prevents panics from shutting down the application. The TraceLog package must never be responsible for shutting down an application. I also have internal go routines that must never terminate until the application is shut down gracefully.

Understanding how to use Defer and Recover in your application can be a bit tricky at first, especially if you are used to using try/catch blocks. There is a pattern you can implement to provide that same type of try/catch protection in Go. Before I can show you this you need to learn how Defer, Panic and Recover work.

First you need to understand the intricacies of the keyword defer. Start with this piece of code:

package main

import (

  "fmt"

)

func main() {

  test()

}

func mimicError(key string) error {

  return fmt.Errorf("Mimic Error : %s", key)

}

func test() { 

  fmt.Println("Start Test")

  err := mimicError("1")

  defer func() {

    fmt.Println("Start Defer")

    if err != nil {

      fmt.Println("Defer Error:", err)

    }

  }()

  fmt.Println("End Test")

}

The MimicError function is a test function that will be used to simulate an error. It is following the Go convention of using the error type to return an indication if something went wrong.

In Go the error type is defined as an interface:

type error interface {

  Error() string

}

If you don't understand what a Go interface is at the moment then this might help for now. Any type that implements the Error() function implements this interface and can be used as a variable of this type. The MimicError function is using errors.New(string) to create an error type variable. The errors type can be found in the errors package.

Test function produces the following output:

Start Test

End Test

Start Defer

Defer Error : Mimic Error : 1

When you study the output you see that the Test function started and ended. Then right before the Test function terminated for good, the inline defer function was called. Two interesting things are happening here. First, the defer keyword is deferring the execution of the inline function until the Test function ends. Second, because Go supports closure, the err variable is accessible to the inline function and its message "Mimic Error : 1" is written to stdout.

You can define a defer function at any time inside your function. If that defer function requires state, as in this case with the err variable, then it must exist before the defer function is defined.

Now change the Test function a bit:

func test() {

  fmt.Println("Start Test")

  err := mimicError("1")

  defer func() {

    fmt.Println("Start Defer")

    if err != nil {

      fmt.Println("Defer Error:", err)

    }

  }()

  err = mimicError("2")

  fmt.Println("End Test")

}

This time the code is calling the MimicError function a second time after creating the inline defer function. Here is the output:

Start Test

End Test

Start Defer

Defer Error : Mimic Error : 2

The output is identical from the first test except for one change. This time the inline defer function wrote "Mimic Error : 2". It appears that the inline defer function has a reference to the err variable. So if the state of the err variable changes at any time before the inline defer function is called, you will see that value. To verify that the inline defer function is getting a reference to the err variable, change the code to write the address of the err variable in both the Test function and the inline defer function.

func test() {

  fmt.Println("Start Test")

  err := mimicError("1")

  fmt. Println("Err Addr:", &err)

  defer func() {

    fmt.Println("Start Defer")

    if err != nil {

      fmt.Println("Err Addr Defer:", &err)

      fmt.Println("Defer Error:", err)

    }

  }()

  err = mimicError("2")

  fmt.Println("End Test")

}

As you can see from the output below, the inline defer function has the same reference to the err variable. The address is the same inside of the Test function and inside of the inline defer function.

Start Test

Err Addr: 0x2101b3200

End Test

Start Defer

Err Addr Defer: 0x2101b3200

Defer Error : Mimic Error : 2

As long as the defer function is stated before the Test function terminates, the defer function will be executed. This is great, but what I want is the ability to always place the defer statement at the beginning of any function. This way the defer function is guaranteed to be called every time the function is executed and I don't have to over think where to place the defer statement. Occam's Razor applies here: "When you have two competing theories that make exactly the same predictions, the simpler one is the better". What I want is an easy pattern that can be duplicated without requiring any thought.

The only problem is that the err variable needs to be defined before the defer statement can be implemented. Fortunately Go allows return variables to have names.

Now change the entire program as follows:

package main

import (

  "fmt"

)

func main() {

  if err := test(); err != nil {

    fmt.Printf("Main Error: %v\n", err)

  }

}

func mimicError(key string) error {

  return fmt.Errorf("Mimic Error : %s", key)

}

func test() (err error) {

  defer func() {

    fmt.Println("Start Defer")

    if err != nil {

      fmt.Println("Defer Error:", err)

    }

  }()

  fmt.Println("Start Test")

  err = mimicError("1")

  fmt.Println("End Test")

  return err

}

The Test function now defines a return variable called err of type error. This is great because the err variable exists immediately and you can put the defer statement at the very beginning of the function. Also, the Test function now follows the Go convention and returns an error type back to the calling routine.

When you run the program you get the following output:

Start Test

End Test

Start Defer

Defer Error : Mimic Error : 1

Main Error: Mimic Error : 1

Now it is time to talk about the built-in function panic. When any Go function calls panic the normal flow of the application stops. The function that calls panic ends immediately and causes a chain reaction of panics up the call stack. All the functions in the same call stack will end, one after the next, like dominos falling down. Eventually the panic reaches the top of the call stack and the application crashes. One good thing is that any existing defer functions will be executed during this panic sequence and they have the ability to stop the crash.

Look at this new Test function that calls the built in panic function and recovers from the call:

func test() error {

  defer func() {

    fmt.Println("Start Panic Defer")

    if r := recover(); r != nil {

      fmt.Println("Defer Panic:", r)

    }

  }()

  fmt.Println("Start Test")

  panic("Mimic Panic")

}

Look closely at the new inline defer function:

  defer func() {

    fmt.Println("Start Panic Defer")

    if r := recover(); r != nil {

      fmt.Println("Defer Panic:", r)

    }

  }()

The inline defer function is now calling another built-in function recover. The recover function stops the chain reaction from going any farther up the call stack. It is like swiping a domino away so no more can fall down. The recover function can only be used inside of a defer function. This is because during the panic chain reaction only defer functions will be executed.

If the recover function is called and there is no panic occurring, the recover function will return nil. If there is a panic occurring, then the panic is stopped and the value given to the panic call will be returned.

This time the code is not calling the MimicError function but the built in panic function to simulate a panic. Look at the output from running the code:

Start Test

Start Panic Defer

Defer Panic : Mimic Panic

The inline defer function captures the panic, prints it to the screen and stops it dead in its tracks. Also notice that "End Test" is never displayed. The function terminated as soon as panic was called.

This is great but if there is an error I still want to display that as well. Something cool about Go and the defer keyword is that you can have more than one defer function stated at a time.

Change the Test function as follows:

func test() (err error) {

  defer func() {

    fmt.Println("Start Panic Defer")

    if r := recover(); r != nil {

      fmt.Println("Defer Panic :", r)

    }

  }()

  defer func() {

    fmt.Println("Start Defer")

    if err != nil {

      fmt.Println("Defer Error:", err)

    }

  }()

  fmt.Println("Start Test")

  err = mimicError("1")

  panic("Mimic Panic")

}

Now both inline defer functions have been incorporated into beginning of the Test function. First the inline defer function that that recovers from panics and then the inline defer function that prints errors. One thing to note is that Go will execute these inline defer functions in the opposite order that are defined (First In - Last Out).

Run the code and look at the output:

Start Test

Start Error Defer

Defer Error : Mimic Error : 1

Start Panic Defer

Defer Panic : Mimic Panic

Main Error: Mimic Error : 1

The Test function starts as expected and the call to panic halts the execution of the Test function. This causes the inline defer function that prints errors to get called first. Since the Test function called the MimicError function before the panic, the error is printed. Then the inline defer function that recovers from panics is called and the panic is recovered.

There is one problem with this code. The main function had no idea that a panic was averted. All the main function knows is that an error occurred thanks to the MimicError function call. This is not good. I want the main function to know about the error that caused the panic. That is really the error that must be reported.

In the inline defer function that handles the panic we need to assign the error that caused the panic to the err variable.

func test() (err error) {

  defer func() {

    fmt.Println("Start Panic Defer")

    if r := recover(); r != nil {

      fmt.Printf("Defer Panic : %v\n", r)

      err = fmt.Errorf("%v", r)

    }

  }()

  defer func() {

    fmt.Println("Start Defer")

    if err != nil {

      fmt.Println("Defer Error:", err)

    }

  }()

  fmt.Println("Start Test")

  err = mimicError("1")

  panic("Mimic Panic")

}

Now when you run the code you get the following output:

Start Test

Start Error Defer

Defer Error : Mimic Error : 1

Start Panic Defer

Defer Panic : Mimic Panic

Main Error: Mimic Panic

This time when main function reports the error that caused the panic.

Everything looks good but this code is not really scalable. Having two inline defer functions is cool but not practical. What I need is a single function that can handle both errors and panics.

Here is a revised version of the full program with a new function called _CatchPanic:

package main

import (

  "fmt"

)

func main() {

  if err := test(); err != nil {

    fmt.Println("Main Error:", err)

  }

}

func catchPanic(err error, functionName string) {

  if r := recover(); r != nil {

    fmt.Printf("%s : PANIC Defered : %v\n", functionName, r)

    if err != nil {

      err = fmt.Errorf("%v", r)

    }

  } else if err != nil {

    fmt.Printf("%s : ERROR : %v\n", functionName, err)

  }

}

func mimicError(key string) error {

  return fmt.Errorf("Mimic Error : %s", key)

}

func test() (err error) {

  defer catchPanic(err, "Test")

  fmt.Println("Start Test")

  err = mimicError("1")

  fmt.Println("End Test")

  return err

}

The new function _CatchPanic incorporates both the panic recover and error handling. This time instead of defining an inline defer function the code is using an external function for the defer statement.

In this first test with the new _CatchPanic defer function, we need to make sure we didn't break our error handling.

Run the code and look at the output:

Start Test

End Test

Main Error: Mimic Error : 1

Everything looks good. Now we need to test a panic.

func test() (err error) {

  defer catchPanic(err, "Test")

  fmt.Println("Start Test")

  err = mimicError("1")

  panic("Mimic Panic")

}

Run the code and look at the output:

Start Test

Test5 : PANIC Defered : Mimic Panic

Main Error: Mimic Error : 1

Houston we have a problem. Main was provided the error from the MimicError function call and not from the panic. What went wrong?

func catchPanic(err error, functionName string) {

  if r := recover(); r != nil {

    fmt.Printf("%s : PANIC Defered : %v\n", functionName, r)

    if err != nil {

      err = fmt.Errorf("%v", r)

    }

  } else if err != nil {

    fmt.Printf("%s : ERROR : %v\n", functionName, err)

  }

}

Because defer is now calling an external function the code lost all the goodness that came with inline functions and Closure.

Change the code to print the address of the err variable from inside the Test function and_CatchPanic defer function.

func _CatchPanic(err error, functionName string) {

  if r := recover(); r != nil {

    fmt.Printf("%s : PANIC Defered : %v\n", functionName, r)

    fmt.Println("Err Addr Defer:", &err)

    if err != nil {

      err = fmt.Errorf("%v", r)

    }

  } else if err != nil {

    fmt.Printf("%s : ERROR : %v\n", functionName, err)

  }

}

func test() (err error) {

  fmt.Println("Err Addr:", &err)

  defer _CatchPanic(err, "Test7")

  fmt.Printf("Start Test\n")

  err = mimicError("1")

  panic("Mimic Panic")

}

When you run the code you can see why main did not get the error from the panic.

Err Addr: 0x2101b31f0

Start Test

Test5 : PANIC Defered : Mimic Panic

Err Addr Defer: 0x2101b3270

Main Error: Mimic Error : 1

When the Test function passes the err variable to the catchPanic defer function it is passing the variable by value. In Go all arguments are passed by value. So the catchPanic defer function has its own copy of the err variable. Any changes to catchPanic's copy remains with catchPanic.

To fix the pass by value problem the code needs to pass the err variable by reference.

package main

import (

  "fmt"

)

func main() {

  if err := testFinal(); err != nil {

    fmt.Println("Main Error:", err)

  }

}

func catchPanic(err *error, functionName string) {

  if r := recover(); r != nil {

    fmt.Printf("%s : PANIC Defered : %v\n", functionName, r)

    if err != nil {

      *err = fmt.Errorf("%v", r)

    }

  } else if err != nil && *err != nil {

    fmt.Printf("%s : ERROR : %v\n", functionName, *err)

  }

}

func mimicError(key string) error {

  return fmt.Errorf("Mimic Error : %s", key)

}

func testFinal() (err error) {

  defer catchPanic(&err, "TestFinal")

  fmt.Println("Start Test")

  err = mimicError("1")

  panic("Mimic Panic")

}

Now run the code and look at the output:

Start Test

Test6 : PANIC Defered : Mimic Panic

Main Error: Mimic Panic

The main function now reports the error that occurred because of the panic.

If you want to capture a stack trace as well just make this change to catchPanic. Remember to import "runtime".

func catchPanic(err *error, functionName string) {

  if r := recover(); r != nil {

    fmt.Printf("%s : PANIC Defered : %v\n", functionName, r)

    // Capture the stack trace

    buf := make([]byte, 10000)

    runtime.Stack(buf, false)

    fmt.Printf("%s : Stack Trace : %s", functionName, string(buf))

    if err != nil {

      *err = fmt.Errorf("%v", r)

    }

  } else if err != nil && *err != nil {

    fmt.Printf("%s : ERROR : %v\n", functionName, *err)

  }

}

With this pattern you can implement Go routines that can handle errors and trap panic situations. In many cases these conditions just need to be logged or reported up the call stack to be handled gracefully. Having a single place to implement this type of code and a simple way to integrate it into each function will reduce errors and keep your code clean.

However I have learned it is best to only use this pattern to catch panics only. Leave the logging of errors to the application logic. If not then you may be logging the errors twice.

func catchPanic(err *error, functionName string) {

  if r := recover(); r != nil {

    fmt.Printf("%s : PANIC Defered : %v\n", functionName, r)

    // Capture the stack trace

    buf := make([]byte, 10000)

    runtime.Stack(buf, false)

    fmt.Printf("%s : Stack Trace : %s", functionName, string(buf))

    if err != nil {

      *err = fmt.Errorf("%v", r)

    }

  }

}

As always I hope this can help you with your Go programming.

Documenting Go Code With Godoc

Wed, 05 Jun 2013 00:00:00 +0000

As you know if you read my blog, I have been building a set of new utility packages so I can start developing an application server I need for a new project. I am brand new to Go and the Mac OS. Needless to say it has been one hell of an education over the past month. But I don’t miss Windows or C# at all.

I made some progress in my coding and wanted to build documentation for the code. I have been using the documentation viewer in LiteIDE and I was hoping to integrate my documentation in there was well. I was really surprised to see see that LiteIDE already had my packages listed inside of their integrated Godoc viewer. So it then begged the question. How is that working?

After some digging around I found this local HTML file. If you have LiteIDE installed you can copy the following url into your browser.

file:///Applications/LiteIDE.app/Contents/Resources/golangdoc/about.html

This is what it shows

Overview

The integrated Godoc viewer in LiteIDE provides an easy way to browse documentation generated by the godoc tool without leaving the editor. Documentation can be viewed for both the official Go language as well as custom packages. The remainder of this page describes ways to invoke the Godoc viewer.

Supported URL Schemes

It is possible to view documentation by directly entering a URL into the Godoc viewer's address bar. When doing this, you can specify what type of documentation you are looking for by prefixing the address with one of the following URL schemes:

find

Searches for packages with a specified string in their name. For example:

list

Lists all packages in a given directory. The main choices are "pkg" and "cmd", which can be found as links in the header of the page. For example:

list:pkg - displays the Golang packages
list:cmd - displays the Golang commands

pdoc

Views documentation for a specified package or command. For example:

file

Views a specified HTML, Markdown, or plain-text file. For example:

file:c:/go/doc/docs.html

Automatic Schemes

For the "file" and "pdoc" schemes, you do not need to type the scheme as part of the URL. For example:

[/doc/code.html](/broken-link)
[/src/pkg](/broken-link)
[/src/cmd](/broken-link)
[/pkg/fmt](/broken-link)
[/cmd/cgo](/broken-link)
[archive/zip](/broken-link)
[Go](/broken-link)

File Browser

You can open the Godoc viewer directly from the file browser by right clicking on a file or directory and selecting "View Godoc Here". The Godoc viewer will automatically open the package documentation for the chosen item.

When I clicked on my package ArdanStudios/threadpool from within the LiteIDE Godoc search tool it used the pdoc URL scheme, pdoc:ArdanStudios/threadpool.

I quickly reasoned that LiteIDE was using the GOROOT and GOPATH variables to find the documentation. There is only one problem, I haven't created any documentation yet.

So I looked around in both /usr/local/go and my own space to find the documentation files and there was nothing. So how the heck was this documentation being generated and published on the screen?

Then I found this document from the Go team:

http://golang.org/cmd/godoc/

The very first line states, "Godoc extracts and generates documentation for Go programs." Ok, so this program is being used by LiteIDE but how? Where are the files that Godoc is generating for all this documentation?

LOL, boy it is difficult coming from a Windows environment for the past 20 years.

After reading the documentation a bunch of times I opened up a Terminal session and ran the following command.

godoc /Users/bill/Spaces/GoPackages/src/ArdanStudios/threadpool

Suddenly the documentation appeared on my screen in text format. But I am seeing HTML inside of LiteIDE? I found the -html option.

godoc -html /Users/bill/Spaces/GoPackages/src/ArdanStudios/threadpool

Now I produced the same documentation I am seeing inside of LiteIDE. There are no extra files on my machine, LiteIDE is streaming the output of Godoc directly into the screen. Very smart way of doing things!!

So if I can see documentation for the standard Go packages, then the source code for those packages must be on my machine. After a bit of looking I found them in:

/usr/local/go/src/pkg

It seems they are located inside a folder called pkg under src. This is because the Go team likes to put source code for reusable libraries within a project under pkg. Not all developers follow that same convention and you have the freedom to choose. I personally don't follow that convention. Apparently the Godoc tool has no problems finding the source code files.

Godoc tool is always reading the source code files to produce the latest version of the documentation. So in LiteIDE when you update your documentation and save the code file, the Godoc tool will show the changes immediately.

Now the next problem I have, my documentation looks really bad. The documentation that I see from the standard library files looks much better. So how do I format my documentation properly inside the Go code files?

I found this document from the Go team:

http://golang.org/doc/articles/godoc_documenting_go_code.html

The introduction reads:

Godoc: documenting Go code

The Go project takes documentation seriously. Documentation is a huge part of making software accessible and maintainable. Of course it must be well-written and accurate, but it also must be easy to write and to maintain. Ideally, it should be coupled to the code itself so the documentation evolves along with the code. The easier it is for programmers to produce good documentation, the better for everyone.

To that end, we have developed the godoc documentation tool. This article describes godoc's approach to documentation, and explains how you can use our conventions and tools to write good documentation for your own projects.

Godoc parses Go source code - including comments - and produces documentation as HTML or plain text. The end result is documentation tightly coupled with the code it documents. For example, through godoc's web interface you can navigate from a function's documentation to its implementation with one click.

Coming from the C# world and using XML tags like <summary> for that past 10 years and having to remember to check the "produce XML documentation file" option, this was a dream. Oh yea, no extra documentation file.

However the rest of the page was lacking. I liked the way the documentation for fmt.Printf looked so I quickly found the go source files and studied what the programmer did. After a bit of playing I finally figured out the 3 basic rules you need to help the Godoc tool format the documentation cleanly.

Here is a sample of the documentation I have for my tracelog package:

![Screen Shot](../../../images/goinggo/Screen-Shot-2013-07-28-at-9.57.53-AM.png)

There are 3 elements in play when writing your documentation. You have header sections, standard text and highlighted text.

At the very top of your code file add the following using the // comment operation or something similar. Obviously you want to give yourself the credit for your work, LOL.

// Copyright 2022 Ardan Studios. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

Then add a block comment operator and we can start. Make sure the package code statement is exactly after the closing comment operator. There can not be no blank lines between the two.

Tabbing is very important. We are using two layers of tabbing. Keep these two layers of tabbing consistent.

/*
->TAB Package TraceLog implements a file based logging.
->TAB The programmer should feel free to tace log as much of the code.
->CRLF
->TAB New Parameters
->CRLF
->TAB The following is a list of parameters for creating a TraceLog:
->TAB baseFilePath: The base location to store all log directories.
->TAB machineName: The name of the machine or system. Information is used.
->TAB writeToStdout: Set to True if you want the system to also write.
->CRLF
->TAB TraceLog File Management
->CRLF
->TAB Every 2 minutes TraceLog will check each open file for two conditions:
->CRLF
->TAB 1. Has a message been written to the file within the last 2 minutes.
->TAB 2. Is the size of the file greater than 10 Meg.
*/
package tracelog

The first section of comments will show at the top of our documentation just below the Overview Section. Also the first sentence will appear in Godoc's package list.

Then we have a blank line and a string that will become a header as long as the next line is double spaced and has the same indentation.

The final component is the second tabbing indentation. This will cause that text to be highlighted with a grey background.

You may need to remove all of your existing documentation from your Go code file and throw it into a text editor. Then put it back in to make sure all the tabs and carriage returns are clean.

Using GoDoc.org

If you are building a public package you can use the GoDoc website to publish your documentation. Check out the GoDoc website:

[http://godoc.org/] (http://godoc.org/)

This website has been setup to read your code files and display all of your great documentation. Enter this url (github.com/goinggo/utilities/v1/workpool) into the search box and see the documentation that GoDoc produces for my workpool package:

![Screen Shot](../../../images/goinggo/Screen-Shot-2013-09-02-at-12.33.05-PM.png)

![Screen Shot](../../../images/goinggo/Screen-Shot-2013-09-02-at-12.33.23-PM.png)

You can see the same documentation that is being given to you locally is now published on the GoDoc website with your own reuseable url:

So how can you best use this url to provide people your documentation? When you create a repository for your package add a README.md file. This is a special "Markdown" file that supports standard text, html and a few special operators of its own. Github has its own extensions and you can find documentation about Markdown here:

https://help.github.com/articles/github-flavored-markdown

If you happened to come across my public workpool package in Github, you would see the following:

![Screen Shot](../../../images/goinggo/Screen-Shot-2013-09-02-at-12.45.10-PM.png)

There is my code file, license file and my readme Markdown file.

Here is a typical README Markdown file that I use:

# Workpool - Version 1.0.0

Copyright 2022 Ardan Studios. All rights reserved.<br />

Use of this source code is governed by a BSD-style license that can be found in the LICENSE handle.

Ardan Studios<br />

12973 SW 112 ST, Suite 153<br />

Miami, FL 33186<br />

bill@ardanstudios.com<br />

[Click To View Documentation](/broken-link)

View The Raw Version Here:
[https://raw.github.com/goinggo/utilities/master/v1/workpool/README.md](/broken-link)

Look at the Markdown link at the bottom of the file. This syntax creates a link to the documentation. The text in the hard brackets [], provides the anchor text for the link.

Since Github always display the Readme Markdown file to the user if one exists, this is what people see when they come to that Github page:

![Screen Shot](../../../images/goinggo/Screen-Shot-2013-09-02-at-12.50.23-PM.png)

Now people have access to the documentation I write on the web as well. I don't need to copy and paste the documentation into the Readme Markdown file, just provide a link. All the documentation is in one place and formatted cleanly and consistently.

As always, I hope this helps you in some small way and your documentation draws people to your work.

Installing Go, Gocode, GDB and LiteIDE

Sun, 02 Jun 2013 00:00:00 +0000

Check out my new installtion document:
https://www.ardanlabs.com/blog/2016/05/installing-go-and-your-workspace.html

I removed the sections about gocode and GDB. These are not necessary any longer. I also added links for more editors.

Windows
This is a great post by Wade Wegner for installing Go on your Windows Machine:
http://www.wadewegner.com/2014/12/easy-go-programming-setup-for-windows/

Mac OS X
The following instructions will guide you through installing Go on your Mac.

Step 1: Download Go

Open your favorite browser and go to the following website:
http://golang.org/dl/

![Screen Shot](../../../images/goinggo/Screen+Shot+2015-08-29+at+8.57.08+PM.png)

This will show you all the latest builds for the different operating systems. Darwin is the name for the Mac OS. Download the installer package for your OS version and architecture.

Once downloaded go to your Downloads folder in Finder and double click on the pkg file to start the installation. The installation will put Go in /usr/local/go. Once the installation is complete you will want to check the installation.

Note: Moving forward we will be using both Terminal and Finder. It helps to be able to see all the files in Finder. Finder by default will not show you everything that is on your hard drive.

To show all files in Finder:

Open a Terminal session. If you don't know where the Terminal program is go to your Applications folder in Finder and then Utilities. Click on Terminal and a bash shell command window will open.

Execute the following commands in Terminal.

defaults write com.apple.finder AppleShowAllFiles TRUE
killall Finder

The killall Finder will reload the Finder program and now you can see all the files.

Step 2: Check Installation

Open a Terminal session and type the following command

go version

You should see the following if everything installed correctly

go version go1.5 darwin/amd64

Now type the which command to verify the installation is in /usr/local/go

which go

You should see that Go can be found in /usr/local/go/bin

/usr/local/go/bin/go

Step 3: Set Your GOPATH

You need a single place to store and work on your Go projects. Create a folder called projects from inside your home folder:

cd $HOME

mkdir projects
cd projects
mkdir src
mkdir bin

Now set projects as your GOPATH. Open the .bash_profile file from the $HOME folder and add the following items to the end.

nano .bash_profile

export GOPATH="$HOME/projects"
export PATH=$PATH:$GOPATH/bin

Then exit the Terminal App and open a new Terminal App. Check that the Go environment now has your new GOPATH.

go env

You should see all the Go related environment variables including GOPATH. Here are some of them:

GOARCH="amd64"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/you/projects"
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"

Step 4: Get, Build and Install Basic Packages

These are Go packages you may want:

// gocode is used by many editors to provide intellisense
go get github.com/nsf/gocode

// goimports is something you should run when saving code to fill in import paths
go get golang.org/x/tools/cmd/goimports

// gorename is used by many editors to provide identifier rename support
go get golang.org/x/tools/cmd/gorename

// oracle is a tool that help with code navigation and search
go get golang.org/x/tools/cmd/oracle

// golint should be run after every build to check your code
go get github.com/golang/lint/golint

Step 5: Debugger

Using GDB is really not an option. If you would like to experiment with Delve, here is a link:
https://github.com/derekparker/delve

Step 6: Install Editor

Sublime
http://www.sublimetext.com/

DisposaBoy wrote this plugin for Sublime
https://github.com/DisposaBoy/GoSublime

Mark Wolfe wrote this post:
http://www.wolfe.id.au/2015/03/05/using-sublime-text-for-go-development/

VIM
http://www.vim.org/download.php

Victor Farazdagi wrote this post on installing Vim for Go:
http://farazdagi.com/blog/2015/vim-as-golang-ide/

Atom
https://atom.io/

Joe Fitzgerald wrote this plugin for Atom
https://github.com/joefitzgerald/go-plus

LiteIDE
http://sourceforge.net/projects/liteide/files/

Emacs
This is the configuration Guillaume Charmes uses:
https://github.com/creack/dotfiles

Helpful Links

Here are other links to web pages that will be very helpful:

http://golang.org/
https://github.com/golang/go/wiki
http://blog.golang.org/
http://www.youtube.com/user/gocoding
http://dave.cheney.net/
http://gophervids.appspot.com/

You must watch these videos on Go Concurrency Patterns

http://www.youtube.com/watch?v=QDDwwePbDtw
http://www.youtube.com/watch?v=f6kdp27TYZs

Thread Pooling in Go Programming

Fri, 31 May 2013 00:00:00 +0000

https://github.com/goinggo/work

Introduction

In my world of server development thread pooling has been the key to building robust code on the Microsoft stack. Microsoft has failed in .Net by giving each Process a single thread pool with thousands of threads and thinking they could manage the concurrency at runtime. Early on I realized this was never going to work. At least not for the servers I was developing.

When I was building servers in C/C++ using the Win32 API, I created a class that abstracted IOCP to give me thread pools I could post work into. This has always worked very well because I could define the number of threads in the pool and the concurrency level (the number of threads allowed to be active at any given time). I ported this code for all of my C# development. If you want to learn more about this, I wrote an article years ago (http://www.theukwebdesigncompany.com/articles/iocp-thread-pooling.php). Using IOCP gave me the performance and flexibility I needed. BTW, the .NET thread pool uses IOCP underneath.

The idea of the thread pool is fairly simple. Work comes into the server and needs to get processed. Most of this work is asynchronous in nature but it doesn’t have to be. Many times the work is coming off a socket or from an internal routine. The thread pool queues up the work and then a thread from the pool is assigned to perform the work. The work is processed in the order it was received. The pool provides a great pattern for performing work efficiently. Spawning a new thread everytime work needs to be processed can put heavy loads on the operating system and cause major performance problems.

So how is the thread pool performance tuned? You need to identify the number of threads each pool should contain to get the work done the quickest. When all the routines are busy processing work, new work stays queued. You want this because at some point having more routines processing work slow things down. This can be for a myriad of reasons such as, the number of cores you have in your machine to the ability of your database to handle requests. During testing you can find that happy number.

I always start with looking at how many cores I have and the type of work being processed. Does this work get blocked and for how long on average. On the Microsoft stack I found that three active threads per core seemed to yield the best performance for most tasks. I have no idea yet what the numbers will be in Go.

You can also create different thread pools for the different types of work the server will need to process. Because each thread pool can be configured, you can spend time performance tuning the server for maximum throughput. Having this type of command and control to maximize performance is crucial.

In Go we don’t create threads but routines. The routines function like multi-threaded functions but Go manages the actual use of OS level threading. To learn more about concurrency in Go check out this document: http://golang.org/doc/effective_go.html#concurrency.

The packages I have created are called workpool and jobpool. These use the channel and go routine constructs to implement pooling.

Workpool

This package creates a pool of go routines that are dedicated to processing work posted into the pool. A single Go routine is used to queue the work. The queue routine provides the safe queuing of work, keeps track of the amount of work in the queue and reports an error if the queue is full.

Posting work into the queue is a blocking call. This is so the caller can verify that the work is queued. Counts for the number of active worker routines are maintained.

Here is some sample code on how to use the workpool:

Why Go Programming

Fri, 31 May 2013 00:00:00 +0000

For the past 20 years I have been writing server based and application software on the Microsoft stack. First in C/C++ leveraging the Win32 API and then in C# when .Net first was released. Over the past few months I have realized that trying to build scalable code on the Microsoft stack is becoming impossible. Why, Technology and Cost!!

Let's start with the licensing. Luckily I was accepted into the Bizspark program. If I didn't have that I would be looking at thousands of dollars to just get access to the tooling I need. Real world applications require a database. If you want to use SQL Server Standard the licensing model is not only convoluted but outrageously expensive once you figure all that out. If I am building something for a client and they need something more than SQL Express I am doing them an injustice. Even using SQL Server in the cloud is no picnic on pricing. Then you have CPU and Memory requirements. I am sorry but trying to run a SQL Server and IIS on anything less than 4 Gig is impossible. Lastly, take this cost for one machine and try to build a scalable architecture around it and your costs are through the roof.

I don't think it is an accident that Facebook, Twitter and the like are running on linux based technologies. The linux based technologies are cheaper, faster and require much less metal. You can build larger systems and architectures for fractions of the cost. These are proven technologies that are driving the biggest websites today.

I have already moved away from SQL Server to MongoDB. MongoDB has been a real win in terms of development, devops and cost. It has been around since 2009, has a huge community of people supporting it and there are drivers for just about every programming language. In fact, I have been able to do more with MongoDB than I could with SQL Server. Oh yea, no cost !!

For web development I am now using Ruby on Rails. This was an easy transition from .Net MVC and I use RubyMine as my IDE. RubyMine is a great tool and it only costs $100. If you are building just web services then using Padrino is a great way to go.

Now the rub, building application servers on the linux stack. I didn't want to use C/C++, I was done with that. Someone suggested Ruby but I have never felt it was a great language for server development. I looked at Erlang but that wasn't right. So I continued to use my C# based Windows service I called the TaskServer.

My TaskServer is really cool. It is a Windows service framework that can load and unload DLL based packages at runtime. Each Plugin is loaded into its own application domain. I added facilities to abstract threading, internal and external plugin communication, trace logging and anything I could do to make Plugin development fast and easy. The TaskServer has served me well but it runs on the Microsoft stack. For my Ruby web apps to talk with it I use RabbitMQ. Eventually I was going to build a Gem that implemented the Socket protocol so I could talk to the server directly.

Then someone told me to look at Go Programming. After reading and watching a few videos I was blown away. This is what I have been searching for. A programming language tailored to server development that would run on the linux stack. I have been able to take all of my C/C++/C# experience and quickly learn the language. Obviously the goroutine/channel construct is quite different but once you get your head wrapped around it you realize how much easier it makes concurrent programming.

As with any new language my first step is to port my utility classes and I have already begun that work. I have finished my ThreadPool package and now I am working on my TraceLog package. Once that is done I am moving to MongoDB and then will begin to build a server that will manage large amounts of data imports for a project I am working on, Details soon to come.

This is the beginning of my journey in Go Programming and never looking back at the Microsoft stack again. Sorry VMWare but I hope I never need to upgrade you again.

Advanced Kubernetes and Docker Training for Teams & Individuals - Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Control Plane
Kubectl
Service Discovery
Kustomize
Capacity Management
Pod Security
Helm
Stateful Applications
& more

Advanced Kubernetes Training

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

This course covers advanced topics like security, capacity planning (including autoscaling), Kubernetes API extensions points (including operators and CRDs), and presents a number of popular operators. It is a great addition to the “Kubernetes Bootstrap”, but can also be taken independently by folks who have a bit of Kubernetes experience and are familiar with key concepts like Pods, Deployments, Services, working with YAML, and want to take their Kubernetes mastery to the next level.

Advanced Rust Training for Teams & Individuals - Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

The Rust Ecosystem
Data Management
Fearless Concurrency
Traits, and Generics
Rust as a Service
More Integrations
& more

Custom Practical Go Training and Boot Camps

Mon, 01 Jan 0001 00:00:00 +0000

Course Description:

Our custom hands-on Go training contains several modules that you can choose from to create a syllabus tailored to your needs. Feel free to reach out if you have a need that is not converted by the following topics - we have a lot of experience with Go and development at large.

These classes are led by Miki Tebeka. Miki has more than 25 years of experience developing software for clients, with 11 of those years writing code in Go. Miki is the author of several books, a LinkedIn Learning author and a very busy teacher. Miki teaching style is “hands on” - you’re going to write a lot of coding during these sessions.

Deep Dive Into Kubernetes Networking with CNI

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course is for devs, ops, or even architects, who want to dive deeper into Kubernetes and learn concepts around the Container Network Interface (CNI). We’ll discuss CNI plugins and how pods communicate together on Kubernetes. As an example, we will change the CNI plugin on a live Kubernetes cluster.

This course is part of the Intensive Docker & Kubernetes track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

Go, Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 100 hours of material

Go, Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 100 hours of material

Go, Docker & K8s Performance Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 100 hours of material

Go, Docker & K8s Performance Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 100 hours of material

Go, Rust, Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 214 hours of material

Go, Rust, Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 214 hours of material

Go, Rust, Docker & K8s Mastery Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 214 hours of material

Go, Rust, Docker & K8s Mastery Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Go, Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 214 hours of material

Intensive Docker

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course will help you establish foundational concepts of Docker & containers. You’ll complete labs and assignments that will help you become productive at designing, packaging, deploying and operating modern applications.

This course is part of the Intensive Docker & Kubernetes track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Intensive Docker & K8s Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Fundamentals
Advanced Engineering
Cloud Native Environments
High Performance Concepts

Intensive Docker & K8s Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Fundamentals
Advanced Engineering
Cloud Native Environments
High Performance Concepts

Intensive Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Fundamentals
Advanced Engineering
Cloud Native Environments
High Performance Concepts
Networking with CNI

Intensive Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Fundamentals
Advanced Engineering
Cloud Native Environments
High Performance Concepts
Networking with CNI

Intensive Docker Training

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

This course will enable you to leverage Docker to build, ship, and run traditional and microservices applications in containers, both in local development and production environments. You will learn how to start/stop/manage containers with the Docker CLI, build optimized container images with Dockerfiles, multi-stage builds, advanced BuildKit features, and more. You will also see how to set up repeatable development stacks of containers with Compose, and how to take them to production.

Intensive Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This is a course for any engineer who needs to deploy, scale, and operate applications in “Cloud Native” environments. You’ll complete labs and assignments that will help you become productive with Kubernetes.

This course is part of the Intensive Docker & Kubernetes track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Intensive Kubernetes: Advanced Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course is for devs, ops, and architects, who have already started working with Kubernetes and want to learn more advanced concepts. Students will complete numerous labs and exercises.

This course is part of the Intensive Docker & Kubernetes track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Requirements:

Understanding of Pods, Deployments, Services.
How to deploy an app and expose it on a Kubernetes cluster.
Basic notions of what are labels, selectors, YAML manifests.
Basic shell commands (navigate directories, set environment variables…).
Using a text editor like vi, nano, or similar.

Course Outline

Day 1: security focus:

Securing access with network policies
Managing permissions with RBAC
Managing secrets with Sealed Secrets
Pod Security Settings and Admission

Day 2: resource management:

Understanding requests and limits
Compressible vs incompressible resources
Pod eviction and kubelet under pressure
Horizontal, vertical, and cluster- autoscaling
Priorities and preemption
Scaling with custom metrics

Day 3: extending the Kubernetes API:

Overview of the different extension mechanisms
Custom Resource Definitions
Dynamic admission control with webhooks
The aggregation layer
Operator concepts

Day 4: operator and controller implementation:

Operator design techniques
Studying operators (e.g. Kyverno, cert-manager)
The kubebuilder framework
Writing a small operator with kubebuilder
Events, finalizers, owners and dependents

Day 5: stateful applications:

Stateful Sets, PV, PVC, Storage Classes
Use-case: running a Consul cluster
Dynamic storage provisioning
Use-case: stateful failover of an active database

Introduction to KinD

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

Learn how to use KinD (Kubernetes-in-Docker) to get a local Kubernetes cluster across Linux, Mac, and Windows environments.

KinD is primarily used for testing Kubernetes but can also be utilized for local development or CI.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Requirements:

No experience with containers necessary.

Course Outline

Introduction to KinD:

1.0 - Using KinD for local Kubernetes development
1.2 - 5 options for running Kubernetes locally
1.3 - Deploying a KinD cluster with multiple nodes
1.4 - Utilizing docker exec/docker stats with KinD

Kubernetes Bootstrap Training

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

Whether you are completely new to Kubernetes or are already dabbling in it, this course will give you a solid understanding of Kubernetes concepts and architecture without skimming on the details. You’ll learn how to deploy, update, scale, and troubleshoot applications on Kubernetes clusters, both on-premises or on-cloud.

Course Outline

Kubernetes concepts and architecture
Pods and pod controllers: Replica Sets, Deployments, Jobs, Daemon Sets…
Exposing pods with services and ingresses
Annotations, labels, and selectors
Managing logs and troubleshooting common issues
Organizing resources with Namespaces
Writing, generating, and deploying with YAML manifests
Working with local development clusters as well as remote ones
Scaling and failover
Rolling updates, healthchecks
Advanced patterns like blue/green deployments
Understanding volumes and using them for data sharing
Managing application configuration with ConfigMaps, downward API, and more

Kubernetes Controller Training

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

Understanding and extending the Kubernetes control plane. The Kubernetes control plane is arguably the most successful orchestration engine ever built. This course will dive into how it works and the concepts behind why it’s so effective. We will demonstrate how anyone can extend the control plane with additional functionality by building a production-grade controller through a series of exercises. All concepts are communicated through live diagrams and reinforced through joint coding sessions. Most teams find this material useful to kickstart company-projects after the course. By the end of this course, you will have the tools necessary to detangle the set of pipelines, scripts, lambda functions, and cron jobs that orchestrate software at most companies.

Managing AWS Resources with Terraform

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

Get started with the basics of Infrastructure-as-code and see a live demo on how to spin up a web server with Terraform.

This course is part of the Intensive Docker & Kubernetes track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Requirements:

Students should be comfortable using the command line.
Some experience with Kubernetes recommended.

Course Outline

Managing AWS With Terraform:

1.1.1 - What is Infrastructure as Code (IaC)
1.1.2 - Benefits of Terraform
1.1.3 - Getting Started: Basic Terraform Commands
1.1.4 - Example Terraform Configuration File
1.1.5 - Intro to Terraform .tf Files
1.1.6 - Creating AWS resources with Terraform
1.1.7 - Enabling SSH connectivity with Terraform
1.1.8 - Turning our server into a web server
1.1.9 - Making the server production ready
1.2.0 - Creating an S3 bucket with Terraform

Performance Go for Developers

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

The Performance Go for Developers training class is aimed for experienced Go developers who’d like to extend their knowledge.

Course Outline

Writing custom errors and the case of the nil error
Stronger abstractions with small interfaces and using type assertions
Customizing behavior by implementing interfaces in the standard library
Reducing code size by using generics
Going faster with CPU optimization
Reducing memory usage
Writing middleware for common tasks
Advanced JSON serialization
Streaming responses for large dynamic data
Securing your server
Utilizing workspaces to developing two dependant modules (such as application & logging library)
Using build tags for platform dependent code
Task automation with “go generate”, code that writes code
Avoiding mocks by spinning services

Practical Go for Developers

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

The Practical Go for Developers training class helps experienced engineers get effective with the Go programming language and learn by writing code to solve common programming tasks.

Course Outline

Strings & formatted output
Calling REST APIs
Working with files
Interfaces & Panics
Sorting
Catching panics
Processing text
Concurrency
Distributing work
Timeouts & cancellation
Project Engineering
Testing your code
Structuring your code
Writing an HTTP server
Adding metrics & logging
Configuration patterns

Practical Go Foundations

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course will help you become familiar with the basic concepts of the Go programming language. Through lectures and coding exercises, you’ll learn about the core concepts needed to become a productive Go developer.

This course is part of the Ultimate Go Track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Practical Go Services

Mon, 01 Jan 0001 00:00:00 +0000

Course Description:

This workshop is for Go developers who need to improve writing services and learn industry best practices. During this workshop we’ll develop a web service and implement the topics we discuss.

Course Outline

Writing a Web Service
Storage & Health
Testing & Performance
Securing Your Application

Rust as a Service

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course teaches integrating Rust into service-oriented architectures, covering REST server development, data handling, error management, and modularization. You’ll learn tracing, automated OpenAPI documentation, configuration for different environments, and alternative connectivity mechanisms like gRPC and WebSockets. Explore deploying a test service in a containerized environment, along with insights into service design, deployment strategies, and scaling. This prepares you for deploying high-performance Rust services in enterprise environments.

This course is part of the Ultimate Rust track. Not sold separately.

Rust as a Service

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

Course Outline

Introduction:

1.0 - Introduction

REST Service:

2.1 - Minimal HTTP Server
2.2 - Service Stack
2.3 - Extractors
2.4 - Add a Simple Tower Layer (State)
2.5 - Add a Simple Tower Layer (Mutable State)
2.6 - Multiple States - Extension Layers
2.7 - Quick Recap on State and Layers
2.8 - Nesting Multiple Routers
2.9 - Nested Routers with State
2.10 - Calling Other Services
2.11 - Returning Status Codes
2.12 - Using IntoResponse
2.13 - Error Handling with IntoResponse
2.14 - Quick Recap on Nesting, Making Calls and Responses
2.15 - Serving Static Content with Tower
2.16 - Simple Header-Based Authentication
2.17 - Simple Header-Based Auth with Middleware
2.18 - Middleware Auth with Injection
2.19 - Selectively Applying Layers
2.20 - Router Layers
2.21 - Layer Recap

Tracing:

3.1 - Minimal Example
3.2 - Logging Axum/Tower
3.3 - Timing Spans
3.4 - Axum Spans
3.5 - Logging to a File
3.6 - Structured Logging to JSON
3.7 - OpenTelemetry

OpenAPI Documentation:

4.1 - OpenAPI Documentation

Handling Service Configuration:

5.1 - Environment Variables with .env
5.2 - The Config Crate - Basics
5.3 - Loading Config via HTTP
5.4 - CLI configuration with Clap
5.5 - Recap

Handling Service Configuration:

6.1 - Hello Tonic - Protocol Definition
6.2 - Hello Tonic - Project Definition and Build
6.3 - Hello Tonic - The Server
6.4 - Hello Tonic - The Client
6.5 - gRPC Streaming
6.6 - gRPC Streaming - Protocol Definition
6.7 - gRPC Streaming - The Server
6.8 - gRPC Streaming - The Client
6.9 - Recap So Far
6.10 - Authentication
6.11 - Tracing
6.12 - When to use gRPC

Web Sockets:

7.1 - Minimal Echo Server
7.2 - A native WS client
7.3 - JSON

Service Deployment:

8.1 - Test Service
8.2 - Native Host Deployment
8.3 - Docker Deployment

Service Design:

9.1 - Understanding Your Company Architecture
9.2 - Designing Individual Services
9.3 - Combining Services into a Modular Monolith
9.4 - Service Exposure
9.5 - Scaling Out

Wrap Up:

10.1 - Wrap Up

Rust From C/C++

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course bridges the transition from C and C++ to Rust by highlighting their similarities and differences. Exploring Rust’s native compilation, memory management, and advanced features, you’ll compare and contrast key concepts like types, control flow, and data structures in this class. Delve into Rust’s strengths in memory safety and concurrency, alongside its unified tool, Cargo, streamlining development. Additionally, learn how Rust facilitates interoperability with C and C++ through Foreign Function Interface (FFI).

Rust From C/C++

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

Rust, Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 75 hours of material

Rust, Docker & K8s Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 75 hours of material

Rust, Docker & K8s Performance Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 75 hours of material

Rust, Docker & K8s Performance Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Includes All Rust, Docker & K8’s Modules
Self-Paced learning to fit your schedule
Learn everything you need to know with over 75 hours of material

Ultimate Debugging

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course is designed for developers who want to become proficient debugging Go software using the Delve debugger. Whether you’ve never used a debugger in your life or are a Delve expert, everybody will walk away with new information that can be used in their day to day development workflow.

The course will initially focus on Delve, which is the de facto Go debugger. We will start with the basics and move into more advanced use cases over the 5 days. On the last day of class, we will dig into new tools and cover profiling / perf tools, how to use them effectively, and how to interpret the data for root cause analysis.

Ultimate Go | Golang Training for Teams & Individuals - Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Concurrency
JSON Web Tokens
Polymorphism
Security
gRPC
REST APIs
Data Semantics
Web Services
Decoupling
& more

Ultimate Go Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Design Philosophies
Syntax to Tooling
Microservices
Concurrency
Debugging

Ultimate Go Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Design Philosophies
Syntax to Tooling
Microservices
Concurrency
Debugging

Ultimate Go Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Design Philosophies
Syntax to Tooling
Microservices
Concurrency
Debugging

Ultimate Go Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Design Philosophies
Syntax to Tooling
Microservices
Concurrency
Debugging

Ultimate Go: Advanced Engineering

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

From the beginning, you will pair program with the instructor, walking through the design philosophies and guidelines used to engineer the code. Throughout the class, you will learn more about Go and the advanced engineering features of the language.

This course is part of the Ultimate Go Track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Ultimate Go: Advanced Engineering

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

The Ultimate Go Advanced Engineering class has been designed over the past year and will teach advanced Go concepts by building a reference implementation of a blockchain in Go! The goal of this class is to share how to code complex engineering tasks required to build a blockchain technology.

From the beginning, you will pair-program with the instructor, walking through the design philosophies and guidelines used to engineer the code. Throughout the class, you will learn more about Go and the advanced engineering features of the language.

Ultimate Go: Language Guide

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

Learn to write better, more idiomatic and performant code in Go with a focus on micro-level engineering decisions. The course begins with a focus on Go internals that are critical to understanding the core tradeoffs on readability, simplicity and performance.

You will learn about data semantics, guidelines, mechanical sympathy, data oriented design, package oriented design, and how to structure your Go projects for the long term.

This course is part of the Ultimate Go Track. Not sold separately.

Ultimate Go: Language Guide

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

The Ultimate Go: Language Guide has been designed over the past 8 years and goes beyond just being a Go language class. There will be very little time spent on specific Go syntax. Our time will be spent learning how to read and comprehend Go code with a big focus on “if performance matters” then these things matter.

Course Outline

Data-Oriented Design
Memory Semantics and Allocations
Mechanical Sympathy with Hardware and Runtime
Data Semantics
Polymorphism with Interfaces
Decoupling Mechanics / Semantics
Compositional Design
Scheduler Semantics
Goroutines
Data Races
Synchronization / Orchestration
CPU and Memory Profiling
Tracing Programs
Micro/Macro level profiling

Ultimate Go: Notebook

Mon, 01 Jan 0001 00:00:00 +0000

About This Book

This notebook has been written and designed to provide a reference to everything covered in our Ultimate Go class. If you have taken the class before, this notebook will be invaluable for reminders on the content. If you have never taken the class, there is still tremendous value in this book. It covers more advanced topics not found in other books today.

Guidelines, design philosophy, white-boarding, and notes shared from the Ultimate Go training.
Learn advanced concepts in Go from types to profiling.
Get started with generic functions and types.

View the full details

Ultimate Go: Software Design with Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

From the beginning of the course, you will pair-program with your instructor Bill Kennedy as he walks you through the design philosophies, architectural decisions, and best practices as they apply to engineering a production-ready Go service.

With each new feature that is added to the service, you will learn how to deploy and manage the Kubernetes environment used to run the service. Throughout the class, the code being worked on is pushed to a repository for personal access and review.

Ultimate Go: Software Design with Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

Ultimate Rust 2: Intermediate Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This is a hands-on course! Not only are there targeted exercises for each topic discussed, there are also a series of project scenarios that walk you through using what you’ve learned to put together playable game prototypes that work on macOS, Linux, and Windows.

These projects will make use of Rusty Engine, a game engine developed specifically for this course to keep game engine concepts to a minimum so you can focus on using exactly what you learned in this course.

Ultimate Rust Best Practices

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course covers many best practices to help you with integrate Rust into your workflow, and let Rust’s tooling work for you. It includes formatting, linting, dependencies, vulnerabilities, code style and general development advice.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Course Outline

Section 1: Tooling

1.0 - Introduction - Formatting
1.1 - Clippy (the linter)
1.2 - Documentation
1.3 - Understanding Dependencies
1.4 - Managing Your Own Dependencies
1.5 - Checking for Vulnerabilities
1.6 - Check for Outdated Dependencies
1.7 - Denying Dependencies by Licensing

Section 2 : Code Best Practices

2.0 - Favor Iterators
2.1 - Minimize Cloning (we have an alternative name as well Cloning Can Be a Code Smell)
2.2 - Don’t Emulate OOP
2.3 - Favor Small Functions
2.4 - Clever Code
2.5 - Floating Point Numbers
2.6 - Platform & Feature Specific Code

Section 3 : General Best Practices

3.0 - TANSTAAFL (There Ain’t No Such Thing As A Free Lunch)
3.1 - YAGNI : You Ain’t Gonna Need It
3.2 - Domain Boundaries
3.3 - Taming Compile Times

Ultimate Rust Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started with Rust
Fearless System Thread Concurrency
Async/Await Concurrency
Memory & Resource Management
Build a Network Service

Ultimate Rust Bundle

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started with Rust
Fearless System Thread Concurrency
Async/Await Concurrency
Memory & Resource Management
Build a Network Service

Ultimate Rust Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started with Rust
Fearless System Thread Concurrency
Async/Await Concurrency
Memory & Resource Management
Build a Network Service

Ultimate Rust Bundle | Ardan Labs

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started with Rust
Fearless System Thread Concurrency
Async/Await Concurrency
Memory & Resource Management
Build a Network Service

Ultimate Rust Foundations

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

Ultimate Rust: This course covers many best practices to help you with integrate Rust into your workflow, and let Rust’s tooling work for you. It includes formatting, linting, dependencies, vulnerabilities, code style and general development advice..

This course is part of the Ultimate Rust track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Ultimate Rust: Crash Course

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

What you’ll learn:

Rust language basics, tooling, and ecosystem
Fundamentals such as variables, scope, functions, modules, scalar & compound types, control flow, strings, structs, traits, enums, and more.
Fun tips & tricks for using Rust and systems programming in general
Concepts essential to understanding Rust such as ownership, references & borrowing, the memory model, and why Rust focuses on safety, concurrency, and speed.

Requirements:

A basic understanding of general programming concepts
Familiarity with other programming languages such as Python, C, etc. is helpful, but not required.
Rust installed and ready to use - you’ll learn more by doing the exercises!

Who this course is for:

Aspiring systems programmers, software developers, engineers, wizards, warriors, and hobbits
Any developer who needs to run code fast, efficiently, securely, under tight restraints, or with a minimum of bugs
A desire to begin using Rust

Course Outline

Module 1: Introduction

Lesson 1.1 - Introduction
Lesson 1.2 - Exercises Overview
Lesson 1.3 - Installation
Lesson 1.4 - Editor/IDE

Module 2: Fundamentals

Lesson 2.1 - Cargo
Lesson 2.2 - Variables
Lesson 2.3 - Scope
Lesson 2.4 - Memory Safety
Lesson 2.5 - Exercise A - Variables
Lesson 2.6 - Functions
Lesson 2.7 - Exercise B - Functions
Lesson 2.8 - Module System
Lesson 2.9 - Exercise C - Module System

Module 3: Primitive Types & Control Flow

Lesson 3.1 - Scalar Types
Lesson 3.2 - Compound Types
Lesson 3.3 - Exercise D - Simple Types
Lesson 3.4 - Control Flow
Lesson 3.5 - Exercise E - Control Flow
Lesson 3.6 - Strings
Lesson 3.7 - String Literals
Lesson 3.8 - Exercise F - Strings

Module 4: The Heart of Rust

Lesson 4.1 - Ownership
Lesson 4.2 - References & Borrowing
Lesson 4.3 - Exercise G - Ownership & References

Module 5: The Meat of Rust

Lesson 5.1 - Structs
Lesson 5.2 - Exercise H - Structs
Lesson 5.3 - Traits
Lesson 5.4 - Exercise I - Traits
Lesson 5.5 - Collections
Lesson 5.6 - Exercise J - Collections
Lesson 5.7 - Enums
Lesson 5.8 - Exercise K - Enums

Ultimate Rust: Foundations

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

The Ultimate Rust Foundations class starts with an introduction to Rust, and why you benefit from using – both as a foundation, and within your existing ecosystem.

Course Outline

Rust includes Cargo, a swiss-army knife tool that can:
Using Rust without Cargo
Rust’s Safety Guarantees
“Hello World” application – the foundation of most languages.
Arrays, vectors and slices.
Strict types and strong typing.
Serializing and De-Serializing data.
Iterative Programming – using iterators to combine operations.
Working with Strings
Protection from data races
Easy multi-threaded concurrency with Rayon
Ultimate control with raw threads
High-performance Input/Output with Asynchronous design and Tokio.
Build a CPU bound program with Rayon, and use all of your CPU power.
Manage raw-threads and shared data.
Spin up a simple server in 10 minutes.
The classic Object-Oriented Programming example of different objects printing different text – but with traits.
Designing a simple generic function with ToString.
Stacked traits – traits that depend upon one another, combined with generic programming.

Ultimate Rust: Foundations - Next Steps

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

This course offers the next steps not covered in Ultimate Rust: Foundations that expands on and optimizes what you’ve already built.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.

Course Outline

Ultimate Rust: Foundations - Next Steps

1.0 - Bi-Directional Communication
1.2 - Sending Commands & Prevent Unbounded Growth
1.3 - Giving the Collector a Diet
1.4 - Giving the Collector a Diet (Cont)
1.5 - Optimizing File Reading & Adapting Files In Flight

Ultimate Rust: Integrating Pipelines

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

Rust doesn’t have to be an all or nothing proposition. Rust is very good at sliding into an existing platform, allowing you to focus on optimizing – both for speed and safety – the parts that matter without throwing away all of your existing hard work.

Course Outline

Introducing FFI, the Foreign Function Interface.
Consuming a C library from Rust:
From C to Rust:
Calling Go from Rust
Building a Rust FFI Crate
Consume the Rust crate from C
Consume the Rust crate from Go
Managing Your Build Process – Practical Observations
Porting Parts to Rust
Build an HTTP authentication service with Actix in under 200 lines of Rust.
Build an efficient, binary protocol server with Tokio.
Combining asynchronous high-performance I/O with CPU-bound computation.
Event tracing
Use a Unikernel to host a Rust service as a single-purpose virtual machine.
Use WebAssembly (WASM) to host high-performance code in the browser.
Build without a standard library for embedded targets

Ultimate Rust: Optimizing Rust & Debugging

Mon, 01 Jan 0001 00:00:00 +0000

Course Description

The Rust compiler catches many bugs for you that are common in other languages. Use the tools to avoid bugs and use Rust’s error system to pass meaningful errors and handle them appropriately.

Optimizing Rust

Measure First: the importance of knowing what to optimize
Use Rust’s built-in benchmark system to test component performance.
Use a Profiler to find hotspots.
When NOT to optimize
Optimize the Algorithm First
Optimization levels, Safety, Link Time Optimization.
Understanding your cache: When Optimizing for Size is faster than Optimizing for Speed.
Using unsafe safely.
Profile-Guided Optimization.
Compiler hints and inlining.
Tracing distributed components
Is your microservice too micro?
Blocking Operations
Waiting for Databases, File and Network I/O
Fast, Efficient Logging

Debugging Rust

Using your tools
Debugging with println statements.
Faster printing: tracing with tokio, and fast logging.
Types of bug
Attaching a debugger
Unit testing to avoid bugs in the first place.
Use Rust’s error system to pass meaningful errors—-and handle them appropriately.
Avoiding Bugs with the Rust Type System.
Strong Types, compile-time checking.
Strong Types, run-time checking.
Floating point errors.
Unicode errors.
Type-conversion errors.
Always test in debug mode to catch overflow and out-of-bounds errors.
Memory Issues
Great Documentation with Rust.
Integration testing with library examples.

Writing Secure Go Code

Mon, 01 Jan 0001 00:00:00 +0000

Course Introduction

Follow along with the instructor as he overviews common vulnerabilities in a live Go application. You’ll learn about tools and strategies to help you identify security vulnerabilities and how to think about security when it comes to your Golang application.

This course is part of the Ultimate Go Track. Not sold separately.

Note: All of our bundles are for a one-year subscription.

At the end of the subscription period, your membership does not automatically renew.