Introduction If you can write a for-loop, you can do statistics. - Jake Vanderplas A lot of developers shy away from problems which involve statistics or probability. Which is shameful since in today’s data-rich environment, you can gain a lot of insights from data. In this blog post, I’ll show you how to write a simulation tool which requires no knowledge in statistics or probability. Simulations are easy to write and can be a very effective tool in research.
Continue reading

Introduction I prefer to use relational (SQL) databases in general since they provide several features that are very useful when working with data. SQLite is a great choice since the database is a single file, which makes it easier to share data. Even though it’s a single file, SQLite can handle up to 281 terabytes of data. SQLite also comes with a command line client called sqlite3 which is great for quick prototyping.
Continue reading

Introduction Every single company I’ve worked at and talked to has the same problem without a single exception so far - poor data quality, especially tracking data. Either there’s incomplete data, missing tracking data, duplicative tracking data. - DJ Patil I spend a lot of my time digging into data at various companies. Most of the time I’m surprised by what I see and so are the engineers and analysts that work at these companies.
Continue reading

Series Index Generics Part 01: Basic Syntax Generics Part 02: Underlying Types Generics Part 03: Struct Types and Data Semantics Introduction In the previous post, I showed you how to declare a user-defined type, based on an underlying type. I did this through the progression of writing different versions of the same type using concrete types, the empty interface and then finally, generics. I also provided information on how the compiler was limited in its ability to infer the substitution for the generic type during zero value construction, but it could with initialized construction.
Continue reading

Series Index Python and Go: Part I - gRPC Python and Go: Part II - Extending Python With Go Python and Go: Part III - Packaging Python Code Python and Go: Part IV - Using Python in Memory Introduction In a previous post we used gRPC to call Python code from Go. gRPC is a great framework, but there is a performance cost to it. Every function call needs to marshal the arguments using protobuf, make a network call over HTTP/2, and then un-marshal the result using protobuf.
Continue reading

Changes The draft is a living document which means these posts will need to change over time. This section documents when changes have taken place to this post. 21/08/20 : Moving forward with the generics design draft Series Index Generics Part 01: Basic Syntax Generics Part 02: Underlying Types Generics Part 03: Struct Types and Data Semantics Introduction In the previous post, I showed you how to write a generic function in Go using the proposed syntax from the generics draft document.
Continue reading

Series Index Python and Go: Part I - gRPC Python and Go: Part II - Extending Python With Go Python and Go: Part III - Packaging Python Code Python and Go: Part IV - Using Python in Memory Introduction In the previous post we compiled Go code to a shared library and used it from the Python interactive shell. In this post we’re going to finish the development process by writing a Python module that hides the low level details of working with a shared library and then package this code as a Python package.
Continue reading

Changes The draft is a living document which means these posts will need to change over time. This section documents when changes have taken place to this post. 21/08/20 : Moving forward with the generics design draft Series Index Generics Part 01: Basic Syntax Generics Part 02: Underlying Types Generics Part 03: Struct Types and Data Semantics Introduction In this series of posts about generics in Go, I will present code and teach to the different aspects of the current generics draft.
Continue reading

Series Index Python and Go: Part I - gRPC Python and Go: Part II - Extending Python With Go Python and Go: Part III - Packaging Python Code Python and Go: Part IV - Using Python in Memory Introduction In the previous post we saw how a Go service can call a Python service using gRPC. Using gRPC to connect a Go and Python program together can be a great choice, but there’s a complexity price that goes with it.
Continue reading

Introduction In this paper written by Manish Jain (the founder of Dgraph) he describes Dgraph as: a distributed graph database which provides horizontal scalability, distributed cluster-wide ACID transactions, low-latency arbitrary-depth joins, synchronous replication, high availability, and crash resilience. There are many claims being stated here which frankly I didn’t understand the meaning of the first time I read this sentence. I decided I wanted to better understand these claims with the hope that it would help me understand Dgraph’s architecture and engineering decisions better.
Continue reading