Wednesday, January 25, 2017

Notes from Golang-Brno #4: Refactoring, containers, plugins, ...

Yesterday, I facilitated an introductory Go workshop here in Brno, CZ. This post is not about the workshop though, but just some notes from the talks that followed in the evening meeting.

Update: I've written some notes about the Go workshop.

There were two talks. The first one was an excellent pick of interesting talks and stories from dotGo 2016, very well presented by Jan Klat. I was glad to meet him :-)

Refactoring: difficulty with refactoring types

One of the talks Jan shared with us was about refactorings. While I haven't watched the original talk, one think that came to mind is the Type Alias proposal that is being tracked here:

Self deploying Go: debuggability

Another talk was about deploying to "the cloud" very small containers with just statically linked Go programs. My remark was that this works really well for demos and proofs-of-concept, but has several limitations and drawbacks in practice.

Yeah, as part of my work in OpenShift I can say "been there, done that". The thing to keep in mind is that often you will want to spawn a remote shell in your container... but if all you have is a single binary... that means no shell, no common Linux utils... and very limited way of debugging problems unless you add all the things into your Go binary. No, don't do that. Save yourself.

Most of the time, it is a false economy and a misconceived objective to try to have 5 MB binaries in your containers in production. Once you deploy your image to something like OpenShift or Kubernetes, your nodes will already have a local copy of your image, so scaling, starting more instances of a container does not require pulling in more data over the network.

A more interesting thing is to layer your images properly to reuse the base layers across your containers images. Say you have a base image for all of your Go micro services that is 500 MB, and you have 3 services. Each binary implementing your service is roughly 10 MB. Now if you share a common base image, loading the first image for the very first time will transfer 110 MB, but the next 2 images will need to transfer only 10 MB each.

Plugins in Go

Jan mentioned some talk about plugins, some "novel" attempts but folks at Drone.io.... but it all sounded to me more or less what we find already in production in software from Hashicorp, like Packer for example.

The problem/solution is at least as old as April 2013:

https://www.youtube.com/watch?v=SRvm3zQQc1Q (all the story behind plugins in Go, early attempts, current design)https://github.com/hashicorp/go-plugin

The second talk was about Mall.cz and their rewriting of an existing system into Go components. Unfortunately there was very little about Go, too much about internal details, and we were left with the feeling that the new solution is buzzword-compliant, more complex, and does not address the problem with the original system...

That's honest feedback. Anyway, the two guys presenting were good, open to questions and explaining their thoughts and decisions, so I stayed until the end of the event and enjoyed it.

My notes:

Iris Web framework, fast?!

Yeah, so they seem to have chosen to use a web framework that I've never heard of, because... because it is fast? Hmm... they came with a very suspicious graph, claiming not only Iris is orders of magnitude faster than anything else, but also that most of the existing "Go web frameworks" are "faster" than net/http in the standard library... how come?
Most, if not all of those frameworks/tools/libraries do delegate the hard work to net/http, so there is no way on Earth or any other planet they would be faster than the net/http.

What's more, finding the source code on GitHub, it turns out that Iris has had code contributions from a single developer. All due respect to the Iris author, but in an Open Source world, that, plus the relative immaturity of the code, plus the speed claims are really really big warning signs.

People are free to choose whatever they want. Without going into much detail, my philosophy is keep is dependencies to a minimum. One must judge really well the ROI, the value, a given dependency is bringing compared to the complexity it is adding to your project. And keep in mind also transitive dependencies.

Trash, Go vendoring tool

There is a myriad of tools out there to help you vendor your dependencies along with your code... and also a lot to be said and learned about this topic.

Now, it was really funny to hear that the solution to Go vendoring is.... "put all of your code to trash". Sometimes that's really good advice -- fear not delete Thy code.

By the way, if your dependencies are "trash", why do you even depend on them?!

I never used trash, but what I found weird during the talk is that the speakers were happy about the fact that it deletes all the "useless" files, including "useless" tests, and parts of the dependencies that you do not use... It does look nice on the first sight, but left me with questions like what happens when you want to upgrade the versions of your dependencies and so on.

Containers in development and/or in production?

They were using Docker containers for development, and automating deployment with Chef in production.

Using net/http == using concurrency and goroutines

One of the reasons they were excited about Go was the existence of goroutines and channels. In the few minutes Go was mentioned, this was a recurring topic. But they seemed too excited about writing concurrent Go, apparently ignoring that is no trivial thing :-)

Throwing goroutines and channels into a code base will NOT magically improve performance. On the contrary, when done wrong it can harm performance, and introduce more subtle bugs.

However, one thing to note is that just because they are using Iris, and underneath net/http, their programs already have goroutines and all the fanciness of concurrency in Go! Yeah, that's another important lesson to be learned... the API of net/http has no channels, no explicit goroutines, and you get to write your handlers as regular functions, all the underlying complexity well factored away from your eyes.

Tests: build tags

They've heard about "build tags", but I think they've misunderstood it. Their example of running tests was something like this:

go test -run Unit
go test -run Integration

That means they understood tags as simply naming every test function as "TestUnitFoo". While that may work, it is a waste of time/characters/whatever to name your tests functions in that fashion.

Build tags, or build constraints, are well documented here:

For tests, one common and reasonable pattern is to write your unit tests normally, in *_test.go files that go along with your packages. Then, for integration or other types of tests, put them in separate packages, also in *_test.go files, but add a build constraint that is only satisfied when you intend to run integration tests. In other words, add this to the top of your files:

// +build integration

package foo

import "testing"


That's all from the talks yesterday :-)

No comments: