Manage Go tools with Go
In almost any code, there comes a time when you need to use external tools for certain functionality. For example, you may want to use
golangci-lint to lint your code, generate mocks with
mockgen, or run your DB schema migrations with
There are few ways to achieve this: some projects use
Makefile to install dependencies, some other use
docker compose and run tools by running containers with mounted volumes.
Today, I want to share a method that is known within Go community, but is still not frequently used.
To understand how it works, let’s start from scratch and install
golangci-lint with Go:
go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest golangci-lint run
The first command installs the
latest version of
golangci-lint and the second command executes it (you will need to have
$GOPATH/bin in your
Of course, you can also install specific version of the binary:
go install firstname.lastname@example.org golangci-lint run
While the second option is better, as you use specific version of the tool, it’s still cumbersome to use. For example, what happens when new version of
golangci-lint comes out and you want to update? You and all your teammates will need to update the version manually. Even with
Makefile, you will need to run a
make command to install the latest version. This problem is even more prominent if you have multiple repositories that all use same tools (you will not always have time to update all of them at once).
This problem can be fixed with
docker compose, but Docker is slow and it doesn’t integrate well with
Luckily, since Go 1.17
go run command accepts an optional version suffix, which lets us run any Go binary without installing it locally:
go run email@example.com run
This is already an improvement, because this line can now be put in e.g.
Makefile and it will work for any repository letting the developers forget about the problems of keeping their local version in sync with the version needed by the project. One person can update the version and it will be automatically used by every team member once they checkout the main branch.
You can also use it with
go generate command:
//go:generate go run firstname.lastname@example.org --name HTTPListener type HTTPListener func(addr string, handler http.Handler) error
This is great, however for large repositories, it may be a bit repetitive.
go run downloads the code for the binary from the internet and just runs it, which could have massive security implications. I like to improve from here by using an older pattern, or as I call it, “the
This is how it looks like in practice (this assumes this file is saved under
//go:build tools // To install everything from this file, run: // go generate -tags tools tools/tools.go package tools import ( _ "github.com/cosmtrek/air" //go:generate go install github.com/cosmtrek/air _ "github.com/daixiang0/gci" //go:generate go install github.com/daixiang0/gci _ "github.com/golangci/golangci-lint/cmd/golangci-lint" //go:generate go install github.com/golangci/golangci-lint/cmd/golangci-lint _ "github.com/vektra/mockery/v2" //go:generate go install github.com/vektra/mockery/v2 _ "gotest.tools/gotestsum" //go:generate go install gotest.tools/gotestsum _ "mvdan.cc/gofumpt" //go:generate go install mvdan.cc/gofumpt )
Let me explain what happens. First, we add
//go:build tools comment which instruments Go to include this file in a package only when
tools tag is explicitly specified.
Then, we specify the package name
tools and a list of imports. Each import path is aliased to
_ (because we don’t actually use it in the file) and a path to the binary that you want to use. I also add
//go:generate comment with
go install command to install the binary in that line. Notice the
go install command does not specify the version this time.
With this pattern, Go is smart enough to include these packages in
go.mod file and thus version them. We can then vendor those, or validate that none of the packages have been modified (with
go mod verify) which improves security and repeatability in CI. Because I also add
//go generate comments, all tools can be installed with
go generate -tags tools tools/tools.go line. Go knows which version to install. Have I mentioned I call it the
go generate commands can then skip the version:
//go:generate go run github.com/vektra/mockery/v2 --name HTTPListener type HTTPListener func(addr string, handler http.Handler) error
Now your Go tools can be installed and versioned by Go. Isn’t that great?
It would be dishonest to not mention a few things that are not great about this approach:
go.sumbecome polluted with dependencies
Depending on how much you care about keeping your dependency list clean, this may not be desired.
If you prefer to not pollute the
go.modfile, you still may use the
go run tool@versionapproach and skip the
toolspattern. It will let you version the dependencies, but is a bit more cumbersome to maintain.
You run self-built binary
Because the dependency versions are resolved by
go.modfile in projects repository, the locally compiled binary may by slightly different than the one officially distributed.
I’ve personally never had any problems with it, but tools such as
golangci-lintthat heavily rely on dependency versions for functionality may behave slightly differently if you update packages only selectively. Personally, I’ve never experienced any problem like that.
!! WARNING: You are using a self-compiled binary which is not officially supported. !! To dismiss this warning, set PSCALE_DISABLE_DEV_WARNING=true
Doesn’t work with non-Go binaries
Sometimes, you just need to have a tool that is not build with Go. For example, when using gRPC,
protoc-gen-gois used which can be versioned with this method, however, its dependency,
protoccannot, as it’s not a Go program.
Unfortunately, it means that for some repositories, you’d still need to maintain an alternative way to version binaries. Depending on the use case, it may still be worth it.
You need to configure your editor to use it
By default, most editors will not run
go run ...for you so it requires a bit of configuration. The good news, is that once it’s done, it will work for any repository that uses this pattern. For most tools, it’s not required anyway.
I really like this pattern as it allows me to forget about managing versions of the tools I use in my projects. Additionally, tools like Dependabot or Renovate can help automate this process and keep your tools (and other dependencies) up to date. In short, I like that it “just works”.
There’s an open proposal in Go’s Github repository to track tool dependencies in
go.mod. This proposal is meant to improve the experience of managing tool dependencies with Go, but it’s unclear when, or how, it will be implemented.