Managing Go Module Deprecation
Finally we have modules in Go, experimental since 1.11.x and due to be non-experimental with the release of 1.13.x. This is a very good thing and a long overdue feature in the Go tool set.
I have been working at a start-up since Jan 2018 where we now have ~100 Go micro-services and growing. We have recently migrated our shared internal Go packages to modules and now we need some tooling that helps us with Go module deprecation in the following ways:
- Fail automated builds that reference deprecated module versions
- Inspect all micro-service repositories and report on deprecated module version references
To satisfy this requirement I have released modcop; a CLI tool that adheres to the unix philosophy of ‘Make each program do one thing well’. modcop reads a deprecation rules file (which it can load locally or via a URL) and will then read a Go.mod file to report on any deprecated module references. In the event that deprecated modules are referenced it will exit with code 1.
Example deprecation rules:
github.com/myles-mcdonnell/blondie v1.0.0>=v3.0.0 #whitelist range equal or more than v1 less than v3 github.com/myles-mcdonnell/blondie =v0.8.0,v0.9.3 #whitelist v0.8.0 and v0.9.3 github.com/myles-mcdonnell/blondie !v1.5.7>=v1.8.3 #blacklist range equal or more than v1.5.7 less than 1.8.3 github.com/myles-mcdonnell/blondie !=v2.5.0 #blacklist
Note that specific versions or ranges can be white and blacklisted. modcop amalgamates these rules as so can apply them all. In the event that a version is white and black listed it is considered deprecated.
Internally we now maintain our deprecation rules in a repository and publish those rules to a well know URL on each commit to master. This way we can manage our deprecation rules in the same way as other changes, via pull requests. We also have a build pipeline on this repository to assert that the rules file is valid and can be parsed.
Next we added modcop to our CI builds. We happen to use Concourse but this could be done with any of the CI platforms that I am aware of; at build time modcop pulls the latest deprecation rules and inspects the Go.mod file(s) that are part of the build, if deprecated versions are found the build fails. Our builds are part of the pull request process and we feed the result back to Github via status checks so pull requests are automatically failed.
Finally, in order to find deprecated module versions across the entire estate we simply script a clone of all repositories and execute modcop on every Go.mod file found in each one. Not sophisticated or exciting but perfectly adequate for now. You can imagine making this more sophisticated, for example perhaps running this scan automatically each time the deprecation rules change, storing the result and surfacing it via Grafana and/or raising alerts etc.
Currently modcop only processes module references for compatible modules by semver, e.g.:
require github.com/stretchr/testify v1.3.0
Go.mod supports other types of references that modcop does not currently cover as this is sufficient for our use case. Hopefully the community will find this tool useful enough to extend it for a wider set of use cases.