SRE book notes: Release Engineering
Changes to any aspect of the release process should be intentional, rather than accidental
These are the notes from Chapter 8: Release Engineering from the book Site Reliability Engineering, How Google Runs Production Systems.
This is a post of a series. The previous post can be seen here:
Changes to any aspect of the release process should be intentional, rather than accidental
Release processes can be automated to the point that they require minimal involvement by the engineers, and many projects are automatically built and released using a combination of our automated build system and our deployment tools. Releases are truly automatic, and only require engineer involvement if and when problems arise.
Small companies are ok not having the release process totally automated. Just watch out to not transform it into toil.
Please check the chapter about eliminating toil, if you have not done so.
We have embraced the philosophy that frequent releases result in fewer changes between versions. This approach makes testing and troubleshooting easier.
Fewer changes between versions also help with code reviews. Teammates do not need to keep a big chunk of logic on their heads when evaluating your changes. It will lead to quicker reviews and happier colleagues.
most major projects don’t release directly from the mainline. Instead, we branch from the mainline at a specific revision and never merge changes from the branch back into the mainline. Bug fixes are submitted to the mainline and then cherry picked into the branch for inclusion in the release.
This process when performed manually, or semi-manually, could quickly become a nightmare. This is not Google’s case, as mentioned before, their releases are automatic.
When equipped with the right tools, proper automation, and well-defined policies, developers and SREs shouldn’t have to worry about releasing software. Releases can be as painless as simply pressing a button.
It’s not just for Googlers... all companies should take the effort to define their release processes whether or not the processes can be automated and/or enforced.
Start Release Engineering at the Beginning... It’s cheaper to put good practices and process in place early, rather than have to retrofit your system later.
I can’t stress enough the importance of this!
It’s much easier to automate, perform validations, and security checks at the beginning when the system can still fit in your head.
This will also immensely help with the cadence you push code to production.
It is essential that the developers, SREs, and release engineers work together. The release engineer needs to understand the intention of how the code should be built and deployed. The developers shouldn’t build and “throw the results over the fence” to be handled by the release engineers.