Thursday, 24 September 2009

Code: Respect the Software Release Process

Several times recently I've run into problems caused by other developers' lackadaisical approach to the construction of software releases.

Many of these were caused by the sloppy habit of creating a release of a local working directory, rather than from a clean checkout.

For example:
  • An external software release was made from a local directory containing uncommitted source file changes. We have no record of exactly what went into that build. And knowing it was built like this, I have no faith in the quality of the release at all.
  • An external software release was made from a local directory that wasn't up-to-date. So it was missing one feature, and some bug fixes. But the developer tagged the HEAD of the repository, and then claimed he'd build that version. The built code begged to differ.
I mean, come on! It's not that hard, is it?

Well, actually: yes it is. Creating a serious high-quality software release is actually a lot more work than just hitting "build" in your IDE and shipping whatever comes out. If you are not prepared to put in this extra work then you should not be creating releases.

Harsh. But fair.

Respect the Software Release Process

Presuming that you are writing software for the benefit of others as well as yourself, it has to get into the hands of your "users" somehow. Whether you end up rolling a software installer shipped on a CD or deploying the software on a live web server, this is the important process of creating a software release.

The software release process is a critical part of your software development regimen, just as important as design, coding, debugging, and testing. To be effective your release process must be:
  • simple
  • repeatable
  • reliable

Get it wrong, and you will be storing up some potentially nasty problems for your future self. When you construct a release you must:
  • Ensure that you can get the exact same code that built it back again from your source control system. (You do use source control, don't you?) This is the only concrete way to prove which bugs were and were not fixed in that release. Then when you have to fix a critical bug in version 1.02 of a product that's five years old, you can do so.
  • Record exactly how it was built (including the compiler optimisation settings, target CPU configuration, etc). These features may have subtly affects how well your code runs, and whether certain bugs manifest.
  • Capture the build log for future reference.

The bare outline of a good release process is:

  • Agree that it's time to spin a new release. A formal release is treated differently to a developer's test build, and should never come from an existing working directory.
  • Agree what the "name" of the release is (e.g. "5.06 Beta1" or "1.2 Release Candidate").
  • Determine exactly what code will constitute this release. In most formal release processes, you will already be working on a release branch in your source control system, so it's the state of that branch right now.
  • Tag the code in source control to record what is going into the release. The tag name must reflect the release name.
  • Check out a virgin copy of the entire codebase at that tag. Never use an existing checkout. You may have uncommitted local changes that change the build. Always tag then checkout the tag. This will avoid many potential problems.
  • Build the software. This step must not involve hand-editing any files at all, otherwise you do not have a versioned record of exactly the code you built.
  • Ideally, the build should be automated: a single button press, or a single script invocation. Checking the mechanics of the build into source control with the code records unambiguously how the code was constructed. Automation reduces the potential for human error in the release process.
  • Package the code (create an installer image, CD ISO images, etc). This step should also be automated for the same reason.
  • Always test the newly constructed release. Yes, you tested the code already to ensure it was time to release, but now you should test this "release" version to ensure it is of suitable release quality.
  • Construct a set of "Release notes" describing how the release differs from the previous release: the new features and the bugs that have been fixed.
  • Store the generated artifacts and the build log for future reference.
  • Deploy the release. Perhaps this involves putting the installer on your website, sending out memos or press releases to people who need to know. Update release servers as appropriate.

This is a large topic tied intimately with configuration management, testing procedures, software product management, and the like. If you have any part in releasing a software product you really must understand and respect the sanctity of the software release process.


Rik Hemsley said...

Continuous integration can manage part of the process for you. It can be set up to check out any new changes, make a build, test it and, if it works, build and tag a release.

We use TeamCity for this, but there are others around which are similar.

Pete Goodliffe said...

Indeed. I use Parabuild.

My Prabuild build script builds the toolchain, the OS, the supporting libraries, my product (in all variants), runs all unit tests and integration tests, packages the product in the various forms required (web update file, physical ROM image for manufacture) and copies the built image to our release server.

This happens nightly. One of those nightly builds gets promoted to an RC for testing, and a successful RC gets promoted to the GM.

A programmer rarely initiates the release build process.

Tim said...

Continuous integration on a separate build server. Every check-in has the potential to be the release!