How to set up Jenkins CI on a Mac

In this post I will describe how to get a running Jenkins server set up on your Mac. Like most free software ("free" as in price and "free" as in freedom), Jenkins is very capable, very functional, and mostly documented. But it didn't quite work out of the box.

As with many such projects, you get far more than you pay for. But you can end up spending more than you expect.

There aren't enough step-by-step how-to guides. And there aren't many documents that help you out when things go wrong. There is a great community behind Jenkins, though, which does help. And plenty of people moaning and blogging. Now I'm adding to that noise.

It took me a few days to get the setup working properly. Hopefully this story-cum-howto will save you some of that effort.

The Prologue

All good developers know that a continuous integration (CI) server is a linchpin of the development effort. I joined a large software project without one, and made loud noises that we needed one. And so, it naturally fell to me to set up.

It's been some time since I set up a CI server. Previously, I've used ViewTier's Parabuild. It was more than adequate. But times have moved on. Although I still have a licence for it, the cools kids are hanging around at other parties these days.

Jenkins (the recent fork of Husdon) seems well-regarded, popular, and to have a good development and support community. It's also open source, so seemed the right way to go. Plenty of people have sung it's praises to me in the past, and that kind of thing counts for a lost.

Our requirements for the builds were:

• to build two products from the same codebase on Mac OSX
• to build two products from the same codebase on Windows

Both of these need 32-bit and 64-bit versions.

That's already a reasonable configuration matrix, and highlighted why we needed CI in place. A developer would check in a tweak that they'd built on one configuration. All the other config could easily get broken without anyone noticing for a while.

So: Jenkins to the rescue.

Almost.

I purchased a small Mac Mini to use as a build server. I downloaded a copy of Jenkins (free, whoop!), installed the Mac dev tools (free, whoop!) bought (and installed) Parallels, Windows 7, and the Visual Studio toolchain (not quite as free), and sat down for a small configuration session.


Getting your project ready for a CI build

Before setting up your build on an CI server, you should first create a simple script that builds everything from a clean checkout. Then check that script into the repository, to be versioned alongside the software itself.

For our project, I already had that in place. The script cleaned, built, versioned and packaged the software in one step.

Such scripts are clearly useful for deployment on a CI server, and also for making official software releases by hand, whether or not you release from the CI server builds. It's a record of the recipe needed to build a release.

With a fixed recipe like this in place, every software release can be guaranteed to be good and reproducible.

That's development 101.




Installing Jenkins on the Mac

The Jenkins website has a handy Mac installer that you can download. (In retrospect, I'm not sure if this was more hassle than it was worth, but this is the route I obviously sought to go down.)

STEP 1: Install Jenkins

Download the Mac installer and run it.

This installer creates a system launch daemon that fires up Jenkins when your machine boots. This runs in the background even if you haven't logged in, making a true stand-alone build server installation.

However, if you have a fresh Lion install you don't yet have Java.

STEP 2: Install Java

Try to run a Java app. Any app. The OS may fumble around for a while looking for Java. If you're lucky, it'll download and install it automatically. Otherwise, install it by hand.

And of course, now, Jenkins "just works".

Not.


Configuring Jenkins on the Mac

The Jenkins war (web application archive) is unpacked into /Users/shared/Jenkins. The application runs from there. All configuration is stored there. The source code checkouts and builds go in there. It's the center of your Jenkins universe.

A launch daemon plist is installed in /Library/LaunchDaemons. It runs a script /Library/Application Support/jenkins-runner.sh as the user "daemon" (a system specific user that runs background processes - it is not shown on the login screen, nor does it have a home directory).

This installation has all the hallmarks of a runnable system. You can now point your browser to http://localhost:8080 and start configuring the Jenkins server. The lights are most definitely on. But no one's home yet. As we're about to see...

STEP 3: Set up Jenkins to build a Mac project

I was a good soldier with a simple shell script that built and packaged my application. If you don't have this, write one now. To build Mac projects, you'll need some cunning invocation of xcodebuild and probably packagemaker.

This single script is a critical step in configuring your build job. Whilst it is possible to place multiple build commands into the Jenkins task itself, it's far better to keep them under source control in a script checked in to your codebase (if you need to ask why then you probably need to go to a more basic tutorial!).

Configure Jenkins to check out your repository, to react to appropriate build triggers (e.g. manual build requests through the UI, automatic detection of the repository changing, or other triggers) and to run the appropriate scripts to kick off the build.

Then press "Build Now" to start your first build.

In all probability Jenkins will crash and burn. But don't tear your hair out just yet. You'll be needing it for later on.


Fix Jenkins so it works

Welcome to the nether-world of almost working builds.

STEP 4: Configure the Java heap size

My project is large. It includes lots of third party libraries, including the vastness that is Boost and many other comparable-size libraries. There are also several SDKs that are shipped as large binaries (with versions for each platform and 32/64 bit). That's a lot of data to shovel around.

Jenkins choked trying to check out this monster. It would collapse with Java heap exhaustion errors before it even got to triggering a build. Goodness only knows why a large heap is required to check files out of a subversion repository, but the solution can only be to increase the heap size to remove the bottleneck.

By default, Java allocates a very conservative default heap size to running applications; I believe its 256M or  so on 32-bit Mac OS.

On the Mac, this can be changed using the "Java Preferences" application (found in the /Applications/Utilities folder). The trick is to adjust the Java launch command line (hit Options...) to include the comand-line sneeze: "-Xmx1024M" (or whatever heap size you want). However, this didn't seem to affect the Jenkins Java process launched through launchd.

To set the heap size in that context, you have to adjust the launch script itself. You can place the command line switch into the jenkins-runner.sh file directly. However, the file does have provision to load the heap size parameter from a configuration plist. This plist does not exist by default, but you can create/edit it with the following incantation:

sudo defaults write /Library/Preferences/org.jenkins-ci heapSize 1024M

This will write a file /Library/Preferences/org.jenkins-ci.plist (note that you must not specify the plist file extension to the defaults command).

To make the system use this new heap size, you can't just restart Jenkins (either gracefully within the web interface, or by "kill -9"-ing the process. You can't even use "sudo launchctl {stop,start} org.jenkins-ci".

You could reboot. Or, more cleanly, you have to force launchd to reload of the configuration for the launch daemon using launchctl by unloading, and then reloading the daemon. It's the reloading that'll force the new configuration to take hold. (It took me a while to figure that one out!)

With an increased heap, Jenkins will fall over less. In my case, I got through a whole checkout.

But once Jenkins manages to check out the project and run the build script, you're still not quite done...

My script called xcodebuild to invoke Xcode from the command line to build the various configurations of the project. This script worked fine when run directly from the command line. However, when running within Jenkins it would bomb out with quite unfathomable errors - e.g. NSAssertions triggered from deep within the Xcode IDE codebase. Or it would just enter a hibernation state; lock-up completely, performing no work, but generating no error.

The reason for the strangeness is that xcodebuild doesn't work when run as a user that has no home directory, like daemon. It throws its toys out of the pram in as baroque a manner as it can muster.

STEP 5: Create a "jenkins" user

So, to solve this we can either have Jenkins run as one of the existing users, or - more cleanly - create a new user specifically for jenkins.

To do this:

• Create a user called "jenkins" from Control Panel. If you care particularly, you might want to create a "hidden" user; follow the instructions here: http://hints.macworld.com/article.php?story=20080127172157404
  
• Stop jenkins again: sudo launchctl unload -w /Library/LaunchAgents/org.jenkins-ci.plist
  
• Edit  /Library/LaunchAgents/org.jenkins-ci.plist, change the username entry from daemon to jenkins
  
• Change all permissions on the existing jenkins files: "sudo chown -R jenkins /User/Shared/jenkins" "sudo chgrp -R staff /User/Shared/Jenkins"
  
• Restart Jenkins: sudo launchctl load -w /Library/LaunchAgents/org.jenkins-ci.plist

Re-start the build. Sit and wait.


Job done

From memory, that was the set-up steps required to get builds to work on a Mac from a fresh Jenkins install. These things aren't really covered by the install guides. They're obvious once you know them. Hindsight is great like that.

Plenty of people followed my whining on Twitter with disbelief, saying that Jenkins "just works" for them. Others suggested moving over to Hudson instead, but I imagine I'd've had the same issues there.

Perhaps I'm unusual, and this stuff does just work for everyone else. If that's not the case, then I hope this rant proves useful.

As a postscript, I now have my Jenkins server working well. I have configured a Windows client, running under a Parallels virtual machine on the same computer. It's not the fastest build server when both run together, but it's passable.

There are definitely some rough edges and features lacking from Jekins, but I can't complain at the price. And there are plenty of excellent plugins that really do make it a very capable build server.