Wednesday 21 September 2011

iOS: Using older SDKs with newer Xcode versions

When you update Xcode versions, the installer automatically removes any old SDKs you have lying around, and replaces them with the latest version.

This is fine behaviour, as the most recent SDKs remain backwards compatible. You can set your project to target older iOS versions. If you do this, the newer SDK features are disabled for you.

However, there are times when you need to use an older SDK.

For example, I am running the latest Xcode with a beta iOS 5 SDK installed. Since this was originally installed on a clean machine, I didn't set the beta install to use a parallel directory and leave the "release version" of the developer tools intact - they simply weren't installed. (Making a parallel install is, in general, the best practice when installing a beta Xcode/SDK set).

Fear not. You can still get your newer Xcode to build with an older SDK, without downgrading your Xcode or making a parallel install:


  • Close any running Xcode instance you have open.
  • Locate the install DMG for an old version of Xcode (e.g. Xcode_3.2.5_and_ios_sdk_4.2_final.dmg, they name them so well) and open it.
  • Do not run the installer!
  • Open the Packages directory in that disk image. It is a hidden directory. Try this terminal incantation: "open /Volumes/Xcode\ and\ iOS\ SDK/Packages"
  • Locate the iPhoneOS and iPhoneSimulator SDKs for the version you want. Run just those pkg files. (e.g. I ran the iPhoneSDK4_2.pkg and it's matching iPhoneSimulatorSDK4_2.pkg)
  • Make sure you specify the /Developer directory as your install location. If you don't, the SDKs will be installed in your root directory, under the /Platforms directory; you'll have to manually copy them into /Developer/Platforms yourself.
  • Now, re-open Xcode. If the SDKs installed in the right locations, they will be selectable in your project now.


Tuesday 20 September 2011

How to set up Jenkins CI on a Mac

In this post I will describe how to get a running Jenkins server set up on your Mac. Like most free software ("free" as in price and "free" as in freedom), Jenkins is very capable, very functional, and mostly documented. But it didn't quite work out of the box.

As with many such projects, you get far more than you pay for. But you can end up spending more than you expect.

There aren't enough step-by-step how-to guides. And there aren't many documents that help you out when things go wrong. There is a great community behind Jenkins, though, which does help. And plenty of people moaning and blogging. Now I'm adding to that noise.

It took me a few days to get the setup working properly. Hopefully this story-cum-howto will save you some of that effort.

The Prologue

All good developers know that a continuous integration (CI) server is a linchpin of the development effort. I joined a large software project without one, and made loud noises that we needed one. And so, it naturally fell to me to set up.

It's been some time since I set up a CI server. Previously, I've used ViewTier's Parabuild. It was more than adequate. But times have moved on. Although I still have a licence for it, the cools kids are hanging around at other parties these days.

Jenkins (the recent fork of Husdon) seems well-regarded, popular, and to have a good development and support community. It's also open source, so seemed the right way to go. Plenty of people have sung it's praises to me in the past, and that kind of thing counts for a lost.

Our requirements for the builds were:
  • to build two products from the same codebase on Mac OSX
  • to build two products from the same codebase on Windows

Both of these need 32-bit and 64-bit versions.

That's already a reasonable configuration matrix, and highlighted why we needed CI in place. A developer would check in a tweak that they'd built on one configuration. All the other config could easily get broken without anyone noticing for a while.

So: Jenkins to the rescue.

Almost.

I purchased a small Mac Mini to use as a build server. I downloaded a copy of Jenkins (free, whoop!), installed the Mac dev tools (free, whoop!) bought (and installed) Parallels, Windows 7, and the Visual Studio toolchain (not quite as free), and sat down for a small configuration session.


Getting your project ready for a CI build

Before setting up your build on an CI server, you should first create a simple script that builds everything from a clean checkout. Then check that script into the repository, to be versioned alongside the software itself.

For our project, I already had that in place. The script cleaned, built, versioned and packaged the software in one step.

Such scripts are clearly useful for deployment on a CI server, and also for making official software releases by hand, whether or not you release from the CI server builds. It's a record of the recipe needed to build a release.

With a fixed recipe like this in place, every software release can be guaranteed to be good and reproducible.

That's development 101.




Installing Jenkins on the Mac

The Jenkins website has a handy Mac installer that you can download. (In retrospect, I'm not sure if this was more hassle than it was worth, but this is the route I obviously sought to go down.)

STEP 1: Install Jenkins

Download the Mac installer and run it.

This installer creates a system launch daemon that fires up Jenkins when your machine boots. This runs in the background even if you haven't logged in, making a true stand-alone build server installation.

However, if you have a fresh Lion install you don't yet have Java.

STEP 2: Install Java

Try to run a Java app. Any app. The OS may fumble around for a while looking for Java. If you're lucky, it'll download and install it automatically. Otherwise, install it by hand.

And of course, now, Jenkins "just works".

Not.


Configuring Jenkins on the Mac

The Jenkins war (web application archive) is unpacked into /Users/shared/Jenkins. The application runs from there. All configuration is stored there. The source code checkouts and builds go in there. It's the center of your Jenkins universe.

A launch daemon plist is installed in /Library/LaunchDaemons. It runs a script /Library/Application Support/jenkins-runner.sh as the user "daemon" (a system specific user that runs background processes - it is not shown on the login screen, nor does it have a home directory).

This installation has all the hallmarks of a runnable system. You can now point your browser to http://localhost:8080 and start configuring the Jenkins server. The lights are most definitely on. But no one's home yet. As we're about to see...

STEP 3: Set up Jenkins to build a Mac project

I was a good soldier with a simple shell script that built and packaged my application. If you don't have this, write one now. To build Mac projects, you'll need some cunning invocation of xcodebuild and probably packagemaker.

This single script is a critical step in configuring your build job. Whilst it is possible to place multiple build commands into the Jenkins task itself, it's far better to keep them under source control in a script checked in to your codebase (if you need to ask why then you probably need to go to a more basic tutorial!).

Configure Jenkins to check out your repository, to react to appropriate build triggers (e.g. manual build requests through the UI, automatic detection of the repository changing, or other triggers) and to run the appropriate scripts to kick off the build.

Then press "Build Now" to start your first build.

In all probability Jenkins will crash and burn. But don't tear your hair out just yet. You'll be needing it for later on.


Fix Jenkins so it works

Welcome to the nether-world of almost working builds.

STEP 4: Configure the Java heap size

My project is large. It includes lots of third party libraries, including the vastness that is Boost and many other comparable-size libraries. There are also several SDKs that are shipped as large binaries (with versions for each platform and 32/64 bit). That's a lot of data to shovel around.

Jenkins choked trying to check out this monster. It would collapse with Java heap exhaustion errors before it even got to triggering a build. Goodness only knows why a large heap is required to check files out of a subversion repository, but the solution can only be to increase the heap size to remove the bottleneck.

By default, Java allocates a very conservative default heap size to running applications; I believe its 256M or  so on 32-bit Mac OS.

On the Mac, this can be changed using the "Java Preferences" application (found in the /Applications/Utilities folder). The trick is to adjust the Java launch command line (hit Options...) to include the comand-line sneeze: "-Xmx1024M" (or whatever heap size you want). However, this didn't seem to affect the Jenkins Java process launched through launchd.

To set the heap size in that context, you have to adjust the launch script itself. You can place the command line switch into the jenkins-runner.sh file directly. However, the file does have provision to load the heap size parameter from a configuration plist. This plist does not exist by default, but you can create/edit it with the following incantation:

sudo defaults write /Library/Preferences/org.jenkins-ci heapSize 1024M

This will write a file /Library/Preferences/org.jenkins-ci.plist (note that you must not specify the plist file extension to the defaults command).

To make the system use this new heap size, you can't just restart Jenkins (either gracefully within the web interface, or by "kill -9"-ing the process. You can't even use "sudo launchctl {stop,start} org.jenkins-ci".

You could reboot. Or, more cleanly, you have to force launchd to reload of the configuration for the launch daemon using launchctl by unloading, and then reloading the daemon. It's the reloading that'll force the new configuration to take hold. (It took me a while to figure that one out!)

With an increased heap, Jenkins will fall over less. In my case, I got through a whole checkout.

But once Jenkins manages to check out the project and run the build script, you're still not quite done...

My script called xcodebuild to invoke Xcode from the command line to build the various configurations of the project. This script worked fine when run directly from the command line. However, when running within Jenkins it would bomb out with quite unfathomable errors - e.g. NSAssertions triggered from deep within the Xcode IDE codebase. Or it would just enter a hibernation state; lock-up completely, performing no work, but generating no error.

The reason for the strangeness is that xcodebuild doesn't work when run as a user that has no home directory, like daemon. It throws its toys out of the pram in as baroque a manner as it can muster.

STEP 5: Create a "jenkins" user

So, to solve this we can either have Jenkins run as one of the existing users, or - more cleanly - create a new user specifically for jenkins.

To do this:

  • Create a user called "jenkins" from Control Panel. If you care particularly, you might want to create a "hidden" user; follow the instructions here: http://hints.macworld.com/article.php?story=20080127172157404
  • Stop jenkins again: sudo launchctl unload -w /Library/LaunchAgents/org.jenkins-ci.plist
  • Edit  /Library/LaunchAgents/org.jenkins-ci.plist, change the username entry from daemon to jenkins
  • Change all permissions on the existing jenkins files: "sudo chown -R jenkins /User/Shared/jenkins" "sudo chgrp -R staff /User/Shared/Jenkins"
  • Restart Jenkins: sudo launchctl load -w /Library/LaunchAgents/org.jenkins-ci.plist
Re-start the build. Sit and wait.


Job done

From memory, that was the set-up steps required to get builds to work on a Mac from a fresh Jenkins install. These things aren't really covered by the install guides. They're obvious once you know them. Hindsight is great like that.

Plenty of people followed my whining on Twitter with disbelief, saying that Jenkins "just works" for them. Others suggested moving over to Hudson instead, but I imagine I'd've had the same issues there.

Perhaps I'm unusual, and this stuff does just work for everyone else. If that's not the case, then I hope this rant proves useful.

As a postscript, I now have my Jenkins server working well. I have configured a Windows client, running under a Parallels virtual machine on the same computer. It's not the fastest build server when both run together, but it's passable.

There are definitely some rough edges and features lacking from Jekins, but I can't complain at the price. And there are plenty of excellent plugins that really do make it a very capable build server.


Thursday 15 September 2011

Writing: Smarter, Not Harder

The September issue of ACCU's C Vu magazine is out now. It contains the latest instalment in my Becoming a Better Programmer column. This one's called Smarter, Not Harder.

In it I investigate how high-quality developers use their skill and experience to be maximally productive. I describe useful tactics to help you solve problems more easily and get the job done in the most effective way. In short: how to pick your battles.

Tuesday 6 September 2011

Many-festos

Confusion of goals and perfection of means seems, in my opinion, to characterise our age.
Albert Einstein

It's becoming an epidemic! They're springing up everywhere. We've got them coming our of our ears. It's as if you can't write a line of code, kick off a development process, or even think about the act of coding without signing up to one.

With all these manifestos for software development, our profession is in danger of becoming more about politics than the actual art, craft, science, and trade of software development.

Of course, a large and important part of professional software development is the people problem. And that necessarily involves politics, to some extent. But we're making even the foundational coding principles a political battle. Is this for the best? Or is it just a fashionable, sound-bite-sized way to get your point across, and to try to garner support for your pet hobby-horse?

These “development” manifestos are often too ambiguous for people to sign up to in any meaningful way. They're so general that they simply must be right. Akin to a development horoscope, if you will. Very few of them break new ground, or introduce anything genuinely radical. And, sadly, when a manifesto becomes popular we see factions form around it, leading to disputes about what the manifesto really stands for. Whole debates spring up around the exegesis of the particular manifesto items.
Software religion is alive and well.

Whether or not manifestos are a good idea, they seem to be springing up for any conceivable purpose. So, in order to stem the flow, and make it easier for future software activists who'd like to pen their own manifesto, here I present the one, the overarching, generic software development manifesto. Manifesto, if you like.

A generic manifesto for software development

We, the undersigned, have an opinion about software development. We are concerned about the future of our profession, and our passion leads us to draw the following conclusions:
  • We believe in a fixed set of immutable ideals
over tailoring our approach to each specific situation.
  • We believe in concentrating on and discussing only the things that interest us
over the bigger problem.
  • We believe in our opinion
over the opinions and experiences of others.
  • We believe in arbitrary black-and-white mandates
over real-world scenarios with complex issues and delicate resolutions.
  • We believe that when our approach is hard to follow
then it only shows how much more important it is.
  • We believe in crafting an arbitrary set of commandments
over the realisation that it's just never that simple.
  • We believe in trying to establish a movement to promote our view
over something that will be genuinely useful.
  • We believe that we are better developers than those who don't agree with us
because they don't agree with us.


That is, we believe we're doing the right thing. And if you don't you're wrong. And if you don't do what we do, you're doing it wrong.

OK, OK

Alright. I'll admit it. I exaggerated for effect. And my tongue is in my cheek. Mostly. 

References

Questions

  1. What foundational development “principles” do you hold dear?
  2. Do you sign up to, or align yourself with, development streams like “agile”, “craftsmanship” and so on? How closely do you agree with each of the items in their manifesto?
  3. What do you think these manifestos do have to offer the development community?
  4. What kinds of harm might they really be able do, if any?
  5. Or do you keep your head down and ignore this kind of thing? Should you actually follow these software fashions and fads to maintain personal development?