Friday 25 September 2009

Code: iPhone linker error (__restore_vfp_d8_d15_regs)

In my iPhone development I encountered link errors along the lines of:
"___restore_vfp_d8_d15_regs", referenced from:
-[Blah blah:] in blah.o
"___save_vfp_d8_d15_regs", referenced from:
-[Blah blah:] in blah.o

Google had a little to say about it. Unfortunately, it all appeared wrong.

The issue was a subtle one. The application's "Library search paths" (LIBRARY_SEARCH_PATHS) variable has some historic cruft in it which caused the linker to pull in and old incompatible version of libstdc++, with a consequence of much hilarity and hair-pulling.

When I removed "$(SDKROOT)/usr/lib/gcc/arm-apple-darwin9/4.0.1" from the list (not sure where this had come from) the application magically linked one more, and kittens and puppies danced with my iPhone once more.

Thursday 24 September 2009

Code: Respect the Software Release Process

Several times recently I've run into problems caused by other developers' lackadaisical approach to the construction of software releases.

Many of these were caused by the sloppy habit of creating a release of a local working directory, rather than from a clean checkout.

For example:
  • An external software release was made from a local directory containing uncommitted source file changes. We have no record of exactly what went into that build. And knowing it was built like this, I have no faith in the quality of the release at all.
  • An external software release was made from a local directory that wasn't up-to-date. So it was missing one feature, and some bug fixes. But the developer tagged the HEAD of the repository, and then claimed he'd build that version. The built code begged to differ.
I mean, come on! It's not that hard, is it?

Well, actually: yes it is. Creating a serious high-quality software release is actually a lot more work than just hitting "build" in your IDE and shipping whatever comes out. If you are not prepared to put in this extra work then you should not be creating releases.

Harsh. But fair.

Respect the Software Release Process

Presuming that you are writing software for the benefit of others as well as yourself, it has to get into the hands of your "users" somehow. Whether you end up rolling a software installer shipped on a CD or deploying the software on a live web server, this is the important process of creating a software release.

The software release process is a critical part of your software development regimen, just as important as design, coding, debugging, and testing. To be effective your release process must be:
  • simple
  • repeatable
  • reliable

Get it wrong, and you will be storing up some potentially nasty problems for your future self. When you construct a release you must:
  • Ensure that you can get the exact same code that built it back again from your source control system. (You do use source control, don't you?) This is the only concrete way to prove which bugs were and were not fixed in that release. Then when you have to fix a critical bug in version 1.02 of a product that's five years old, you can do so.
  • Record exactly how it was built (including the compiler optimisation settings, target CPU configuration, etc). These features may have subtly affects how well your code runs, and whether certain bugs manifest.
  • Capture the build log for future reference.

The bare outline of a good release process is:

  • Agree that it's time to spin a new release. A formal release is treated differently to a developer's test build, and should never come from an existing working directory.
  • Agree what the "name" of the release is (e.g. "5.06 Beta1" or "1.2 Release Candidate").
  • Determine exactly what code will constitute this release. In most formal release processes, you will already be working on a release branch in your source control system, so it's the state of that branch right now.
  • Tag the code in source control to record what is going into the release. The tag name must reflect the release name.
  • Check out a virgin copy of the entire codebase at that tag. Never use an existing checkout. You may have uncommitted local changes that change the build. Always tag then checkout the tag. This will avoid many potential problems.
  • Build the software. This step must not involve hand-editing any files at all, otherwise you do not have a versioned record of exactly the code you built.
  • Ideally, the build should be automated: a single button press, or a single script invocation. Checking the mechanics of the build into source control with the code records unambiguously how the code was constructed. Automation reduces the potential for human error in the release process.
  • Package the code (create an installer image, CD ISO images, etc). This step should also be automated for the same reason.
  • Always test the newly constructed release. Yes, you tested the code already to ensure it was time to release, but now you should test this "release" version to ensure it is of suitable release quality.
  • Construct a set of "Release notes" describing how the release differs from the previous release: the new features and the bugs that have been fixed.
  • Store the generated artifacts and the build log for future reference.
  • Deploy the release. Perhaps this involves putting the installer on your website, sending out memos or press releases to people who need to know. Update release servers as appropriate.

This is a large topic tied intimately with configuration management, testing procedures, software product management, and the like. If you have any part in releasing a software product you really must understand and respect the sanctity of the software release process.

Friday 18 September 2009

Code: Creating a framework for the iPhone

Summary
This article explains how to build your own framework for Apple's iPhone OS.

The problem

Apple's Xcode development environment does not let programmers create their own framework for use in iPhone OS applications. This has caused many iPhone developers great frustration, although the restriction is for fairly sensible reasons.

So why the restriction?

A framework usually contains a dynamically loaded shared library (and the associated header files to be able to access its facilities). iPhone OS keeps applications very separate from one another, and so there is no concept of a user-created dynamic library shared between applications. There is no central library install point accessible to the developer. Indeed, managing such a software pool would be rather complex on iPhone-like devices, and preventing developers from installing their own shared frameworks neatly sidesteps a whole world of painful shared library compatibility issues, and well as simplifying the application uninstall process.

Its one, fairly final, way to avoid DLL hell!

All applications may link to the blessed, system-provided frameworks. The only other libraries they may use must be standard static libraries, linked directly to the application itself.

Those of us who'd like to supply functionality to other users in library form are left at somewhat of a disadvantage, though. Most developers are used to the simplicity of dragging a framework into their application target in Xcode, and not worrying about header paths or link issues.

It's nowhere near as neat to have to provide a static library and a set of associated header files in a flat directory. This requires your clients to work out the installation in their application for themselves. It's not hard, but it is tedious. You've also got to ship a library version for each platform the developer will need (at the very least, an arm code library for use on the iPhone OS device itself, and an i386 build for them to use in the iPhone simulator).

It's clumsy.

But fear not, there is a way...


How to build your own framework

I've worked out how to create a usable Framework that you can ship to other iPhone OS application writers. You can ship libraries that are easy to incorporate into other projects, and can exploit the standard framework versioning facilities.

There is one caveat: the framework will not be a standard shared library, it will provide a statically linked library. But the application writer need not be concerned about this issue. As far as they're concerned everything will just work. We are using Apple machines, after all.

Here's how to do it:

1. Structure your framework's header files.

Let's say your library is called "MyLib". Structure your project with a
top-level directory called "Include", and inside that make a "MyLib" subdirectory. Put all your public header files in there.

To be idiomatic, you'll want to create an umbrella header file "Include/MyLib/MyLib.h" that includes all the other headers for the user's convenience.

Set up your Xcode project "Header Search Paths" build parameter to "Include".
Now your source files can happily #import <mylib/mylib.h> in the same way they'd use any other framework. Everything will include properly.

2. Put your source elsewhere

I have a "Source" directory containing subdirectories "Source/MyLib" and "Source/Tests". You can put your implementation files (and private header files) wherever you want. Just, obviously, not in the Include directory!

3. Create a static library target

Create an iPhone OS static library target that builds all your library sources. Call it MyLib, and by default it will create a static library called libMyLib.a.

4. Create the framework plist file

Create a plist file that will be placed inside your framework, describing it.

I keep mine in Resources/Framework.plist. It's a peice of XML joy that should look like this:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>English</string>
<key>CFBundleExecutable</key>
<string>MyLib</string>
<key>CFBundleIdentifier</key>
<string>com.MyLovelyDomain.MyLib</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundlePackageType</key>
<string>FMWK</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>1.0</string>
</dict>
</plist>
5. The magic part... build your framework by hand

Create a shell script to build your framework. I have a "Scripts" directory that contains it, because I like to keep things neat like that.

The first line is the canonical hashbang:
#!/bin/bash
There are two parts to this file...

5a. Build all the configurations that you need your framework to support

This must be at least armv6 for the device, and x386 for the simulator. You'll want these to be Release configuration libraries.
xcodebuild -configuration Release -target "MyLib" -sdk iphoneos3.0
xcodebuild -configuration Release -target "MyLib" -sdk iphonesimulator3.0

So that's our libraries built. Now...

5b. Piece it all together

With a little understanding of the canonical structure of a framework directory, our ability to write a plist, and the knowledge that putting a static library in the framework instead of a dynamic library works fine, you can create your framework like this (apologies that blogger has kinda killed the formatting here):

# Define these to suit your nefarious purposes
FRAMEWORK_NAME=MyLib
FRAMEWORK_VERSION=A
FRAMEWORK_CURRENT_VERSION=1
FRAMEWORK_COMPATIBILITY_VERSION=1
BUILD_TYPE=Release

# Where we'll put the build framework. The script presumes we're in the
# project root directory
FRAMEWORK_BUILD_PATH="build/Framework"

# Clean any existing framework that might be there already
echo "Framework: Cleaning framework..."
[ -d "$FRAMEWORK_BUILD_PATH" ] && rm -rf "$FRAMEWORK_BUILD_PATH"

# The full name of the framework we'll build
FRAMEWORK_DIR=$FRAMEWORK_BUILD_PATH/$FRAMEWORK_NAME.framework

echo "Framework: Setting up directories..."
mkdir -p $FRAMEWORK_DIR
mkdir -p $FRAMEWORK_DIR/Versions
mkdir -p $FRAMEWORK_DIR/Versions/$FRAMEWORK_VERSION
mkdir -p $FRAMEWORK_DIR/Versions/$FRAMEWORK_VERSION/Resources
mkdir -p $FRAMEWORK_DIR/Versions/$FRAMEWORK_VERSION/Headers

echo "Framework: Creating symlinks..."
ln -s $FRAMEWORK_VERSION $FRAMEWORK_DIR/Versions/Current
ln -s Versions/Current/Headers $FRAMEWORK_DIR/Headers
ln -s Versions/Current/Resources $FRAMEWORK_DIR/Resources
ln -s Versions/Current/$FRAMEWORK_NAME $FRAMEWORK_DIR/$FRAMEWORK_NAME

# Check this is what your static libraries are called
FRAMEWORK_INPUT_ARM_FILES="build/$BUILD_TYPE-iphoneos/libMyLib.a"
FRAMEWORK_INPUT_I386_FILES="build/$BUILD_TYPE-iphonesimulator/libMyLib.a"

# The trick for creating a fully usable library is to use lipo to glue the
# different library versions together into one file. When an application is linked
# to this library, the linker will extract the appropriate platform version and
# use that.
# The library file is given the same name as the framework with no .a extension.
echo "Framework: Creating library..."
lipo \
-create \
-arch armv6 "$FRAMEWORK_INPUT_ARM_FILES" \
-arch i386 "$FRAMEWORK_INPUT_I386_FILES" \
-o "$FRAMEWORK_DIR/Versions/Current/$FRAMEWORK_NAME"

# Now copy the final asserts over: your library header files and a plist file
echo "Framework: Copying assets into current version..."
cp Include/$FRAMEWORK_NAME/* $FRAMEWORK_DIR/Headers/
cp Resources/Framework.plist $FRAMEWORK_DIR/Resources/Info.plist
That's it. Run that script and you'll create a framework in "build/Framework". In there is MyLib.framework. This directory can be shipped to your external application developers. They can incorporate it into their iPhone OS applications like any other framework.

Congratulations, you are now a hero.


Other remarks

I have presented here the most basic structure of a shell file. My production version includes more robust error handling, and other facilties that are relevant to my particular project.

I also have a build script that automatically creates documentation for the framework that I can ship with it. Indeed, I have a release script that applies versioning information to the project, build the libraries, creates a framework, assembles the documentation, compiles release notes and packages the whole thing in a pretty DMG. But that's another story.

If calling scripts from the command line scares you, you may chose to make a "Run Script Build Phase" in your Xcode project to call your framework script. Then you can create a framework without having to creep to the command line continually.

In summary, the final file layout of my project looks like this:
....Include/
........MyLib/
............MyLib.h
............IncludeFile1.h
............IncludeFile2.h
....Source/
........MyLib/
............PrivateHeader.h
............ImplFile1.m
............ImplFile2.m
........Tests
............MyLibTest1.m
............MyLibTest2.m
....Scripts/
........framework.sh

I hope you have found this tutorial useful. Let me know what frameworks you manage to build.

Friday 4 September 2009

Code: 97 Things Every Programmer Should Know

The development website for O'Reilly's latest project in it's "97" series has now been made public.

97 Things Every Programmer Should Know looks like it will be an interesting book. Edited by Kevlin Henney, it contains a series of 97 (I'm not quite sure why 97 exactly, its a fairly weak gimmick) very short, pithy pieces of advice aimed at software developers to help them craft better code.

I have submitted a number of entries to this project.

You can take a look at all the entries submitted so far, and perhaps contribute something yourself. The site is a Wiki, and is now open for collaboration by the wider programming community.