Wednesday, 21 October 2009

Book Review: iPhone Cool Projects

Name: iPhone Cool Projects
Author: Gary Bennet et al
Publisher: APress
Price: $39.99
Pages: 209
Reviewed by: Pete Goodliffe
Verdict: Recommended
iPhone programming is one of the current "hot topics" and we're seeing an increasing number of books published on this topic. This one is a bit of a mixed bag.

This is not an introductory tome; it requires significant prior understanding of the iPhone toolset and development environment. Instead, the book presents a number of complete fully-working iPhone applications covering various core iPhone technologies. It fits into a series of other Apress iPhone titles. Not having seem the other books, I can't say how well it complements the other titles in the series.

The book is effectively a collection essays by many authors, one per chapter, all "experts" at various aspects of iPhone development. Some of them have produced very successful iPhone applications.

The topics covered are: simple game programming, peer-to-peer networking, multi threaded applications, creating multi-touch interfaces, physics and 2D animation libraries, audio streaming, and creating a location-aware application with navigation-based UI. There are no topics covered that you can't fathom relatively easily from the free Apple documentation and a bit of careful thought. However the useful piece of the jigsaw is seeing how other developers have already learnt iPhone OS and solved the common problems.

The production quality of the book is high. It has been very well presented, in full colour throughout, with many iPhone and Xcode screen shots. On the whole, the writing is good. Some chapters appear to have been better proof read than others.

Perhaps the most useful part of the book is the availability of the source code for all the sample applications (from the publisher's website), so you can run and take apart the projects at your leisure. There is no bundled CD, and I'm more than happy with that.

As with many such multi-authored books, some chapters are better than others. Each chapter is relatively short, and they all basically provide an overview of their topic - enough to pique your interest, but not enough to answer any serious questions. For some topics this works better than others.

Highlights are the first game-writing chapter, the multi-touch interface chapters, and the location-based application chapter. These present useful information about how to write a "real" iPhone application. I felt let down by the threading chapter which presents a fairly glib and un-thorough overview of the perils of writing threaded apps. The networking chapter is a very simplistic introduction; nothing it says is wrong, but to write your own serious networked application you'd really need to know a lot more about network technologies.

If you're an experienced programmer who wants a casual introduction to some more meaty iPhone projects than you've seen in the introductory tests, this book may be interesting for you. It's easy to read, fast paced, and pretty.

Friday, 25 September 2009

Code: iPhone linker error (__restore_vfp_d8_d15_regs)

In my iPhone development I encountered link errors along the lines of:
"___restore_vfp_d8_d15_regs", referenced from:
-[Blah blah:] in blah.o
"___save_vfp_d8_d15_regs", referenced from:
-[Blah blah:] in blah.o

Google had a little to say about it. Unfortunately, it all appeared wrong.

The issue was a subtle one. The application's "Library search paths" (LIBRARY_SEARCH_PATHS) variable has some historic cruft in it which caused the linker to pull in and old incompatible version of libstdc++, with a consequence of much hilarity and hair-pulling.

When I removed "$(SDKROOT)/usr/lib/gcc/arm-apple-darwin9/4.0.1" from the list (not sure where this had come from) the application magically linked one more, and kittens and puppies danced with my iPhone once more.

Thursday, 24 September 2009

Code: Respect the Software Release Process

Several times recently I've run into problems caused by other developers' lackadaisical approach to the construction of software releases.

Many of these were caused by the sloppy habit of creating a release of a local working directory, rather than from a clean checkout.

For example:
  • An external software release was made from a local directory containing uncommitted source file changes. We have no record of exactly what went into that build. And knowing it was built like this, I have no faith in the quality of the release at all.
  • An external software release was made from a local directory that wasn't up-to-date. So it was missing one feature, and some bug fixes. But the developer tagged the HEAD of the repository, and then claimed he'd build that version. The built code begged to differ.
I mean, come on! It's not that hard, is it?

Well, actually: yes it is. Creating a serious high-quality software release is actually a lot more work than just hitting "build" in your IDE and shipping whatever comes out. If you are not prepared to put in this extra work then you should not be creating releases.

Harsh. But fair.

Respect the Software Release Process

Presuming that you are writing software for the benefit of others as well as yourself, it has to get into the hands of your "users" somehow. Whether you end up rolling a software installer shipped on a CD or deploying the software on a live web server, this is the important process of creating a software release.

The software release process is a critical part of your software development regimen, just as important as design, coding, debugging, and testing. To be effective your release process must be:
  • simple
  • repeatable
  • reliable

Get it wrong, and you will be storing up some potentially nasty problems for your future self. When you construct a release you must:
  • Ensure that you can get the exact same code that built it back again from your source control system. (You do use source control, don't you?) This is the only concrete way to prove which bugs were and were not fixed in that release. Then when you have to fix a critical bug in version 1.02 of a product that's five years old, you can do so.
  • Record exactly how it was built (including the compiler optimisation settings, target CPU configuration, etc). These features may have subtly affects how well your code runs, and whether certain bugs manifest.
  • Capture the build log for future reference.

The bare outline of a good release process is:

  • Agree that it's time to spin a new release. A formal release is treated differently to a developer's test build, and should never come from an existing working directory.
  • Agree what the "name" of the release is (e.g. "5.06 Beta1" or "1.2 Release Candidate").
  • Determine exactly what code will constitute this release. In most formal release processes, you will already be working on a release branch in your source control system, so it's the state of that branch right now.
  • Tag the code in source control to record what is going into the release. The tag name must reflect the release name.
  • Check out a virgin copy of the entire codebase at that tag. Never use an existing checkout. You may have uncommitted local changes that change the build. Always tag then checkout the tag. This will avoid many potential problems.
  • Build the software. This step must not involve hand-editing any files at all, otherwise you do not have a versioned record of exactly the code you built.
  • Ideally, the build should be automated: a single button press, or a single script invocation. Checking the mechanics of the build into source control with the code records unambiguously how the code was constructed. Automation reduces the potential for human error in the release process.
  • Package the code (create an installer image, CD ISO images, etc). This step should also be automated for the same reason.
  • Always test the newly constructed release. Yes, you tested the code already to ensure it was time to release, but now you should test this "release" version to ensure it is of suitable release quality.
  • Construct a set of "Release notes" describing how the release differs from the previous release: the new features and the bugs that have been fixed.
  • Store the generated artifacts and the build log for future reference.
  • Deploy the release. Perhaps this involves putting the installer on your website, sending out memos or press releases to people who need to know. Update release servers as appropriate.

This is a large topic tied intimately with configuration management, testing procedures, software product management, and the like. If you have any part in releasing a software product you really must understand and respect the sanctity of the software release process.

Friday, 18 September 2009

Code: Creating a framework for the iPhone

Summary
This article explains how to build your own framework for Apple's iPhone OS.

The problem

Apple's Xcode development environment does not let programmers create their own framework for use in iPhone OS applications. This has caused many iPhone developers great frustration, although the restriction is for fairly sensible reasons.

So why the restriction?

A framework usually contains a dynamically loaded shared library (and the associated header files to be able to access its facilities). iPhone OS keeps applications very separate from one another, and so there is no concept of a user-created dynamic library shared between applications. There is no central library install point accessible to the developer. Indeed, managing such a software pool would be rather complex on iPhone-like devices, and preventing developers from installing their own shared frameworks neatly sidesteps a whole world of painful shared library compatibility issues, and well as simplifying the application uninstall process.

Its one, fairly final, way to avoid DLL hell!

All applications may link to the blessed, system-provided frameworks. The only other libraries they may use must be standard static libraries, linked directly to the application itself.

Those of us who'd like to supply functionality to other users in library form are left at somewhat of a disadvantage, though. Most developers are used to the simplicity of dragging a framework into their application target in Xcode, and not worrying about header paths or link issues.

It's nowhere near as neat to have to provide a static library and a set of associated header files in a flat directory. This requires your clients to work out the installation in their application for themselves. It's not hard, but it is tedious. You've also got to ship a library version for each platform the developer will need (at the very least, an arm code library for use on the iPhone OS device itself, and an i386 build for them to use in the iPhone simulator).

It's clumsy.

But fear not, there is a way...


How to build your own framework

I've worked out how to create a usable Framework that you can ship to other iPhone OS application writers. You can ship libraries that are easy to incorporate into other projects, and can exploit the standard framework versioning facilities.

There is one caveat: the framework will not be a standard shared library, it will provide a statically linked library. But the application writer need not be concerned about this issue. As far as they're concerned everything will just work. We are using Apple machines, after all.

Here's how to do it:

1. Structure your framework's header files.

Let's say your library is called "MyLib". Structure your project with a
top-level directory called "Include", and inside that make a "MyLib" subdirectory. Put all your public header files in there.

To be idiomatic, you'll want to create an umbrella header file "Include/MyLib/MyLib.h" that includes all the other headers for the user's convenience.

Set up your Xcode project "Header Search Paths" build parameter to "Include".
Now your source files can happily #import <mylib/mylib.h> in the same way they'd use any other framework. Everything will include properly.

2. Put your source elsewhere

I have a "Source" directory containing subdirectories "Source/MyLib" and "Source/Tests". You can put your implementation files (and private header files) wherever you want. Just, obviously, not in the Include directory!

3. Create a static library target

Create an iPhone OS static library target that builds all your library sources. Call it MyLib, and by default it will create a static library called libMyLib.a.

4. Create the framework plist file

Create a plist file that will be placed inside your framework, describing it.

I keep mine in Resources/Framework.plist. It's a peice of XML joy that should look like this:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>English</string>
<key>CFBundleExecutable</key>
<string>MyLib</string>
<key>CFBundleIdentifier</key>
<string>com.MyLovelyDomain.MyLib</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundlePackageType</key>
<string>FMWK</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>1.0</string>
</dict>
</plist>
5. The magic part... build your framework by hand

Create a shell script to build your framework. I have a "Scripts" directory that contains it, because I like to keep things neat like that.

The first line is the canonical hashbang:
#!/bin/bash
There are two parts to this file...

5a. Build all the configurations that you need your framework to support

This must be at least armv6 for the device, and x386 for the simulator. You'll want these to be Release configuration libraries.
xcodebuild -configuration Release -target "MyLib" -sdk iphoneos3.0
xcodebuild -configuration Release -target "MyLib" -sdk iphonesimulator3.0

So that's our libraries built. Now...

5b. Piece it all together

With a little understanding of the canonical structure of a framework directory, our ability to write a plist, and the knowledge that putting a static library in the framework instead of a dynamic library works fine, you can create your framework like this (apologies that blogger has kinda killed the formatting here):

# Define these to suit your nefarious purposes
FRAMEWORK_NAME=MyLib
FRAMEWORK_VERSION=A
FRAMEWORK_CURRENT_VERSION=1
FRAMEWORK_COMPATIBILITY_VERSION=1
BUILD_TYPE=Release

# Where we'll put the build framework. The script presumes we're in the
# project root directory
FRAMEWORK_BUILD_PATH="build/Framework"

# Clean any existing framework that might be there already
echo "Framework: Cleaning framework..."
[ -d "$FRAMEWORK_BUILD_PATH" ] && rm -rf "$FRAMEWORK_BUILD_PATH"

# The full name of the framework we'll build
FRAMEWORK_DIR=$FRAMEWORK_BUILD_PATH/$FRAMEWORK_NAME.framework

echo "Framework: Setting up directories..."
mkdir -p $FRAMEWORK_DIR
mkdir -p $FRAMEWORK_DIR/Versions
mkdir -p $FRAMEWORK_DIR/Versions/$FRAMEWORK_VERSION
mkdir -p $FRAMEWORK_DIR/Versions/$FRAMEWORK_VERSION/Resources
mkdir -p $FRAMEWORK_DIR/Versions/$FRAMEWORK_VERSION/Headers

echo "Framework: Creating symlinks..."
ln -s $FRAMEWORK_VERSION $FRAMEWORK_DIR/Versions/Current
ln -s Versions/Current/Headers $FRAMEWORK_DIR/Headers
ln -s Versions/Current/Resources $FRAMEWORK_DIR/Resources
ln -s Versions/Current/$FRAMEWORK_NAME $FRAMEWORK_DIR/$FRAMEWORK_NAME

# Check this is what your static libraries are called
FRAMEWORK_INPUT_ARM_FILES="build/$BUILD_TYPE-iphoneos/libMyLib.a"
FRAMEWORK_INPUT_I386_FILES="build/$BUILD_TYPE-iphonesimulator/libMyLib.a"

# The trick for creating a fully usable library is to use lipo to glue the
# different library versions together into one file. When an application is linked
# to this library, the linker will extract the appropriate platform version and
# use that.
# The library file is given the same name as the framework with no .a extension.
echo "Framework: Creating library..."
lipo \
-create \
-arch armv6 "$FRAMEWORK_INPUT_ARM_FILES" \
-arch i386 "$FRAMEWORK_INPUT_I386_FILES" \
-o "$FRAMEWORK_DIR/Versions/Current/$FRAMEWORK_NAME"

# Now copy the final asserts over: your library header files and a plist file
echo "Framework: Copying assets into current version..."
cp Include/$FRAMEWORK_NAME/* $FRAMEWORK_DIR/Headers/
cp Resources/Framework.plist $FRAMEWORK_DIR/Resources/Info.plist
That's it. Run that script and you'll create a framework in "build/Framework". In there is MyLib.framework. This directory can be shipped to your external application developers. They can incorporate it into their iPhone OS applications like any other framework.

Congratulations, you are now a hero.


Other remarks

I have presented here the most basic structure of a shell file. My production version includes more robust error handling, and other facilties that are relevant to my particular project.

I also have a build script that automatically creates documentation for the framework that I can ship with it. Indeed, I have a release script that applies versioning information to the project, build the libraries, creates a framework, assembles the documentation, compiles release notes and packages the whole thing in a pretty DMG. But that's another story.

If calling scripts from the command line scares you, you may chose to make a "Run Script Build Phase" in your Xcode project to call your framework script. Then you can create a framework without having to creep to the command line continually.

In summary, the final file layout of my project looks like this:
....Include/
........MyLib/
............MyLib.h
............IncludeFile1.h
............IncludeFile2.h
....Source/
........MyLib/
............PrivateHeader.h
............ImplFile1.m
............ImplFile2.m
........Tests
............MyLibTest1.m
............MyLibTest2.m
....Scripts/
........framework.sh

I hope you have found this tutorial useful. Let me know what frameworks you manage to build.

Friday, 4 September 2009

Code: 97 Things Every Programmer Should Know

The development website for O'Reilly's latest project in it's "97" series has now been made public.

97 Things Every Programmer Should Know looks like it will be an interesting book. Edited by Kevlin Henney, it contains a series of 97 (I'm not quite sure why 97 exactly, its a fairly weak gimmick) very short, pithy pieces of advice aimed at software developers to help them craft better code.

I have submitted a number of entries to this project.

You can take a look at all the entries submitted so far, and perhaps contribute something yourself. The site is a Wiki, and is now open for collaboration by the wider programming community.

Wednesday, 19 August 2009

Code: How to spell svnversion in git

I've been increasingly using git to clone and work on remote Subversion repos, as it's tedious to perform VCS operations over the atlantic when your web connection is tantamount to IP over carrier pigeon.

One operation I require in order to write sane autobuild scripts is something akin to svnversion. I need it to work for me, in my git svn-cloned repo, and for the mortals in other offices using plain ol' svn.

Took me a while, and here's what I came up with.
#!/bin/bash

# Returns the svn version number of the current directory.
# Works in an svn working copy, or in a git svn clone of an svn repo.
get_svnversion()
{
alias git_svnversion="git svn find-rev `git log -1 --pretty=format:%H 2>/dev/null` 2>/dev/null"
SVNVERSION=`svnversion`
if [ "X$SVNVERSION" == "Xexported" -o "X$SVNVERSION" == "X" ]; then
git_svnversion
else
echo $SVNVERSION
fi
}

echo $(get_svnversion)
Of course, there are some unpleasant dependencies on the particular version of svnversion you have installed and it's output on a non-svn tree. You could probably make this a little more bullet-proof if required.

Oh, and before you ask: blogger ate my formatting.

Wednesday, 15 July 2009

How to move your iTunes media onto a new hard disk

I run iTunes on Mac OS. I'm at the latest version (8.2 at the time of writing).

I have filled a hard disk with 11,000 tracks, many videos, podcasts subscriptions, and applications. I like to keep a separate hard disk for all my iTunes use, distinct from the main drive. This drive contains the audio, video, and the iTunes database files.

Having filled up my media disk, it was time upgrade to a larger disk in the same machine. It's not too hard if you let iTunes manage your music manually, just follow this HOWTO.

However, I like to manage my files manually. I have a directory structure for files which separates them according to use (I use my media with programs other than iTunes, but you need iTunes to sync with iPods). iTunes permits, but doesn't like this.

Moving the iTunes media onto a new disk is hard because:
  • iTunes does not cope well with its database files being moved
  • I sync with a number of iPods and an iPhone. Don't want to lose sync with those devices.
  • The iTunes database file is a closed binary format, and not easy to edit.
(On the mac, at least) iTunes is clever enough to track file movements on the same disk. You can rearrange your media files, and iTunes won't get confused. This is true for HFS+ at least (perhaps iTunes is tracking files by ID rather than filename, I never worried how it works). However, iTunes cannot cope with files moved between disks, which makes migrating your iTunes database somewhat complex.

There are a number of good HOWTOs on the net for this kind of thing (see the references at the end), but they all didn't quite describe what I wanted to do exactly.

So here's my HOWTO. If you know any better versions of these steps, let me know.

Prerequisites:
  1. You have installed the new drive, and formatted it, etc.
  2. You can see new drive at same time as old drive.
Steps...
  1. Quit iTunes. Best not to have it updating the database or downloading podcasts whilst you're working on it!
  2. Copy entire contents of old media disk to new one, including all your media and the "iTunes" directory full of database files, application downloads, podcasts, etc.
  3. For safety's sake, I renamed the "iTunes" folder on the old disk to try to prevent iTunes from using it again and confusing matters. Based on iTunes cleverness, it might have spotted the rename magically - perhaps it would have been better to archive the old "iTunes" directory and delete the original?
  4. Go to new media drive. Look in iTunes directory. Open the "iTunes Library" file - it's iTunes's binary-format database. Open it with a text editor, select everything in it, and delete it all. Save the file. Ensure it has size zero.
  5. Open the "iTunes Library.xml" file in the same directory in a text editor (this is a XML human-readable version of most of the data in the database). Do a global search and replace for all "/Volumes/OldMediaDriveName" to "/Volumes/NewMediaDriveName" (changing those names appropriately, obviously).
  6. Start iTunes with the Option (alt) key held down. It asks for you to provide the location of a new iTunes data file. Select the new drive's iTunes directory.
  7. Get ready for a long wait. iTunes will now rebuild it's binary database from the XML file. For a large database this takes a VEEEEERY long time. I was waiting for over 30 minutes (on a PPC Dual G4, to be fair). Answer nagging questions as required. Trashing the iTunes database loses a lot of important, but non-essential information like album art associations, window setup, etc. iTunes will spend a while churning through all your albums trying to download cover art, work out volume normalisation, etc.
  8. Sort out the applications you have downloaded. Look under "Applications" in iTunes' source list and you'll see that iTunes hasn't picked and of them up. Drag all the "*.ipa" files from your "iTunes/iPod Games" and "iTunes/Mobile Applications" directories into the Applications view. They'll magically appear. Purchased applications will copy over fine.
  9. Sort out your podcast subscriptions. Sadly, they've been lost, too. Despite some tutorials descriptions, I can't easily find a way resubscribe. You'll have imported a load of podcast mp3 files which have genre "Podcast" - you can see them in your library with a simple search. The corresponding podcast subscriptions have been lost. You can therefore see the podcasts you were subscribed to; their feed URLs is available in the "Get Info" iTunes dialogue box for each file. You'll have to resubscribe manually.
That's it. We're all done. You can now sync your iPods fine. Sync settings are NOT lost, thankfully.

References