How to move your iTunes media onto a new hard disk

I run iTunes on Mac OS. I'm at the latest version (8.2 at the time of writing).

I have filled a hard disk with 11,000 tracks, many videos, podcasts subscriptions, and applications. I like to keep a separate hard disk for all my iTunes use, distinct from the main drive. This drive contains the audio, video, and the iTunes database files.

Having filled up my media disk, it was time upgrade to a larger disk in the same machine. It's not too hard if you let iTunes manage your music manually, just follow this HOWTO.

However, I like to manage my files manually. I have a directory structure for files which separates them according to use (I use my media with programs other than iTunes, but you need iTunes to sync with iPods). iTunes permits, but doesn't like this.

Moving the iTunes media onto a new disk is hard because:

• iTunes does not cope well with its database files being moved
  
• I sync with a number of iPods and an iPhone. Don't want to lose sync with those devices.
  
• The iTunes database file is a closed binary format, and not easy to edit.

(On the mac, at least) iTunes is clever enough to track file movements on the same disk. You can rearrange your media files, and iTunes won't get confused. This is true for HFS+ at least (perhaps iTunes is tracking files by ID rather than filename, I never worried how it works). However, iTunes cannot cope with files moved between disks, which makes migrating your iTunes database somewhat complex.

There are a number of good HOWTOs on the net for this kind of thing (see the references at the end), but they all didn't quite describe what I wanted to do exactly.

So here's my HOWTO. If you know any better versions of these steps, let me know.

Prerequisites:

1. You have installed the new drive, and formatted it, etc.
   
2. You can see new drive at same time as old drive.

Steps...

1. Quit iTunes. Best not to have it updating the database or downloading podcasts whilst you're working on it!
   
2. Copy entire contents of old media disk to new one, including all your media and the "iTunes" directory full of database files, application downloads, podcasts, etc.
   
3. For safety's sake, I renamed the "iTunes" folder on the old disk to try to prevent iTunes from using it again and confusing matters. Based on iTunes cleverness, it might have spotted the rename magically - perhaps it would have been better to archive the old "iTunes" directory and delete the original?
   
4. Go to new media drive. Look in iTunes directory. Open the "iTunes Library" file - it's iTunes's binary-format database. Open it with a text editor, select everything in it, and delete it all. Save the file. Ensure it has size zero.
   
5. Open the "iTunes Library.xml" file in the same directory in a text editor (this is a XML human-readable version of most of the data in the database). Do a global search and replace for all "/Volumes/OldMediaDriveName" to "/Volumes/NewMediaDriveName" (changing those names appropriately, obviously).
   
6. Start iTunes with the Option (alt) key held down. It asks for you to provide the location of a new iTunes data file. Select the new drive's iTunes directory.
   
7. Get ready for a long wait. iTunes will now rebuild it's binary database from the XML file. For a large database this takes a VEEEEERY long time. I was waiting for over 30 minutes (on a PPC Dual G4, to be fair). Answer nagging questions as required. Trashing the iTunes database loses a lot of important, but non-essential information like album art associations, window setup, etc. iTunes will spend a while churning through all your albums trying to download cover art, work out volume normalisation, etc.
   
8. Sort out the applications you have downloaded. Look under "Applications" in iTunes' source list and you'll see that iTunes hasn't picked and of them up. Drag all the "*.ipa" files from your "iTunes/iPod Games" and "iTunes/Mobile Applications" directories into the Applications view. They'll magically appear. Purchased applications will copy over fine.
   
9. Sort out your podcast subscriptions. Sadly, they've been lost, too. Despite some tutorials descriptions, I can't easily find a way resubscribe. You'll have imported a load of podcast mp3 files which have genre "Podcast" - you can see them in your library with a simple search. The corresponding podcast subscriptions have been lost. You can therefore see the podcasts you were subscribed to; their feed URLs is available in the "Get Info" iTunes dialogue box for each file. You'll have to resubscribe manually.
   

That's it. We're all done. You can now sync your iPods fine. Sync settings are NOT lost, thankfully.

References

• Apple document: how to move your iTunes Music folder. This doesn't move the iTunes database, as far as I can see. http://support.apple.com/kb/HT1449
  
• HiFi's blog: http://hifiblog.com/past/2006/05/11/howto-move-your-itunes-music-while-preserving-library-data-when-you-dont-let-itunes-manage-your-music-library/ http://hifiblog.com/past/2006/05/11/howto-move-your-itunes-music-while-preserving-library-data-when-you-dont-let-itunes-manage-your-music-library/. This is PC-centric, and the closest guide yet. But it's not entirely accurate (iPod applications aren't covered, and it hints that you can copy podcast mp3 files into the podcast "source" which isn't possible - on the Mac, at least).
  
• Lifehacker http://lifehacker.com/238296/ultranewb--how-to-move-your-itunes-library-to-an-external-drive Requires you to consolidate your library and lose your directory structure.

Writing: Improve code by removing it

The July 2009 issue of ACCU's CVu magazine is about to land on doormats over the world. It contains my latest Professionalism in Programming column entitled: "Improve code by removing it".

It does what it says on the tin.

When I first wrote it, the article was really ropey. Then I trashed half of it, and it looked a lot better. I took out a couple more paragraphs and it was almost perfect. Then I took out the rest of it, leaving just a title and the author bio. Now it's marvelous.

(Becoming a) Git

I wanted to learn something new. I hadn't had much exposure to distributed version control systems. So took the plunge and installed git. Throwing caution to the wind, I relied on it immediately for critical project work. It was an interesting, and not entirely unpleasant experience.

I chose git for a couple of reasons:

people I knew had been using it, and gave me favourable reports

it has good svn (Subversion) integration

I know people who favour bzr (Bazaar). However, git seems the more powerful puppy, and the one that might teach me more overall.

So far, I think that git is a very, very good tool. However, even though it's becoming more mature, it is not a friendly beast and not for the timid.

Becoming distributed

There are plenty of good articles floating around the net that describe the advantages of DVCS over the traditional centralised model. It makes a lot of sense. Even so, centralised version control isn't going anywhere soon.

The main advantage of git for me is the svn integration. My project's repository is held on the other side of the Atlantic and I'm attached to it by a thin wet string, so access times to the repository are pitifully poor. Running something like git provides me with a local mirror so query operations are far faster, and I have the ability to make "local" checkins that are versioned but not yet pushed up to the central svn for public consumption.

Both of these are neat tricks.

Installation

I've installed git on Linux, MacOS and Windows. Naturally, the Linux install was the easiest. I pulled it in through Kubuntu's package manager, and everything worked swimmingly.

The Windows port is interesting. Since Windows is not sufficiently Unix-like to run Git in any sane way, the nice Windows distributors ship with a minimal bash environment. For this old Unix-head it's a wonderfully useful thing, and saves me reaching for cygwin so much. It might be a bit of a bodge, but I like it.

I'm doing most of my work on Mac OS at the moment. There are a few ways to get git on the mac, but the Git on MacOS installer project seems to most sensible (at least, at first glance). It works well enough, however, its still not running perfectly for me. The svn integration is bust. The git svn dcommit script doesn't complete correctly. After each subversion commit the tool needs to do a git svn rebase. However its internal script paths are incorrect and this always generates an error. So you have to manually git svn rebase to sort it all out. If you forget to do this all sorts of chaos ensue as repos get out of date, and not all of your changes propagate upstream.

Using git

Like most DVCS you certainly have to have some kind of idea what's going on before you dive straight into git usage. Git has a quite steep learning curve, and it's documentation is still not at the same level as other tools, no matter what you may hear from other people. Many problems must be resolved by Google searches rather than looking in the "git book".

General git workflows are simple and pleasant, though. To this old Subversion-head, the idea of staging the changes you will make prior to checking them in at first seemed clunky. However, after a few commits I have really come to like the workflow. And the fact you can cherry pick individual parts of a file to commit is very neat indeed.

The git stash is also a cute feature, allowing you to temporarily park the changes your working on (effectively in a short-lived temporary branch), to do something else, and then to reapply your changes once you're ready to come back to them.

Tool support is a lot sparser than other version control systems. The command line distributions all ship with a Tcl/Tk application called gitk which is remarkable useful and powerful, albeit crap-to-look-at in an early 1980s stylee. On the mac there is gitx which is cute, but not quite as powerful as gitk.

Complex merges seem harder to resolve in git than other systems, but this might entirely be because of a lack of understanding on my part. There is a lot more power under the hood, that for sure. But it makes harnessing it a struggle. But this is a feature: git was not designed for idiots.

It's discomforting to come from a place where you know your version control tool inside out to a place of relative ignorance. But it's Good For You to make this jump every now and again. It puts hairs on your chest. (If you're female, you might not want to do this too often, then.)

SVN integration

Apart from the Mac install issue, I've mostly found using git as a local svn mirror to be remarkably effective.

I have found, however, that its best to keep each git repository separate, each a clone of the main svn repo. I have a number of machines which I build the code on. I'd initially hoped to make one svn clone repo, then clone repos on the other machines based on that one git clone. The clones work OK, but pushing back the changes appears to cause all sorts of confusion and lead to some bogus git svn dcommits at the top of the git tree.

This problem became so bad that I gave up on the idea of a fan-out repository structure, and just cloned each repo from svn individually on each machine. This seems lumpy, and does mean there's more trans-atlantic svn traffic than I'd like. I believe that Bazaar is much better in this respect, but would like to hear it confirmed by someone more knowledgable.

The future is bright. The future is git.

I've had a few hiccups along the way, but I'm happy enough to keep going with git. There are plenty of advanced use cases I've yet to encounter, and I dread and look forward to the pain in equal measure.

Git is the C++ of version control systems

My observation based on a few months of use is that git is the C++ of version control systems. This is ironic based on what Linus thinks of C++. However git is the powerful,-can-do-everything,-allows-you-to-shoot-yourself-in-the-foot-if-you-don't-know-enough-about-it version control system. People will be prejudiced against git because of its complexity. Some people will love it because of its complexity and power. Sometimes it's the best tool for the job, though.

Git: it's good for you. Just like C++.

Code Craft: Now available in Russian

I've just been sent the new Russian translation of my software development book, Code Craft. Interesting reading, indeed. If you understand Russian, that is. I have no idea how good this translation is; maybe any eager Russian readers could tell me?

As you can see, it's an interesting choice of cover for this translation. I wonder what it says about the Russian market? The Japanese and Chinese translations had bright and interesting covers, this is a rather somber affair.

Code Craft has been out for about two years now and has proved very popular. Nevertheless, it's frustrating that Amazon is not capable of showing the correct cover image in their listing.

Now perhaps it's about time I got to writing my second tome?

C++: How to say "#warning" to the Visual Studio C++ compiler

I encountered a piece of code that would no longer compile in a particular build variant of our product. I wanted to hack it out, and leave a compiler warning in it's place so we wouldn't lose track of the change.

It's easy in gcc. You simply say:

#warning FIXME: Code removed because...

So that's me sorted for Linux and MacOS. I'm happy in the Fun Place.

But in the Dark Place I was clueless. How do you say #warning to Visual Studio? You can happily write #warning in C#, but not C++.

Interestingly, the answer fell below my Google/Boredom Threshold (i.e. a web search didn't reveal the answer in sufficient few clicks that I lost interest). I just shoved in a run-time assertion instead. It'd do the job, but not as immediately as I would have liked.

Thanks to hashpling and the miracle that is Twitter, I now know the answer, and share it with you in the vein hope it might come higher up the Google rankings for those poor souls that follow me:

#pragma message ("FIXME: Code removed because...")

Needless to say, this is all tediously non-standard.

For bonus points

This still doesn't get us exactly the same behaviour as gcc's #warning. The message is produced, but without file and line information. This means that if you double-click the message in the VS IDE it will not jump to the warning in the editor window. It also means that build logs aren't much use.

Sam Saariste pointed this out, and here's the standard preprocessor mumbo-jumbo you have to jump through to get the exact warning behaviour I was after:

#define STRINGIZE_HELPER(x) #x
#define STRINGIZE(x) STRINGIZE_HELPER(x)
#define WARNING(desc) message(__FILE__ "(" STRINGIZE(__LINE__) ") : Warning: " #desc)

// usage:
#pragma WARNING(FIXME: Code removed because...)

Couldn't be simpler, could it?!