How to move your iTunes media onto a new hard disk

I run iTunes on Mac OS. I'm at the latest version (8.2 at the time of writing).

I have filled a hard disk with 11,000 tracks, many videos, podcasts subscriptions, and applications. I like to keep a separate hard disk for all my iTunes use, distinct from the main drive. This drive contains the audio, video, and the iTunes database files.

Having filled up my media disk, it was time upgrade to a larger disk in the same machine. It's not too hard if you let iTunes manage your music manually, just follow this HOWTO.

However, I like to manage my files manually. I have a directory structure for files which separates them according to use (I use my media with programs other than iTunes, but you need iTunes to sync with iPods). iTunes permits, but doesn't like this.

Moving the iTunes media onto a new disk is hard because:

• iTunes does not cope well with its database files being moved
  
• I sync with a number of iPods and an iPhone. Don't want to lose sync with those devices.
  
• The iTunes database file is a closed binary format, and not easy to edit.

(On the mac, at least) iTunes is clever enough to track file movements on the same disk. You can rearrange your media files, and iTunes won't get confused. This is true for HFS+ at least (perhaps iTunes is tracking files by ID rather than filename, I never worried how it works). However, iTunes cannot cope with files moved between disks, which makes migrating your iTunes database somewhat complex.

There are a number of good HOWTOs on the net for this kind of thing (see the references at the end), but they all didn't quite describe what I wanted to do exactly.

So here's my HOWTO. If you know any better versions of these steps, let me know.

Prerequisites:

1. You have installed the new drive, and formatted it, etc.
   
2. You can see new drive at same time as old drive.

Steps...

1. Quit iTunes. Best not to have it updating the database or downloading podcasts whilst you're working on it!
   
2. Copy entire contents of old media disk to new one, including all your media and the "iTunes" directory full of database files, application downloads, podcasts, etc.
   
3. For safety's sake, I renamed the "iTunes" folder on the old disk to try to prevent iTunes from using it again and confusing matters. Based on iTunes cleverness, it might have spotted the rename magically - perhaps it would have been better to archive the old "iTunes" directory and delete the original?
   
4. Go to new media drive. Look in iTunes directory. Open the "iTunes Library" file - it's iTunes's binary-format database. Open it with a text editor, select everything in it, and delete it all. Save the file. Ensure it has size zero.
   
5. Open the "iTunes Library.xml" file in the same directory in a text editor (this is a XML human-readable version of most of the data in the database). Do a global search and replace for all "/Volumes/OldMediaDriveName" to "/Volumes/NewMediaDriveName" (changing those names appropriately, obviously).
   
6. Start iTunes with the Option (alt) key held down. It asks for you to provide the location of a new iTunes data file. Select the new drive's iTunes directory.
   
7. Get ready for a long wait. iTunes will now rebuild it's binary database from the XML file. For a large database this takes a VEEEEERY long time. I was waiting for over 30 minutes (on a PPC Dual G4, to be fair). Answer nagging questions as required. Trashing the iTunes database loses a lot of important, but non-essential information like album art associations, window setup, etc. iTunes will spend a while churning through all your albums trying to download cover art, work out volume normalisation, etc.
   
8. Sort out the applications you have downloaded. Look under "Applications" in iTunes' source list and you'll see that iTunes hasn't picked and of them up. Drag all the "*.ipa" files from your "iTunes/iPod Games" and "iTunes/Mobile Applications" directories into the Applications view. They'll magically appear. Purchased applications will copy over fine.
   
9. Sort out your podcast subscriptions. Sadly, they've been lost, too. Despite some tutorials descriptions, I can't easily find a way resubscribe. You'll have imported a load of podcast mp3 files which have genre "Podcast" - you can see them in your library with a simple search. The corresponding podcast subscriptions have been lost. You can therefore see the podcasts you were subscribed to; their feed URLs is available in the "Get Info" iTunes dialogue box for each file. You'll have to resubscribe manually.
   

That's it. We're all done. You can now sync your iPods fine. Sync settings are NOT lost, thankfully.

References

• Apple document: how to move your iTunes Music folder. This doesn't move the iTunes database, as far as I can see. http://support.apple.com/kb/HT1449
  
• HiFi's blog: http://hifiblog.com/past/2006/05/11/howto-move-your-itunes-music-while-preserving-library-data-when-you-dont-let-itunes-manage-your-music-library/ http://hifiblog.com/past/2006/05/11/howto-move-your-itunes-music-while-preserving-library-data-when-you-dont-let-itunes-manage-your-music-library/. This is PC-centric, and the closest guide yet. But it's not entirely accurate (iPod applications aren't covered, and it hints that you can copy podcast mp3 files into the podcast "source" which isn't possible - on the Mac, at least).
  
• Lifehacker http://lifehacker.com/238296/ultranewb--how-to-move-your-itunes-library-to-an-external-drive Requires you to consolidate your library and lose your directory structure.

Writing: Improve code by removing it

The July 2009 issue of ACCU's CVu magazine is about to land on doormats over the world. It contains my latest Professionalism in Programming column entitled: "Improve code by removing it".

It does what it says on the tin.

When I first wrote it, the article was really ropey. Then I trashed half of it, and it looked a lot better. I took out a couple more paragraphs and it was almost perfect. Then I took out the rest of it, leaving just a title and the author bio. Now it's marvelous.

(Becoming a) Git

I wanted to learn something new. I hadn't had much exposure to distributed version control systems. So took the plunge and installed git. Throwing caution to the wind, I relied on it immediately for critical project work. It was an interesting, and not entirely unpleasant experience.

I chose git for a couple of reasons:

people I knew had been using it, and gave me favourable reports

it has good svn (Subversion) integration

I know people who favour bzr (Bazaar). However, git seems the more powerful puppy, and the one that might teach me more overall.

So far, I think that git is a very, very good tool. However, even though it's becoming more mature, it is not a friendly beast and not for the timid.

Becoming distributed

There are plenty of good articles floating around the net that describe the advantages of DVCS over the traditional centralised model. It makes a lot of sense. Even so, centralised version control isn't going anywhere soon.

The main advantage of git for me is the svn integration. My project's repository is held on the other side of the Atlantic and I'm attached to it by a thin wet string, so access times to the repository are pitifully poor. Running something like git provides me with a local mirror so query operations are far faster, and I have the ability to make "local" checkins that are versioned but not yet pushed up to the central svn for public consumption.

Both of these are neat tricks.

Installation

I've installed git on Linux, MacOS and Windows. Naturally, the Linux install was the easiest. I pulled it in through Kubuntu's package manager, and everything worked swimmingly.

The Windows port is interesting. Since Windows is not sufficiently Unix-like to run Git in any sane way, the nice Windows distributors ship with a minimal bash environment. For this old Unix-head it's a wonderfully useful thing, and saves me reaching for cygwin so much. It might be a bit of a bodge, but I like it.

I'm doing most of my work on Mac OS at the moment. There are a few ways to get git on the mac, but the Git on MacOS installer project seems to most sensible (at least, at first glance). It works well enough, however, its still not running perfectly for me. The svn integration is bust. The git svn dcommit script doesn't complete correctly. After each subversion commit the tool needs to do a git svn rebase. However its internal script paths are incorrect and this always generates an error. So you have to manually git svn rebase to sort it all out. If you forget to do this all sorts of chaos ensue as repos get out of date, and not all of your changes propagate upstream.

Using git

Like most DVCS you certainly have to have some kind of idea what's going on before you dive straight into git usage. Git has a quite steep learning curve, and it's documentation is still not at the same level as other tools, no matter what you may hear from other people. Many problems must be resolved by Google searches rather than looking in the "git book".

General git workflows are simple and pleasant, though. To this old Subversion-head, the idea of staging the changes you will make prior to checking them in at first seemed clunky. However, after a few commits I have really come to like the workflow. And the fact you can cherry pick individual parts of a file to commit is very neat indeed.

The git stash is also a cute feature, allowing you to temporarily park the changes your working on (effectively in a short-lived temporary branch), to do something else, and then to reapply your changes once you're ready to come back to them.

Tool support is a lot sparser than other version control systems. The command line distributions all ship with a Tcl/Tk application called gitk which is remarkable useful and powerful, albeit crap-to-look-at in an early 1980s stylee. On the mac there is gitx which is cute, but not quite as powerful as gitk.

Complex merges seem harder to resolve in git than other systems, but this might entirely be because of a lack of understanding on my part. There is a lot more power under the hood, that for sure. But it makes harnessing it a struggle. But this is a feature: git was not designed for idiots.

It's discomforting to come from a place where you know your version control tool inside out to a place of relative ignorance. But it's Good For You to make this jump every now and again. It puts hairs on your chest. (If you're female, you might not want to do this too often, then.)

SVN integration

Apart from the Mac install issue, I've mostly found using git as a local svn mirror to be remarkably effective.

I have found, however, that its best to keep each git repository separate, each a clone of the main svn repo. I have a number of machines which I build the code on. I'd initially hoped to make one svn clone repo, then clone repos on the other machines based on that one git clone. The clones work OK, but pushing back the changes appears to cause all sorts of confusion and lead to some bogus git svn dcommits at the top of the git tree.

This problem became so bad that I gave up on the idea of a fan-out repository structure, and just cloned each repo from svn individually on each machine. This seems lumpy, and does mean there's more trans-atlantic svn traffic than I'd like. I believe that Bazaar is much better in this respect, but would like to hear it confirmed by someone more knowledgable.

The future is bright. The future is git.

I've had a few hiccups along the way, but I'm happy enough to keep going with git. There are plenty of advanced use cases I've yet to encounter, and I dread and look forward to the pain in equal measure.

Git is the C++ of version control systems

My observation based on a few months of use is that git is the C++ of version control systems. This is ironic based on what Linus thinks of C++. However git is the powerful,-can-do-everything,-allows-you-to-shoot-yourself-in-the-foot-if-you-don't-know-enough-about-it version control system. People will be prejudiced against git because of its complexity. Some people will love it because of its complexity and power. Sometimes it's the best tool for the job, though.

Git: it's good for you. Just like C++.

Code Craft: Now available in Russian

I've just been sent the new Russian translation of my software development book, Code Craft. Interesting reading, indeed. If you understand Russian, that is. I have no idea how good this translation is; maybe any eager Russian readers could tell me?

As you can see, it's an interesting choice of cover for this translation. I wonder what it says about the Russian market? The Japanese and Chinese translations had bright and interesting covers, this is a rather somber affair.

Code Craft has been out for about two years now and has proved very popular. Nevertheless, it's frustrating that Amazon is not capable of showing the correct cover image in their listing.

Now perhaps it's about time I got to writing my second tome?

C++: How to say "#warning" to the Visual Studio C++ compiler

I encountered a piece of code that would no longer compile in a particular build variant of our product. I wanted to hack it out, and leave a compiler warning in it's place so we wouldn't lose track of the change.

It's easy in gcc. You simply say:

#warning FIXME: Code removed because...

So that's me sorted for Linux and MacOS. I'm happy in the Fun Place.

But in the Dark Place I was clueless. How do you say #warning to Visual Studio? You can happily write #warning in C#, but not C++.

Interestingly, the answer fell below my Google/Boredom Threshold (i.e. a web search didn't reveal the answer in sufficient few clicks that I lost interest). I just shoved in a run-time assertion instead. It'd do the job, but not as immediately as I would have liked.

Thanks to hashpling and the miracle that is Twitter, I now know the answer, and share it with you in the vein hope it might come higher up the Google rankings for those poor souls that follow me:

#pragma message ("FIXME: Code removed because...")

Needless to say, this is all tediously non-standard.

For bonus points

This still doesn't get us exactly the same behaviour as gcc's #warning. The message is produced, but without file and line information. This means that if you double-click the message in the VS IDE it will not jump to the warning in the editor window. It also means that build logs aren't much use.

Sam Saariste pointed this out, and here's the standard preprocessor mumbo-jumbo you have to jump through to get the exact warning behaviour I was after:

#define STRINGIZE_HELPER(x) #x
#define STRINGIZE(x) STRINGIZE_HELPER(x)
#define WARNING(desc) message(__FILE__ "(" STRINGIZE(__LINE__) ") : Warning: " #desc)

// usage:
#pragma WARNING(FIXME: Code removed because...)

Couldn't be simpler, could it?!

C++: #include <rules>

My blood is boiling. I'm seething. I'm going to go mad. Is that an overstatement? Possibly. But it's not far off...

You see, you can get so used to doing things the Right Way that when you stumble across someone doing it the Wrong Way it comes as quite a shock. And a frustration.

Lately I've been working on a C++ project with appalling include file discipline. It's embarrassingly bad. There is a well-known gentleman's agreement over include files. A #include <etiquette> if you like. Doesn't everyone know about this?

(Of course, many programmers will cite the fact that C and C++ require such "good practice", rather than ENFORCE it, as a weakness of the language. Perhaps they are correct, but that's a different story.)

The most basic of the #include <rules> are:

1. A header file must be self-contained and complete.

#include-ing it CANNOT produce a build error.

It does not require you to include more files first. Anything it references is either forward declared (if possible) or #included in that file.

If this is not the case, the user has to jump through innumerable hoops to work out exactly where the undefined bits of code live, and so what other files must be included first. Naturally, when those files do not include cleanly the task recurses painfully.

It's annoying. It's wrong. It makes extra work, reduces the code's self-understandability, and opens the door for errors (for example: the wrong files may be included; and it is possible to write header files that behave differently when different sets of include files are brought in beforehand).

2. Include files internally prevent problems from multiple inclusion

The canonical format for a header file is:

#ifndef SOME_WELL_CHOSEN_RANDOM_UNIQUE_NAME

#define SOME_WELL_CHOSEN_RANDOM_UNIQUE_NAME

// any required #includes go here

// header file contents go here

#endif

That "unique" name should be well chosen, and usually based on the name of the include file in question. It should also include the name of the project, and possibly the name of the subsystem, too, in order to avoid conflicts with header files you mnight be importing from a any third party libraries.

Many compilers provide #pragma once as a helpful way to write the same thing. However, this is NOT a standard C or C++ feature. It's cute, but it means that your code is not portable. Does this matter to you? It really ought to. The best advice is to used #pragma once AND the standard include guards together.

If you do not do this, then multiple includes of a header file will almost certainly generate build errors as the compiler sees re-definitions of the same code constructs.

Of course, some (very few, comparatively) headers are designed for multiple inclusion (e.g. by defining types based on some preset #define value that you establish prior to the #include). You CAN still define include guards for these headers using a little preprocessor ## macro string gluing.

Objective-C provides an interesting alternative to #include guards or #pragma once: the #import directive. This states at the include site that if the file has already been included, do not include it again. Otherwise go ahead and include it now.

It's cute, but it is just plain wrong. The calling site is NOT the place to specify if a file should be included one time only. This is a part of the contract provided and required by the include file, and so should be stated and enforced there. Also, import and include can be freely interchanged on the same files. The user should not be able to break the contract by accidentally including rather than importing the header.

Precompiled header files are another source of weirdness that I don't have the time to moan about properly here. They bring their own set of potential misuses.

3. Define things in ONE FILE ONLY

Do not have multiple #include files with different declarations of the same name.

This is a violation of the ODR (One Definition Rule). If the files define specific variations that are needed for different build configurations or targets, then still put them all in the same file. DO NOT leave it up to the #include-r to work out which of the files to use themselves. Any silly #ifdef determination logic should be INTERNAL to the header file, not external at the include-site.

It's obvious what will happen if you ignore this rule. Someone will accidentally include the wrong header version somewhere, and cause errors. Probably subtle, odd, and hard to track down runtime errors, too.

4. Differentiate "public" interface header files from implementation files

Many projects contain subcomponents with a small number of "public" header files defining their interface, and many internal .h files for implementation classes.

Differentiate these files.

This should be done by placing them in very different file locations. This will prevent a subcomponent interface user from accidentally including an internal header and using it as if it's part of the public interface.

There are more good practice rules than those. But these are the most basic and important ones.

The project I've been working on breaks these rules all over the place and it makes working with the code really, really complicated. Yes, I'm moaning. But we should know better than this by now. And we really should have developed sharper standard tools for this by now.

The greater the size of the project, the harder these problems are to fix. I'd love to dive in and sort the whole thing out in the current project, but I fear it will take a very, very long time. It's particularly hard as it's a large codebase that builds on several platforms with a number of compilers.

Tech toys: Synergy

I have four machines on the desk in front of me, each in active daily use. The upshot? A tangle of keyboards and mice on the work surface, and no space for my arms.

But I found the solution. Perhaps I'm a bit late to join the party, but it's here: Synergy.

Synergy is a wonderful application that shares one keyboard and mouse between many machines. It's as if you were using a multi-head display. But with many bodies, too. Synergy is relatively easy to install, although by default requires a little text file configuration. Nothing frightening for a techie, but a casual user might need a deep breath and a good run up.

Performance is really good (it does slow down when one machine is heavily CPU loaded or performs a lot of network access, but this is relatively rare). The application works on Linux, Mac OS, and Windows. I run the server on Linux, with two mac clients and a Windows client. The code hasn't been updated in about two years, but it seems every bit as good as it must have been two years ago!

And now? I have one keyboard. One mouse. Lots of free desk space. And deep joy.

Postscript

There are several Mac OS GUI front ends that might be useful floating around the ether. I've not used any of them, so the many or many not be any good...

SynergyOSX

SynergyKM

QuickSynergy