Friday 20 March 2009

Solo Software Development

Many people have been asking me how life in my little world of Solo Software Development is going. So I figured I'd best just write about it here and be done with it.

The prologue: those that know me will recall that just before Christmas my company laid off my entire office. Except me. I was the sole survivor. Left holding the bomb, as it were. We were using XP to develop cool C++ code.

It's an interesting experience moving from work in a highly social collaborative software development shop to a bloke on his own.

But I'm really enjoying it.

Working on your own presents a developer with a new and different set of challenges. I'm aiming to keep up a very high quality of code, and wherever possible will maintain the XP processes we used for development. Is that possible, and how much has my XP experience changed?
  • Pair programming was perhaps the most obvious casualty. Clearly, in a team of one there isn't any pairing. We didn't religiously pair program, but wherever pairing was missing we would always perform a code review. I really, really miss this discipline. The best substitute I can come up with is to carefully go over every checkin prior to committing.
  • Planning development with stories, tasks, etc is still very useful. I can keep the planning game, and I'm maintaining it all in the same xplanner system we used previously. I've switched down from two week to one week iterations, as that seems to fit my one-person efforts better.
  • Unit testing. I'm keeping up with this. Without the accountability of other coders sharing the codebase, TDD definitely requires more discipline. It's very tempting to skip writing the tests and just code form the hip. But I've enjoyed the benefits of automated tests for so long, I simply couldn't consider living without them. When I skip tests the code is often incomplete, buggy, or badly designed. My unit tests are the closest to team accountability I have!
  • Morning standup meetings. These provided a great heartbeat to the weekly development effort, and I miss them. Without this meeting I miss the chance to assess my progress and discuss if I'm still on track or doing the right thing. I've tried to set myself daily goals and reassess each morning, but honestly it's not going too well.
Being a one-man-band has given me the liberty to make some sweeping code changes that had been put off for a long time. With the whole team in place as they were too intrusive to other people's work, but they'd been annoying me for long enough that it feels great to purge my demons. I've had the chance to perform a lot of overdue tidying up of the code, the design, and restructuring of the file structure and build system.

There are many pros to my situation: for a start, I still have a job. And I get to work on good code, and I get to ensure that it stays good. Oh, and no one complains when I play music very loud.

There are many cons to Solo Software Development, many of which I will have to adjust to or work out ways to deal with:
  • No one to talk to
  • Motivation is harder to get; you have less accountability
  • No chance to learn tips and techniques from peers
  • Trips to The Wrestlers have taken a hit
  • Getting "stuck" on a gnarly problem halts the entire development effort, not just one person's tasks.
And many things are just plain different. For example:
  • Version control discipline is different. I needn't always create a branch for a piece of work. No one else is going to mind when I shuffle stuff around on the trunk. (That doesn't mean branching isn't valuable)
  • I'm spending longer on the phone to the managers in the US. My working hours are consequently changing.
  • I'm liaising with people I'd not have talked to before - service providers, estate agents, admin and finance people.
I'll be honest, work on my own isn't ideal, but it's not turned out as bad as I expected it to.

It looks like I'll be moving on to some different projects shortly. This will serves to highlight how good our old code was, but it's good not not get stuck in a ghetto.

And I'll be moving office soon. Sadly, to something less palatial.

Thursday 12 March 2009

Code: File structure matters

When working on your software project, how much thought do you give to the shape of your project's file structure?

Many people plug away within their IDE of choice (most commonly Visual Studio or Xcode, but perhaps Eclipse or somesuch). They accept the default file structure imposed on them by the tool. Depending on the size and nature of your project this may be sensible. It may not.

I've seen gargantuan projects managed like this with thousands of source files in one directory. If the IDE provides a mechanism for drawing logical hierarchies of files, but leaves the filesystem in a mess does it matter?

Yes, it does.

Good file structure makes working with your project easier:
  • Managing/viewing code changes in source control is sensible, especially when you're using tools outside the IDEs limited set.
  • Introducing a newcomer to your project and showing them the structure is easier.
  • Using other build system is easier, perhaps significant when porting the code to another platform.
  • Navigating around the project outside of an IDE is easier.
  • It's easier to see old files that are no longer relevant and that can be deleted.
  • Keeping a tidy house encourages code quality in many other areas.
At the most basic level, I try to maintain these principles for file naming:
  • The file name matches contents of the file. Essentially, there is only ever one class per file, and the file reflects that class name. E.g. ClassName.java, or ClassName.h and ClassName.cpp)
  • Directory structure matches namespace/package names. E.g. Foo::Bar::ClassName is in the file "Foo/Bar/ClassName.h"
  • Tests are kept in consistently named files beside the corresponding source files. I create a tests subdirectory with a test file of the same name as its testee. E.g. "Foo/Bar/ClassName.cpp"'s unit tests are kept in "Foo/Bar/tests/ClassName.cpp"
  • For libraries, I keep the public header files in a covariant source tree. Internal "private" header files are therefore clearly distinguished from public heeaders. E.g. "include/Foo/Bar/PublicClass.h" with implementation in "lib/Foo/Bar/PublicClass.cpp".
The naming and location of derived built files (e.g. object files or java bytecode) is another thing. These days I like to keep another covariant tree containing object files. The trees are separates by target platform name and build type (release/debug/etc).

What rules do you follow to name and structure your software project? Do you know how to configure your development environment to change the default locations of files? Do you think this matters at all?

Software: Dropbox

This week I was given a referral to try the beta version of Dropbox. I was impressed.

Dropbox is a simple file synchronisation service for multiple computers. It's your granny's rsync, if you like. Installation is simple, as is signup. You point the software at a magic folder that it will keep in sync with every other computer tied to your Dropbox account. It's simple, fast, and works well.

The free version offers 2G of space. You can pay for more, if you need it. If you're not on a machine with Dropbox installed there's a web interface for you to access your files.

It's working really well for me. I'm working on my presentation for this year's ACCU conference, and Dropbox is making my life easier. Rather than carry around the presentation with me to edit at work or at home, I just save it in my Dropbox folder, and the latest version is ready for me to edit wherever I am.

Check it out.

Toys: Alesis MasterControl



My new toy arrived a few days ago: an Alesis MasterControl. This is little baby is a FireWire control surface and audio interface for controlling your DAW. My weapon of choice is an old friend, Cubase. So far, the pair are getting on really well together.

It's taken a while for Alesis to release the MasterControl, but from my initial experiments it's been worth the wait. It's a well built unit and very capable. It's not bad looking either. Very good value for money.

You get 8 channel strips each with a touch-sensitive motorised fader, continuous pot and select/record arm/solo/mute buttons as well as a motorised main mix fader. You get dedicated transport control buttons and a jog/shuttle wheel with cursor keys and zoom/scrub buttons, plus a bunch of assignable buttons for skipping around your project and setting markers, etc. You get 2 XLR inputs with preamps, phantom power, and inserts, plus another 6 line inputs. There's MIDI and digital input, too. There are three stereo outs, two headphone mixes, a talkback facility and more.

First impressions with Cubase were good. I inserted the channel strips for my software, installed the drivers, pointed Cubase at the MasterControl and everything just worked. Manual schmanual. Touching a fader selects a channel, which is a neat little trick. The user experience is very smooth. If it was a dedicated Cubase controller some of the button layout could be better, but since this drives many software packages I can understand the compromises made.

There are things it doesn't do - the LCD screen is small compared to, say, a Mackie Control. But compare the prices. Indeed, compared to all the other alternatives from the likes of M-Audio, Yamaha, Mackie and Digidesign this is a very competitive product in terms of the feature/price point.

It's definitely been a worthwhile investment, and I'd recommend it.

Find more of my MasterControl photos here. And more info on the Alesis website here.

Wednesday 11 March 2009

Safari 4 vs a Netgear router: Safari won.

It's true, I am am a technical magpie. I can't help but gather shiny new things. To put on my shiny MacBook Pro, Apple recently released a shiny new version of Safari.

Well, Safari 4 is technically a beta, but it's shiny and new so I just had to download and install it. Must... click... install...

Most people seem to think Safari 4 is marvelous, when they're not arguing over the position of the tabs (tabs in the title bar, that's heresy!). And indeed, it is. It's quite fast, and the new pretty shiny things it provides are both pretty and shiny. If idiots can't find where they've hidden the "stop" and "reload" buttons, then it serves them right. All good stuff.

Almost.

Sadly, Safari 4 has lead to a networking fiasco chez Goodliffe. After a few minutes of Safari use, my laptop now stops accessing the internet. Well, at least it fails to complete any http or rtsp traffic. Pinging IP addresses works OK, and my VPN connection can be started and stopped (the VPN connection uses a fixed IP address). DNS lookups also fail. Traceroutes work.

It's taken me ages to track down the problem. I discovered that my trusty Netgear FWG114P wireless router is a feature-packed network device that can insulate me from networking problems. Including denial of service attacks perpetrated by Safari on my laptop... Safari 4's speedy page fetching provokes the router into thinking it's a hacker's SYN flood. The computer is ceremonially booted from the network for a while as penance.

The solution, if you suffer this problem, is to go to the Rules page on the router's web-based admin interface, and disable Block TCP flood and Block UDP flood. Then hope that this is something Apple will fix, and you'll remember to switch the features back on next time.

That's a short description of a problem that's taken me ages to track down and sort out.

Apples, eh? They Just Work.

Article: This "Software" Stuff

The third installment in my mini series This "Software" Stuff has been published in the March issue of ACCU's CVu magazine.

Read it to find out how software is like child's play and a chore.

This issue of CVu was guest edited by Roger Orr, who has done a sterling job.

The cover I produced this month includes some top-secret implementation drawings of an embedded product's USB driver. Bonus points if anyone can work out which chip it's for!

Saturday 7 March 2009

It's not unusual

They say that some people see the glass half full, some see it half empty. But most programmers don't see the glass at all; they write code that simply does not consider unusual situations. They are neither optimists not pessimists. They are not even realists. They're ignorists.

When writing your code don't consider only the thread of execution you expect to happen. At every step consider all of the unusual things that might occur, no matter how unlikely you think they'll be.

For example:

Errors

Any function you make a call to may not work as you expect.
  • If you are lucky, it will return an error code to signal this. If so, you should check that value; never ignore it.
  • The function might throw an exception if it cannot honour its contract. In this case, ensure that your code will cope with an exception bubbling up through it. Should you catch the exception and handle it, or allow it to pass further up the call stack? If you allow it to rise upwards does your function leak resources or leave the program in an invalid state in the process?
  • Or the function might return no indication of failure, but silently not do what you expected. You ask a function to print a message; will it always print it? Might it sometimes fail and consume the message.
Always consider errors that you can recover from, and write that recovery code. Consider also the errors that you cannot recover from; the storage failures or running out of critical resources. Even if you can't recover in these circumstances, write your code to do the best thing possible; don't just ignore it.

Threading

Too many programmers try to work in an idealistic single-threaded bliss. However, the world has moved around them to a complex, highly threaded environment. Reasoning about what might happen in your code in the presence of multiple threads is much harder.

Unusual interactions between pieces of code are staple here, and it's hard to enumerate every possible interweaving of code paths, let alone reproduce one particular problematic interaction more than once.

To tame this level of unpredictability, make sure you understand basic concurrency principles, how to ensure mutual exclusion in code blocks and how to decouple threads so they cannot interact in dangerous ways.

Understand mechanisms for the safe creation and destruction of objects in threads, and the correct primitives to reliably and quickly pass messages between thread contexts without introducing race conditions or blocking the threads unnecessarily.

Shutdown

We make great plans for how to construct our systems: how to create all the objects, how get all the plates to spin, and how to keep those objects running and those plates spinning.

Less attention is given to the other end of the lifecycle: how to bring the code to a graceful halt without leaking resources, locking up, or going wrong.

Shutting down your system and destroying all the objects is especially hard in a multi-threaded system. Objects that depend on inter-thread interactions can be particularly prone to subtle destruction problems: deadlock, dependency inversion and the like.

As your application shuts down and destroys its worker objects, make sure you can't leave one object attempting to use another that has already been deleted. Don't enqueue threaded callbacks that target objects which have been deleted on another thread.

Make sure that you thoroughly test your code for destruction issues as well as construction issues. Test it in an environment where you can control threading issues with mock objects, but also cover it with integration tests that glue real parts of the system together — the particular timings and order of operations there may highlight other shutdown problems.

The moral of the story

The unexpected is not the unusual. You need to write your code in the light of this.

It's important to think about these issues early on in your code development. You can't tack this kind of correctness as an afterthought; the problems are insidious and run deeply into the grain of your code. Such demons are very hard to exorcise after the code has been fleshed out.

Writing good code is not about being an optimist or a pessimist. It's not about how much water is in the glass right now. It's about making a water-tight glass so that there will be no spillages, no matter how much water the glass contains.

Wednesday 4 March 2009

Lifestyle changes

At the end of last year I got a new mobile phone and changed phone number at the same time. I'd deferred changing phone numbers for as long as possible since I expected it to be a massive hassle. It actually wasn't too painful; I don't use the phone that often and there weren't too many people who new my old number.

Of course there might be a few people that still have my old phone number (mental note: check whether the girls' school has my new phone number). But what I don't know people don't know won't hurt me, right? Or at least, I won't know that I don't know it hurts me.

So that bodes well for my next lifestyle change, then.

This week I have to move my email address. Move my email address? It can't be that hard can it? It's kinda like moving house, but with fewer vans and boxes.

This is an email address that I have been using for the last, ooh, more than ten years. Turns out it's a lot more personal than a phone number. Whilst I've been using my goodliffe.net domain as my main contact email address for some time now, the threat of an old email address actually disappearing (cthree.org is moving mail provision service and rationalising their addresses in the process) is a sobering thought.

When I consider how many services I've subscribed to with that address, how much software I've registered using that as a contact address, and how many products and suppliers might contact me using that address, I realise that if the old address suddenly goes poof! I'll be in a bit of bother.

So what online malarkey has my email address? Off hand I can think of:
  • Banks and credit cards.
  • Several professional organisations.
  • Service providers like electricity/gas and my ISP.
  • eBay.
  • PayPal.
  • Amazon.
  • Social networking places like: Facebook, LinkedIn, Myspace, etc.
  • Skype and MSN-ny services.
  • Many other shops (how can I remember them all?)
  • My internet domain registrations and DNS services.
  • Various product subscriptions.
  • My software registrations.
  • My book publishers.
  • My friends.
And I'm sure I'm missing a load of stuff there. Frustratingly, if the old address is disabled then it's much harder to move some of these; many services require access to the old address in order to change to a new address.

I hope that I remember everything. And I'm left wishing there was a decent way to track who has your contact details.

Bootnote: if you have my old cthree.org email address in your address book then please update it to my new goodliffe.net address. You can probably guess it! Normal service will be resumed shortly.

Monday 2 March 2009

Article: The Ghost of a Codebase Past

My latest installment in the Code Craft column, The Ghost of a Codebase Past, has been published in the current issue of Better Software magazine.

It's an encouragement to look back at your old code. Revisiting old code can be a frightening experience. But it's worthwhile; take a look to see how your technique has progressed, how your skills have grown, and what you can learn.

Read it here.

Speaking: ACCU 2009 (Legacy Code)

I'll be speaking at the ACCU 2009 conference, in Oxford UK. The conference runs from Wednesday 22nd to Saturday 25th April 2009.

Here's the summary of my presentation:
Legacy code - learning to live with it

Legacy code. You can't live with it. You can't live without it.

Well, you can't avoid it, at least. Spend long enough in the software factory, and you'll inevitably run into other people's old code. And of course, none of this old stuff is any good. It's nothing like the high quality software you craft. Pure tripe.

Let's be honest, sometimes you might even stumble across some of your own old code, and embarrassing as it is, you have to admit that you don't know how it works, let alone how to fix it.

This presentation will look at practical strategies for working with "old" crufty code. We'll see how to:
  • start working with a completely unfamiliar codebase
  • understand old spaghetti programming
  • make correct modifications
  • prevent bad code from causing more pain in the future
It should be a lot of fun.

If you're a developer and you've never been the the ACCU conference before, then I highly recommend it. It's excellent value for money, it has a great atmosphere, and you'll learn a tonne of really useful stuff.