Friday, 27 June 2008

New workspace

I've moved 10 feet to the left in my office, and have occupied a new space. Since you spend so much time in a working environment, it's really important to ensure that it's as comfortable and productive an environment as you can make it.

In this instance, I wanted to make sure I had plenty of desk space to get two chairs in - to facilitate pair programming, to have enough space to access all the hardware devices I'd be working on, and to have good access to my keyboards and guitar - for when I get bored. (That happens fairly frequently).

A cute extra I got in the desk move was a pair of Samsung 214T 21" widescreen LCD monitors to replace my 17" ones. For any engineering manager reading this: it's really important to buy your developers decent screens. And it's also really important to ensure that they can have a dual head setup. Until you try working with two monitors, it's hard to appreciate just how useful it is. The amount of code, and associated control malarkey, that you can get on the screen at once is great. It is a real productivity aid.

These Samsung screens had a useful trick up their sleve - they can rotate. So I've set them up portrait (rather than landscape). Suddenly the amount of code I can view at once from a single file has almost doubled. It's awesome! Just don't ask me how much pain it was to persuade Linux to let me rotate the screens (when the net connection's down so you can't Google for the answer).

X is great. When it works.

For reference, here's my hit list for essential working environment features for productive code development:
  • A large desk space that you can get at least two chairs to easily for pair programming and working together.
  • High quality seating, so your back doesn't hurt after a day's work.
  • Decent specification computer(s) with at least two monitors.
  • Good monitors. Developers spend all day staring at these things. Make them good. And big.
  • Light: plenty of it. Natural light, desk lights, fairy lights.
  • A whiteboard to scribble ideas on. At every desk.
  • Ventilation. Seriously... Give me aircon, so I don't become drowsy though heat exhaustion, and give me oxygen so my brain can function.
  • The chance to personalise the space to make it feel "homely". I have several keyboards (of the musical variety), a guitar, and load blaring music (luckily I have a personal office space and understanding workmates).
  • Storage space to store books, possessions, and other kit.
  • Good coffee.
  • Toys. Seriously, developers are like kids - they like toys. Each developer should have plenty of cool toys (nice machines to work on, plus gadgets like Skype phones and the like to stroke their pleasure glands). And there should be nice R&R toys, too. Our kitchen area comes equipped with a pinball machine (this one) and air hockey.
I've also inherited a fish tank. I hope I don't kill the poor things!

Friday, 20 June 2008

Unit testing threads is hard (part 4)

In this next part of my occasional series on threading, we'll look at some of the classic problems that I've encountered when testing threaded code.

First, here are some facts:
  • Concurrent threads of code will execute differently on multiple CPU systems than on single CPUs with OS simulated multi-threading. Inter-thread synchronisation primitives will behave subtly differently causing the code to behave in different ways. This is a great way to expose subtle bugs when you least expect it.
  • CPU load (i.e. the number of processes running at one time, and the amount of work they are doing) can dramatically effect the performance of your process. What usually takes 10 microseconds could, sometimes, take 10 seconds, or even 100 seconds.
  • Without a rich set of thread primitives, and a good understanding of how to use them correctly, you will never be able to craft good unit tests for threaded code.
So, based on these, here are some classic mistakes in threaded unit tests:
  • Problem: Testing regularly occurring events that happen every N seconds by waiting Nx10 seconds and checking 10 events were fired. Result: A test that often gets the wrong number of events. Why: Asking a thread to wait for a period of time does not guarantee that it will wake up after that time exactly. You're at the mercy of the OS as to when your thread wakes up. If the computer is heavily loaded it might be a long time after you expected. More events may have fired by then.
  • Problem: Testing for a result that should be returned asynchronously by waiting for the event to occur, or for 'N' seconds to expire, because the result will "never" take N seconds to come back, right? Result: A test that fails whenever the test computer is heavily loaded. Why: If the computer is heavily loaded then its entirely possible that the background thread will take a long time to calculate the answer. Longer than N. The timeout will one day time out. I've seen this happen frequently on a machine with two or more builds running simultaneously.
  • Problem: Thread creation is not adequately managed in the code. There are races in construction/destruction of threaded objects. Result: Unit tests occasionally crash. Why: You can't safely construct test scaffolding around the threaded object unless you can control when and how the thread runs.
  • Problem: Tests that don't honour thread requirements. Calls that should only be made on a certain background thread are made on the foreground thread in the test because it should work OK. Result: Occasional crashes. Why: You did the Wrong Thing. If you shoot yourself in the foot, you should expect to hobble.
  • Problem: When objects are deleted in the unit test, background threads still exist that reference these objects. Result: Bang. Why: If the background thread fires the event after callback objects are destructed then you're effectively dereferencing a dangling pointer. Things go horribly wrong. This often is a subtle problem: frequently the unit tests run successfully, and the cleanup is fine. But if your test fails, the unit test framework signals this by throwing an exception. The exception causes objects to be destroyed as the stack unwinds. If the background thread does not get stopped as the stack unwinds, one unit test failure may manifest as a bizarre runtime crash. Ouch.
  • Problem: Testing threaded components as if they were not threaded. We abstract the threadedness out to make the tests easy and desterministic. It's what I'm preaching about in these posts! Result: The tests are easy to write, and never go wrong. Hurrah! However. the software component is still buggy as you have not tested it in the way it is normally run. Why: Go figure.
So we can see that when unit tests for threaded code are written badly, the tests can:
  • Behave erratically, sometimes working, sometimes not working.
  • Occasionally crash (segfault on Linux, for example)
  • Not actually test the threaded code in the way it is used in Real Life

Speaking in LA

I'll be speaking at an Numark/Akai/Alesis Engineering Summit in LA in late July. The talk I'm giving is "A Tale of Two Systems" and it should be great fun.

The amusing thing about this speaking trip is that I get back to the UK one day before I fly to Uganda. That'll be fun...

Monday, 16 June 2008

New article: Write less code!

The latest article in my "Professionalism in Programming" column is out now in C Vu magazine. It's called "Write Less Code", which does seem vaguely heretical for a software developer.

Here's a teaser:
It's a sad fact in our modern world that there's just too much code. I can cope with the fact that my car engine is controlled by a computer, there's obviously software cooking the food in my microwave, and it wouldn't surprise me if my genetically modified cucumbers had an embedded micro controller in them. That's all fine; its not what I'm obsessing about. I'm worried about all the unnecessary code out there.

There's simply too much unnecessary code kicking around. Like weeds, these evil lines of code clog up our precious byes of storage, obfuscate our revision control histories, stubbornly get in the way of our development, and use up precious code space, choking the good code around them.

Why is there so much unnecessary code? Perhaps it's due to genetic flaws. Some people like the sound of their own voice. You've met them; you just can't shut them up. They're the kind of people you don't want to get stuck with at parties. Yada yada yada. Other people like their own code too much. They like it so much they write reams of it. { yada->yada.yada(); } Or perhaps they're the programmers with misguided managers who judge progress by how many thousands of lines of code have been written a day.

Join the excellent ACCU to get C Vu and read the rest of that reveting article! And yes - that is my barnet on the cover :-)

Wednesday, 11 June 2008

Do you know your tools?

The crafty aspects of programming (as in, the aspects that are like a craft, not the aspects that are cunning like a devious weasel hatching a dastardly plot) require us to use tools to fashion our work. Like any other artisans, we're reliant on quality tools to help us quickly produce software masterpieces. You could produce an elegant sculpture using only an blunt nail file, brute force, and perseverance, but you'd be better off using a full set of sharp chisels, points, claws, and hammers. They sure do help.

Good tools are important, but we also need to know how to use them. To be really productive, we really need to know how to use them - so their use is second nature. When we achieve this, we don't have to get distracted by using the tools, and our attention can be devoted fully to the code we're crafting, not the tool we're driving.

A friend of mine plays keyboards in a band. Recently he bought himself a new instrument - a top-of-the-line monster machine that can make tea and toast, all while you play incredible keyboard licks using only one finger. It's got loads of functions, and there's a lot to learn about it. Even the simple functions are hidden amongst the complex array of knobs and buttons.

He took the keyboard out to a gig not long after having bought it, and it all went spectacularly wrong. The keyboard seemed to have a mind of it's own, wouldn't behave itself at all, and after a while stopped making sounds at all.

Convinced it was faulty, he arranged to have it shipped back to the manufacturer and repaired (under guarantee, naturally). But they sent it straight back. Nothing was wrong with it, he just had no idea how to use it. His lack of understand of his tool had caused the problem.

Not knowing his tool cost him in two ways: he performed badly when it mattered most, and he wasted time and money needlessly (it's not cheap to post one of those large beasts). When you're counting on them, it's really important to know how to use your tools.

Do you know how to use your tools? Properly? What tools are you using? Could you be more productive using them? Are there better tools you could be using right now?

Know your tools. Know what's in your toolbox.

Thursday, 5 June 2008

A !Zap colour scheme for Vim

It's common knowledge that Vi (well, Vim) is the best editor in the world, with no exception. I have used Vim for the majority of my programming career, and I love it. It's as much a way of thinking - a way of life - as it is an editor.

Before using Vim, when incarcerated on Acorn computers, I used an editor called Zap (or !Zap if you're that way inclined). It was a really powerful little beast, with great support for different filetypes and awesome syntax highlighting. It was the power of Zap that forced me to look for a similarly powerful editor... Vim.

Ever since my move to Vim I've been using a custom colour scheme that made Vim look more Zap-like. A few other people have used it over the years, but it's always been a few random lines pasted into my .vimrc file, rather than a proper Vim colourscheme.

Well, that is, until now. Vim users rejoice, here is my Zap colorscheme file for Vim. Save this as ~/.vim/colours/zap.vim and then add "colorscheme zap" to your ~/.vimrc file.


" =============================================================================
" Name: zap
" Purpose: Zap-like colour scheme
" Maintainer: Pete Goodliffe (
" Last change: January 2008
" =============================================================================

" Zap is an archaic editor from RISC OS (an archaic computer)
" But it was great.

" =============================================================================
" Preamble
" =============================================================================

set background=dark

hi clear

if exists("syntax-on")
syntax reset

let colors_name = "zap"

" =============================================================================
" Vim >= 7.0 specific colours
" =============================================================================

if version >= 700
hi CursorLine term=underline cterm=underline guibg=#111133
" hi CursorColoumn
" hi MatchParen
" hi Pmenu
" hi PmenuSel

" =============================================================================
" General colours
" =============================================================================

hi Normal guibg=Black guifg=White ctermbg=Black ctermfg=White
hi Cursor gui=none guibg=White guifg=Black ctermbg=White ctermfg=Black

hi Folded guifg=Orange guibg=DarkBlue
hi FoldColumn guifg=Orange guibg=DarkBlue

" The following values have never been set by my vimrc, but this these are the
" default values that I end up using...

"NonText xxx gui=bold term=bold cterm=bold ctermfg=4 gui=bold guifg=Blue
"LineNr xxx term=underline cterm=bold ctermfg=3 guifg=Yellow

"hi StatusLine xxx term=bold,reverse cterm=bold,reverse gui=bold,reverse
"hi StatusLineNC xxx term=reverse cterm=reverse gui=reverse
"hi VertSplit xxx term=reverse cterm=reverse gui=reverse
"hi Title xxx term=bold cterm=bold ctermfg=5 gui=bold guifg=Magenta
"hi Visual xxx term=reverse cterm=reverse guibg=DarkGrey
"hi SpecialKey xxx term=bold cterm=bold ctermfg=4 guifg=Cyan

" =============================================================================
" Syntax highlighting
" =============================================================================

hi Comment gui=none term=bold guifg=Green ctermfg=Green
hi Todo gui=bold term=bold guibg=yellow guifg=black
hi Constant term=underline guifg=#8dffff
hi Identifier gui=bold term=underline guifg=#ffff60
hi Function gui=none guifg=#ffcabd
hi Type gui=bold term=underline guifg=#aaaa00
hi Statement term=bold guifg=#ffff20
hi PreProc term=underline guifg=#00bbff ctermfg=LightBlue
hi Special term=bold guifg=Orange

hi Search gui=underline guifg=#fe0000 guibg=#553333
hi QtClass guifg=Orange ctermfg=LightBlue
Oh, and if you're looking for an excellent version of Vim that runs on Mac OS, the check out MacVim.

Tuesday, 3 June 2008

Unit testing threads is hard (part 2)

In a previous post, I started a constructive moan about how difficult it is to write unit tests for threaded code. This is a large and complex topic, and I'll going to tackle it here in a few separate postings. All going well, some answers should emerge by the end of the process.

Why is it hard to test threaded code?

Let's start by working out why testing threaded code is hard. In the previous post, I stated that it is hard to unit test code that specifically does any of these:
  • Spawns a new thread
  • Waits for a thread to finish
  • Synchronises with another thread
I'll add one more item to that list:
  • Performs events after a period of time
It's hard to tests threaded code in general, but these are specifically complex points in even the simplest threaded code. (No doubt there are other particular pain areas; I'm sure that this list will grow as my pain threshold grows.)

Aside: What is a unit test? It's important to understand what a unit test is; some developers get this subtly wrong. There are many forms of tests, most of which can be automated and run during the build processes as an instant validation of the code under construction. However, not all of them are unit tests.

Unit tests exercise individual sections (or
units) of code. To do this the code's connections with the outside world are replaced with stub or mock components that represent "real" components, but that are drivable from within the test harness. The unit test therefore only tests the small section of code in a controlled environment. It does not test the code's integration into the entire software system. Clearly, unit tests cannot therefore interface over network connections, or with databases (those connections would be components themselves with stub-implementations for testing purposes).
We use stub- and/or mock implementations of external interfaces in our unit tests to ensure that the unit operates in a deterministic environment. And we avoid access to potentially non-deterministic entities (databases, networks, filesystems) to ensure that our tests are simple, valid, reliable, and repeatable.

Then along come threads and rain on that little reliable, repeatable parade.

We most often use threads to split up tasks that can be run concurrently, in order to increase program performance. This is great, as long as the threads do not need to interact. Very scalable systems can be built this way. But if the threads must interact, very non-scalable systems can be the result. Unfortunately, good thread practice is way outside the scope of this article.

When you have multiple threads of control running in parallel, it becomes harder to reason about the correctness of your application. This code:

void a() { a1; a2; a3; a4; }
void b() { b1; b2; b3; b4; }
Clearly runs operations in the order a1, a2, a3, a4, b1, b2, b3, b4. If a() and b() were launched concurrently in separate threads, then you might see them run in order a1, b1, a2, b3, a3, b3, or perhaps a1, a2, b1, b2, a3, a4, b3, b4, or any other order. In fact, the only thing you can (probably) guarantee is that a1 will happen before a2, a2, before a3 and so on... (and that will only hold if your optimising compiler hasn't taken it upon itself to reorder your code statements to generate "faster" code).

By spawning a thread, we specifically release some control over the execution of our program, which inevitably makes it much, much harder to unit test. We can no longer run the "unit" in a carefully controlled environment.

Interacting threads intertwine in a way that is largely random, one run of a unit test for a threaded component may be very different from the next. This is because thread behaviour changes considerably with:
  • the physical attributes of the machine (e.g. real parallelism from multiple CPUs or multiple cores in one CPU vs simulated parallelism from OS-level threading)
  • the load of the machine (when the code has less time to run in because other applications/processes are hogging the CPU the thread behaviour can become very lumpy and unpredictable - sometimes it is an interesting test for your threaded app to run it on a loaded machine)
  • the speed of the CPU(s) in the machine, and of the memory bus/network/disks
  • the nature of the operation running on a background thread (is it CPU intensive, heavily contending with the "main" thread of control for CPU cycles, or is it an IO-bound batch-process, mostly blocking on data throughput)
  • the way the wind is blowing (who knows which thread unblocks and gets a chance to run this time the test executes?)
Threads bugs are hard to find

Apparently, breaking up is hard to do. I'd agree: breaking up thread behaviour so it's testable is practically impossible. Threads interact in a very un-repeatable way, and problems stemming from bad thread interactions are remarkably hard to find.

Most of the time your code would operate perfectly, but once in a blue moon you get a data-race, a deadlock, or a timing error. In fact, it's practically impossible to write a unit test that proves that none of those conditions can occur.

When testing single-threaded code we must consider the tests' code coverage; whether every line of code and every condition has been covered. In a multi-threaded environment the problem explodes. We must consider coverage in terms of every possible interaction of every line of code, The threaded environment is akin to shuffling a deck of cards before running the threads - each time you deal out the program instructions you'll get a different set of instructions. How can you be sure that each of those sets of instructions results in the same - or at least in a correct - result?

What have we learnt so far?

In later postings we'll look at how to write tests for threaded code, but until then, here are two very helpful tips for writing, and testing threaded code:
  • Avoid writing code that spawns another thread unless you absolutely have to
  • Avoid threaded code that has to do something externally visible to other threads other than at the beginning/end of the thread's execution
  • Do not tie the thread-spawning aspect of the code from the code that runs on that thread. For example, ensure that the algorithm is neatly encapsulated and testable in isolation in a single-threaded environment. Then, if necessary, write a threaded component that employs that algorithm on it's thread.

More will follow...

Monday, 2 June 2008

All code and no play (ACCU magazine covers)

All code and no play makes jack a dull boy. Or something like that.

I like to spread out and do a few different things. Using your brain in different ways helps you to be a better programmer.

I love writing; it stretches a different part of the brain that helps me to reason about structure, clarity, and presentation. And it's a real buzz to see your name in print. I spend a lot of time playing music (I play keyboards, am still "learning" guitar, and enjoy song writing). And I enjoy graphic art - I produce a magazine covers and a load of fliers for events.

The latest CVu and Overload magazine covers are now at the printers. One less thing to do this month :-)

The Crossing 2008

I've just got back from an awesome weekend at C3's annual leadership conference: The Crossing.

It was great fun, but tiring work - I was playing keyboards in the band.

Kudos to all the people involved that made it such an awesome event, from the organisational brains behind the scenes, to the speakers (which included Jeff Lucus, Paul and Priscilla Reid, and Steve Campbell), to the band and choir, the PA/rigging guys, the stewards, refreshments team, and every single other person. It takes an immense effort to put on an event like this, and it was an honour to be a part of the team.

Now to get some sleep (or tackle the next threading problem)...