[Code] Building old toolchains on a new Linux (disabling Fortify)

We used to build our embedded Linux platform on an aged Debian Linux distribution (a 3-year-old vintage). Reently, I moved the build/release servers and my development workstation to Kubuntu 8.10. Shiny new, and still Linux. What could possibly go wrong?

Well, all the old toolchain buildy stuff stopped working.

The toolchain (which was build with the then-aged crosstool) exploded messily during program linkage, with aborts due to heap corruption or double-freeing. I've been scratching my head over this problem for a while.

The telling symptom in the crash backtraces was the function __fortify_fail just before the abort. The kubuntu glibc has been build with Fortify support (the "best" Fortify homepage I can find is a random geocities one; there must be a better one). Fortify is a memory access checker. Very useful to make sure your code is safe to run.

However, the old toolchain we rely on (specifically, the ar in binutils 2.16.1) seems to be badly behaved. It generates good code, but tramples on its own memory whilst it does it. So Fortify kills the process before an executable sees the light of day.

There are three solutions:

• Fix the toolchain
• Port the build system to a newer toolchain
• Switch off fortify checking

Sadly, I don't have time to dig into debugging an archaic binutils right now; and I'm happy from experience that the code it's generating is good. I've been working on migrating our build to a new set of tools and a newer Linux kernel, but if you've ever played with toolchains and build systems you'll know that it's easier said than done. In fact, it's a nightmare, and I just need to get the existing stuff building again right now.

So option 3, it is then. But without any adequate documentation, it's hard to work out out to disable fortify.

The trick is to define a preprocessor variable _FORTIFY_SOURCE=0. Then all glibc memory checking is disabled and the toolchain can continue to not work as well as it did in the past. A handy trick that's almost obvious when you know where to look for it.

Now, how you get that through crosstool to the binutils build stage is left as an excercise for the reader. Suffice to say that our old crosstool is now incredibly bodged and patched up to just work in 2009.

[Code] Improving code by removing it

Less is more. It's a quite trite little maxim, but sometimes it is really true.

One of the improvements I've made to our codebase over the last few weeks is removing chunks of it.

We'd written the software following XP tenets, including YAGNI (that is, You Aren't Going to Need It). Inevitably, in a few places we fell short.

I observed that the product was taking too long to execute certain tasks. Simple tasks that should have been near instantaneous. This was because they were over-implemented with extra bells and whistles that were not required, but that seemed like a good idea at the time.

So I've simplified the code, improved the product performance, and reduced the level of global code entropy by simply removing the offending features from the codebase. Helpfully, my unit tests tell me that I haven't broken anything else during the operation.

A simple and thoroughly satisfying experience.

So why did the unnecessary code end up there in the first place? Why did one developer feel the need to write extra code, and how did it get past review (or the pairing process?) Almost certainly something like:

• It was a fun bit of extra stuff, and the developer wanted to write it. (Hint: Write code because it adds value, not because it amuses you)
  
• Someone thought that it might be needed in the future, so it was best to code it now. (Hint: That isn't YAGNI. If you don't need it right now, don't write it right now)
  
• It didn't appear to be that big an "extra", so it was easier to implement it rather than go back to the customer to see whether it was really required. (Hint: It always takes longer to write, and to maintain extra code. And the customer is actually quite approachable. A small extra bit of code snowballs over time to a large piece of work that needs maintenance.)
• The developer invented extra requirements that were not documented in the story that justified the extra feature. The requirement was actually bogus. (Hint: Developers do not set requirements; the customer does.)

What are you working on right now? It is all needed?

The triumph of mind over matter

Nobody should let me loose on IT. I'm a programmer. Software, not hardware. Still, my primitive software brain has proved that mind can conquer matter. Eventually. Even if it only manages a somewhat Pyrrhic victory.

Example 1: Moving a tape drive

It was a simple requirement: move a working tape drive from a big shiny server into a ickle replacement server so I can continue to make backups. Here's the score sheet:

Time to completion of task:	Months of effort, on-and-off (over Christmas, if that makes any difference)
Extra hardware bought that wasn't necessary:	A new SCSI card.
Wasted money:	£30. Ish.
Reason hardware wasn't necessary:	The SCSI card wasn't the problem.
Reason new hardware was a particularly bad purchase:	It didn't have the right SCSI connector on it anyway. Who knew there are about a million different SCSI connectors? Good grief! Is this some kind of old archaic connector standard that's gone through many,. many revisions, or what? What.
Observation:	Software, not hardware. Don't let Pete near IT.
Does it get better than that:	Yes. Sadly.
Other extra hardware hardware bought that wasn't necessary:	A new tape drive
Wasted money:	Ahem.
Wasted money:	I'd rather not say
Wasted money:	OK, about £500.
Oops:	Indeed.
Reason hardware was a particularly bad purchase:	The tape drive wasn't the problem.
Even better reason the hardware was a particularly bad purchase:	I bought a tape drive with the wrong kind of SCSI connector. It didn't attach to either of the SCSI cards I had. Sigh.
So, mind over matter:	Yeah. I kept fiddling. Installed kernel drivers. New versions of the Linux mt command. Googled for Britain.
And?:	I fiddled with the SCSI card BIOS. A lot. And found a very well hidden random little setting that made it work.
Result:	Mind beats matter. And the wallet.

Example 2: Redirecting a phone call

Less expensive, this one. And marginally less frustrating.

Another simple requirement: get my office phone to forward to my mobile, so if I'm not in the office (skyving) or away from my desk (making tea, or pooing) I don't miss a call. The phone system can do this. We've done it before. It just wasn't me that did it. And there's no one else around these days. How hard can it be? Consult the score sheet:

Time to completion of the task:	A little under one month.
Observation:	Phone exchanges are nasty, complicated, stupid things.
Time to find exchange:	Quite some time. Turns out it's a nondescript box on the wall.
Time to find the exchange's UI:	A little while longer. There's not even a light on the box on the wall. Turns out you just configure it by dialing magic numbers on your phone. Very magic numbers.
Time to find the manual describing the magic numbers on the phone:	Not long. Once I figured I had to look for one.
Time to work out how to divert phone calls:	Some time. The manual is a fairly incomprehensible jumble of text and numbers. You have to know the phoney rhetoric. I'm an email/instant message kinda guy. Yeah, I have a mobile. But I'm not a corporate phone whore. Turns out it's "Call-Forward External" Obvious when you know what it's called. And can deal with the lack of contents page or index.
Time to work out how to switch it on:	2 seconds.
Time to work out how to set the number it forwards to:	1 hour. Turns out it forwards to Speed Memo *49. Whatever that is.
Setting Speed Memo *49:	Not too bad. Lots of random button pushes. A bit like playing Daley Thompson's Decathlon on my old Speccy.
Does the speed dial number dial my mobile:	Yes
Does the call forward work:	No
Why:	Beats me.
Keep playing with the phones for how much longer:	Oooh, weeks.
The epiphany:	Although speed dial *49 works fine without adding the magic '9' for an outside line, if you add one (and the speed dial no longer works as a speed dial) the call forward now works.
Result:	Skyving (or pooing) is easier.

So yeah, hardware and me don't really get on. Oh, and don't get me started on the wireless network. I've lost a lot of sleep over that one.

I'm having great fun cutting code, though. So it's not all bad.

What's in a name? (Or, in a job title)

There have been many shenanigans in my work life recently. I've gone from being one developer in a team of six to the developer in a team of one. There are lots of consequent changes. One of them is my job title.

Historically, I've not really given much thought to my job title. I'm a good software developer. I know it. My employers have known it. My CV speaks for itself.

In the previous team I was one of four software developers. These were very senior, very experienced, very talented developers. A crack team. And each person's job title was "Software Developer". I liked this; our team had no hierarchy and the job titles reflected this.

All too often a job title gives a false impression of your importance or significance to the organisation. Or a faux job-title change is given in lieu of a payrise to keep a developer happy. Too often, job titles lead to silly political games.

But now I'm on my own. I am the software team. And I've pushed for a job title change. I'm now a "Principal Software Developer". Why did I push for this change?

• It describes what I'm doing better. My working responsibility hasn't changed much. I'm still responsible for coding, for architecture and design, for integration, for the build system, for planning. However, I'm now the only one responsible for it.
• It lets others in my organisation (they're primarily in the US) know who I am and what I'm doing a little better. It makes my role clearer to them.
• It lets others I meet professionally know what I do better.
• When the job situation was getting complex, I got job offers with this title. This is what other companies would be calling me. If I want to change jobs in the future, having this job title now will help potential employers understand my current level of seniority.
• I specifically chose not to be a "Principal Software Engineer". I think the whole "Engineer" job title for a programmer is a bit of a minefield.
• Vanity. Hey, it does sound like a promotion. And it sounds kinda nice, doesn't it?

Are you happy with your job title? Does your job title reflect what you are doing? Does it reflect your position in your organisation?

Article: This "Software" Stuff, Part 2

The second part of my series looking at the odd "stuff" called Software is out now in the latest C Vu magazine. Yes, that's me grinning on the cover.

C Vu is one of the magazines published by ACCU. This issue has been edited by Faye Williams. She's done a great job - it's enormous.

Be still, my beating heart.

There's NAMM so blind as those that cannot see. But at least we have the guide dog that is the web to let us see what the NAMM is going on.

The Winter NAMM show, the major annual audio industry trade event has just kicked off. It's a popular time for all the MI companies to announce their latest wares. My company is exhibiting there, and I'm sure everyone loves our shiney new stuff. But I'm more interested in some of the other things I've seen announced...

It's a Keytar, Baby!

First up - and I think my heart has missed several beats - Roland have announced a new keytar, the AX Synth. They stopped making the AX-7 "shoulder mounted keyboard" eons ago, and these instruments have since entered keyboard geek folklore. Keytars are so bad they're good, and I love looking like a muppet playing one live. Now Roland have created a new one. And it looks pretty good. Well, visually it actually looks pretty darn awful, but that's par for the course. I want one. I'd settle for two.

Cubase 5

In other news, I've been waiting to see what Steinberg have been cooking. A new Cubase version has been in the pipeline for some time. It's quite suprising what they've done in this release; a lot of unexpected features have been integrated.

They've pulled in Yamaha's pitch correction software and integrated it into the audio editor. That's novel. Sadly, it won't sound as nice as Melodyne, but it's a fairly impressive thing to find in a DAW. They've also added a cut down version of Groove Agent (in the same vein as the cut down Halion in Cubase 4). Novel. Both of those are annoying extras, as I already bought Groove Agent and Melodyne. The external plugins will still, unoubtedly be better than Cubase's native code. But tight integration is cute and handy. And not having to muck around with Melodyne plugin's data files would be pleasant.

They've not particularly spruced up the rather functional-looking project window (which I'd've imagined would have been on their list), although various popup windows have had a chintzy spring clean. So it's even more of the standard Cubase random not-quite matching visual interface, then.

There's loads of other changes in Cubase 5. They've definitely upped the ante for what a DAW provides. It does look interesting, but I'm not sure I actually need to fork out any money to upgrade. But I might; it's annoying to not be on the cutting edge any more.

Melodyne

And of course, there's more tantalising news of the new Melodyne with it's awesome DNA sorcery. They've now called the first release Melodyne Editor. It's still annoyingly "not quite" out.


So what do I spend my bucks on?

Food for the kids would probably be the best option...

2009: The scores so far

2009. A new year. A new world order. The scores so far...

• Working days: 1
• Features implemented: 1
• Nasty, horrible, subtle bugs fixed: 1
• Number of times I've fallen off my bike: 1

So, this year I do intend to finish off the circular buffer series and delight and interest you, dear reader, as much as I humanly can.

Of course, the value of "as much as I humanly can" has changed now that I'm a sole developer. So far, my new work regime (as a sole programmer) has definitely changed that: I'm working harder and later, on the phone to America more, and playing music louder. And there's less time to write stuff.