Make some noise! (Audio on iOS)

There are two main APIs on iOS for playing programatically-generated audio:

• Audio Units [see here]
• Audio Queues [see here]

Audio units are a lower-level, lower-latency technology. And also, a relatively complex API. Audio queues are higher-level and simpler to use, but have far greater latency.

I have produced a sample project showing the simplest creation of an audio unit and audio queue output so you can compare and contrast the two APIs. Get it on Gitorious here.

The following series of blog entries will describe each technology.

About the project

The project builds a sample iOS application that plays a steady sine wave as long as it is running. If you look at main.mm you'll see a single #define for USE_AUDIO_QUEUE_OUTPUT that can be used to switch between the audio unit and audio queue back ends.

There are extra things a well-behaved iOS audio app should do that are not (yet) illustrated by this example project. Notably, your app should use the Audio Sessions API to define what kind of audio session it requires and to handle interruptions to the audio output (e.g. when a phone call it received whilst your app is running).

Make some noise!

Before we work out how to make noise with our iOS devices, we need some audio to start with. For this reason, the AudioProducer protocol defines a simple interface that the audio backends can grab an audio stream from.

It looks like this:

/// Type of audio sample produced by an AudioProducer

typedef SInt16 Sample;

/// Protocol for objects that produce audio

@protocol AudioProducer

@property (nonatomic) float sampleRate;

/// Fills a buffer with "size" samples.

/// The buffer should be filled in with interleaved stereo samples.

- (void) produceSamples:(Sample *)audioBuffer size:(size_t)size;

@end

That is, we'll be peddling signed 16-bit integer samples, and have a single method that can be called to grab the next block of interleaved stereo audio samples.

Given that, I have written a simple SineWave audio generator. It's not the most elegant generator, I'll admit. It uses a resonant filter to approximate a sine wave rather than the maths library trig functions for performance reasons.

The SineWave interface adopts the AudioProducer protocol, and adds two extra properties - the frequency of the sine wave and the peak (amplitude) of the wave. You can see the interface here and the implementation here.

Atomicity

Note that since this is a simplistic example I have made all of these properties nonatomic. However, both audio units and audio queues will pull audio from your application in a background thread (not the main user interface thread).

The audio unit uses a very high priority background thread, as it is a very low latency audio pipeline with little buffering. The audio queue thread is not set as high-priority, as it employs a large amount of buffering.

You must bear these threads in mind when writing an audio generator. Ensure that any parameter that can be changed is thread-safe. Make sure that if the UI is in the process of adjusting values and gets interrupted by the audio thread that no disasters (e.g. nasty audio glitches) can result.

What this looks like in practice is different for each application. But this is an important warning to heed.

Next time

So now we have some audio to play, next time we'll look at how to use the audio queue APIs to play it.

Writing: This Time I've Got It...

The November issue of ACCU's C Vu magazine is out now. It contains the latest installment in my Becoming a Better Programmer column. This one's called This Time I've Got It... and is a software development parable. As the article strap-line puts it: Pete Goodliffe tells us a story of stress, short-sightedness, and solutions.

The magazine should have landed on ACCU members' doormats already. The PDF version is available for download on the website.

Sending MIDI through CoreMIDI

The 4.2 iOS GM seed is out. CoreMIDI is coming to the masses. So I've updated my CoreMIDI example to show how to send MIDI out through the USB port. (I've had a lot of people ask me for this.)

Go to the GitHub project page and check out the update. It's a simple addition, but shows the simplest way of getting MIDI data out of your application and through CoreMIDI.

Interestingly, the later - and the GM - versions of iOS 4.2 are far less forgiving of attached USB control devices. Most of the devices I test with are rejected by the iPad because they draw too much power. These are devices with LEDs, but that are certainly within USB power specifications. That's quite a shame.

As ever, I welcome feedback about this example project. Let me know if you find it useful.

Using CoreMIDI in iOS (an example)

In iOS 4.2 the CoreMIDI framework is dropping into the standard operating system. MIDI connectivity using generic class-compliant USB MIDI hardware will be available to all.

It's not a big framework, with a handful of pure-C APIs. But its affect will be profound. What was previously only possible using Akai's AkaiConnect SDK (which is a very nice Objective-C API) or Line 6's MIDI Mobilizer SDK will be available to all.

From what I can see, the iOS CoreMIDI version is going to be the exact same API as the Mac OS desktop variant. I expect developer take-up of this API to be far faster than for the existing two (non-standard) iOS MIDI interface APIs. Pretty soon every iPhone synth app will support Apple's generic MIDI API.

That is, if they can work out how to.

CoreMIDI isn't the best documented API (this is all you get: a list of typedefs and functions). Fortunately, the headers are well-commented and the API is mercifully simple and sensible. However, with no compelling examples (that I could find) it takes a little digging to work out how to use CoreMIDI.

So here I present a very simple example project, with a reusable Objective-C MIDI interface class. With this code, you can get up to speed with CoreMIDI quickly. Indeed, if you just paste my MidiInput class into your synth app you'd be good to go.

Grab it from the GitHub project here. (Update note: the repo master used to be hosted on Gitorious here but we've moved to GitHub in these enlightened times)

Please let me know if you find it useful, or if you use it in your own projects.

Integrating CoreMIDI in your application

First, you need to decide if you want your app to support devices running iOS versions before 4.2. Since CoreMIDI is only introduced in 4.2, you have to jump through a few hoops to keep your app running on earlier versions:

• Weakly link to the CoreMIDI framework, so on OS versions without it your application will still launch. (You do this by going to your application's target in the Xcode tree view, selecting "Get info", and in the "General" tab's "Linked libraries" section ensuring that CoreMIDI is set to "Weak" not "Required".)
• Including CoreMIDI functionality conditionally. The best way to do this is inspect the kCFCoreFoundationVersionNumber variable and only initialise your MIDI handling if the value represents iOS 4.2 or later. (See the iOS version detection header file in my example project for an elegant way to do this).

A demonstration of all of this is available in my example project.

Using CoreMIDI

CoreMIDI itself isn't too complex if you're happy to read the headers and work out what's going on. You need a basic grasp of how Mac OS's Core Foundation works, to understand lifetime management issues and to access string properties, etc.

CoreMIDI has a few basic concepts: most importantly clients, devices (with endpoints) and ports. The C APIs let you enumerate these, and register a notification to keep abreast of changes in MIDI connection state.

MIDI inputs are "sources" in CoreMIDI. MIDI outputs are "destinations".

Given those basic facts, you should be able to read through my MidiInput class and figure out what's going on.

Parsing MIDI

My example program just spits out a stream of binary MIDI data. I'm not showing any parsing of the MIDI data stream here. This parsing isn't rocket science, but is another step you have to perform.

Contact me if you want to know more about parsing MIDI.

Get the code here.

Learning: Some Practical Ideas To Help You Learn

A I wind down my blog series on software developers learning, I present some practical ideas to help you improve your learning:

• Cultivate a healthy set of attitudes: take responsibility for your learning.

• Learn one programming language per year. (This is excellent advice in The Pragmatic Programmer that is still very valid today.)

• Scratch an itch! (What are you curious about? Consider something “leftfield”, not programming-related. Ensure that at all times you have something you're learning about that isn't directly related to your day job).

• Read at least one book every two months (Read more if you want, but set some kind of benchmark to aim for).

• Look after your learning machine – get good nutrition, and plenty of rest. Avoid stress. Have fun!

• Build mental maps as you learn.

• Try to use both sides of your brain.

• Perform deliberate practice & exercise as you learn.

• Network: actively learn from others, and seek to teach/mentor others.

• Enjoy learning. This stuff is fun.

• Apply any new knowledge cautiously.

  
  

Conclusion

You have to take responsibility for your own learning. It's not up to your employer, your state education system, an assigned mentor, or any other person.

You are in charge of your own learning. It's important to continually improve your skills to improve as a developer. And to do that you have to learn to learn. To make it rewarding you have to learn to love doing it.

Learn to live to love to learn.

Questions to ponder

• When were you last in a situation that required learning?

• How did you approach it?

• How successful were you?

• How quickly did you learn?

• How could you have performed better?

• Did you learn, then work, or learn as you worked?

• Which do you think is most effective?

Write Less Code!

It's a sad fact that in our modern world that there's simply too much code. I can cope with the fact that my car engine is controlled by a computer, there's obviously software cooking the food in my microwave, and it wouldn't surprise me if my genetically modified cucumbers had an embedded micro controller in them. That's all fine; its not what I'm obsessing about. I'm worried about all the unnecessary code out there.

There's simply too much unnecessary code kicking around. Like weeds, these evil lines of code clog up our precious byes of storage, obfuscate our revision control histories, stubbornly get in the way of our development, and use up precious code space, choking the good code around them.

Why is there so much unnecessary code? Perhaps it's due to genetic flaws. Some people like the sound of their own voice. You've met them; you just can't shut them up. They're the kind of people you don't want to get stuck with at parties. Yada yada yada. Other people like their own code too much. They like it so much they write reams of it. { yada->yada.yada(); } Or perhaps they're the programmers with misguided managers who judge progress by how many thousands of lines of code have been written a day.

Writing lots of code does not mean that you've written lots of software. Indeed, some code can actually negatively affect the amount of software you have – it gets in the way, causes faults, and reduces the quality of the user experience. The programming equivalent of anti-matter.

Some of my best software improvement work has been by removing code. I fondly remember one time when I lopped literally thousands of lines of code out of a sprawling system, and replaced it with a mere ten lines of code. What a wonderfully smug feeling of satisfaction. I suggest you try it some time.

Why should we care?

So why is this phenomenon bad, rather than merely annoying? There are many reasons why unnecessary code is the root of all evil. Here are a few headlines:

• Writing a fresh line of code is the birth of a little lifeform. It will need to be lovingly nurtured into a useful and profitable member of software society. Then you release the product.

Over the life of the software system, that line of code needs maintenance. Each line of code costs a little. The more you write, the higher the cost. The longer they live, the higher the cost. Clearly, unnecessary code needs to meet a timely demise before it bankrupts us.

• More code means there is more to read – it makes our programs harder to comprehend. Unnecessary code can mask the purpose of a function, or hide small but important differences in otherwise similar code.

• The more code there is, the more work required to make modifications – the program is harder to modify.

• Code harbours bugs. There more code you have, the more places there are for bugs to hide.

• Duplicated code is particularly pernicious; you can fix a bug in one copy of the code and, unbeknown to you, still have another thirty two identical little bugs kicking around elsewhere.

Unnecessary code comes in many guises: unused components, dead code, pointless comments, unnecessary verbosity, and so on. In a future post, I'll look at some of these pernicious beasties in a little more detail.

Learning: Mapping your learning journey

In the last few posts on learning I've been reviewing a number of tools that help us, as software developers, learn.The final tool we'll look at in our arsenal is a powerful old favourite: the mind map. Mind maps are a powerful outlining technique that appeases the left brain's linear organisational fetish whilst satisfying the right brain's spatial, relational desires.

Mind maps are structured outlines in which you start with a key concept in the middle and add information around it forming a web. You can add extra relations between items, and freely use colour, illustration, size and position to highlight extra information.

I often produce mind maps for talks and articles I'm writing. Being an unashamed sad geek, I tend to produce these electronically (I used FreeMind which works on Windows, Mac, and Linux). Then I can keep them under source control and version them alongside my articles.

However, many people claim that you will miss the full benefit of the mind mapping technique by doing this. The more visceral, physical act of creating a hand-written version helps you explore and retain information. By laying out the relations yourself rather than relying on a machine to typeset the “document” for you, you consider inter-relations and build a tangible picture in your mind that will aid later recall.

Next time you're learning, why not try to record knowledge in a mind maps you go? When investigating a learning route and working out what you need to know, record your findings in a mind map.

Use mind maps to catalogue information. They are a proven, powerful tool.