iPhone Apps | The New "Assistant Engineer"

Feb 1, 2010 12:00 PM, By Sarah Benzuly



Education Guide

Mix is gearing up to present its longstanding annual Audio Education Guide in its November 2014 issue. Want to have your school listed in the directory, or do you need to update your current directory listing? Add an image, program description, or a logo to your listing! Get your school in the Mix Education Guide 2014.


You may have just downloaded your first audio app onto your iPhone or iPod touch and are having a great time playing with the SPL meter, grabbing a recording on-the-go or coming up with some cool effects to slide into your next project. But do you know how these apps were created? Our version of “GeekSquad” gets behind the cool-looking user interface and finds out just how these programs work. To do so, we turned to Stuart Dubey, owner of audio post house DubeyTunes Studios (San Francisco), who is an Emmy- and Clio-winning composer and producer, as well as a sound designer/consultant for many Apple apps.

Tell me about your history in working with the apps
Over the years, I’ve been the lead composer and creating sound design for Apple’s products and applications. We worked on a number of games and apps for iPods, and then the iPhone and iPod touch devices were launched, which quickly became game changers for the mobile industry. It’s been exciting to watch the platform evolve and create a whole new ecosystem of applications and games.

One of the cool things about the iPhone platform is that most people are listening with earphones and earbuds, which from an audio standpoint opens a lot of creative possibilities. You’ve got an audience that can plug into a very immersive sound experience.You can bring them in close to the dimension and sonic magic of a mix.

I recently launched an application myself called BrainBaths, which incorporates music and nature environments recorded with a binaural microphone. It’s essentially a stereo mic imbedded in a replica of the human head so it records the head-related transfer functions, which are the reflections that come off your face and your earflaps, and that information is how your brain computes the spatial position of sound. When listened to through headphones, your brain is sort of tricked into thinking it's real and experiences the sound as holographic and 3-D. It can be really quite striking.

I’m currently working on a series of new audio-centric app that will be coming out later this year.

How would an engineer get an app onto the iPhone platform? 
The iPhone really is a mini touch-screen computer with a little-brother version of the Mac OS so it would be similar to writing a piece of software that runs on a Mac. One of the reasons the iPhone is so successful is that Apple has done a great job of supplying development tools that streamline the process of coding, testing and getting the app into iTunes App Store. For most developers I talk to, it’s one of the easier mobile platforms to develop applications for. 

On a high level, the stages of developing an app include first coming up with the concept, defining the features and user experience and then having a software engineer—code writer—who can translate those features into a software application that runs on the platform. There’s also the graphics and user interface design, which is an art unto itself for making the complex seem easy and intuitive.

Are there advantages or other considerations when developing for this platform?
The iPhone and iPod touch come with a well-designed audio engine that supports some great features and control. You can now simultaneously play and loop multiple files of compressed audio—like AAC files—and this was a big breakthrough. At first, you could only play back one compressed audio file, which put limits on audio-intensive apps. But now you can play multiple streams of compressed audio, which adds a lot of versatility and the ability to keep the file sizes smaller.

The apps are downloaded via the cellular data network or over Wi-Fi, so there are always file size considerations and audio files tend to add up quickly. I work on a lot of iPhone games and long background tracks are often done as seamless loops. So from a music compositional sense, you need to have enough texture and diversity to create a rich listening experience but steer away from certain melodic phrases that can give the user a clue that it’s a loop. Make sure you’ve put that fine detail in on the loop points so you don’t get clicks or jumps in perspective because when listening with headphones, you can really pick up changes in perspective a lot more than with speakers.

When mixing for iPhone or iPod games, you need to keep in mind that the delivery platform is mostly through earbuds or headphones. I’ve got a great mix studio, but then I’ll check the mix through a variety of earbuds because they all sound different. I’ll usually add some extra attention to the low- and high-end detail that gets lost in many earbuds and do a mastering pass so that the overall level is comparable to most music tracks that people may hear when transitioning from their music library to applications. I’ll also make sure the mix sounds fairly balanced through the small iPhone speaker. If layering music and sound effects, I’ll render the individual files at the right volume levels so they mix well. It’s similar concept to mixing stems in post-production.

What is your direction in terms of the sound design?
It really depends on the project and the role the audio plays. When working with iPhones and iPod touches, you’ve got a small screen but a big stereo field to work with. I’ve been incorporating 3-D binaural sounds to create immersive environments. For example, I’m working on a new game called Tilt right now that’s about saving the rain forest, so there’s a music track with primitive organic percussion, fretless bass and breathy ocarinas surrounded in a 3-D jungle environment. I’ll do things like use birds as part of the melody and place them in different parts of the stereo field so there’s a lot of detail and dimension taking place.

What are you sending to the code writers?
When I’m working on a game or a project like that, we’ll come up with an audio asset list that spells out every sound, sound effect and music track for all the different levels of the game. Then I create and mix all those elements so that they can be layered on top of each other and triggered by the user's actions or progress. Each audio element is rendered at the correct volume level rather then using processing power to adjust the volumes in software. I usually deliver each element as an AIFF audio file. which is then converted to a CAF file format using an Apple coding utility called AF Convert.

Are there any limitations our readers should know about?
Probably the biggest limitation is the size of the audio files because they are downloaded inside the application along with all the other code and graphics. Small applications can be download over the AT&T data network. If it’s a larger application, you’re prompted to log your phone into a Wi-Fi network or download to iTunes on your computer. The app then loads to your iPhone when you next plug it in to synchronize. Also, with more complex games and applications, you don’t have as much processing power to work with for audio, so that can limit how you approach designing the sound and features like software-controlled dynamic panning.

Where do you see the future of audio apps going? Will we have a Pro Tools app?
There are some audio recording and editing apps out there, and some that turn the touchscreen into a remote-control surface for Pro Tools and other audio DAWs. Being able to do audio editing on the iPhone could be a good thing if you’re sitting on the train, but I think you’d probably tend to do more of that on your laptop—it’s just easier to get around on a bigger screen. I think the applications geared toward convenience are where the iPhone excels for pro audio—being able to quickly pull out your phone and get a sound pressure level or tune your guitar or record something without having to carry a separate digital recorder. And the cool thing is that it’s all connected, so you can quickly send that new recording to your e-mail or post it to Facebook.

For consumer sound apps, there are some really creative musical- and synthesis-type applications that incorporate the iPhone’s accelerometer, which senses direction and velocity to control different parameters: playing music by tilting, blowing or shaking it like a rhythm instrument. One of the more intriguing features of the platform is that you and your location are interactively connected to other users around the globe. I think we’ll see more interesting musical and audio collaboration type of apps coming out and communities building out of it because you can see in real time where everybody is and who has downloaded and played the instrument.

Click on the next page to see what users have to say about their favorite app. Join the conversation: Use the comments box at the bottom of the page to let us know what your favorite app is!

Acceptable Use Policy
blog comments powered by Disqus

Mix Books

Modern Recording and Mixing

This 2-DVD set will show you how the best in the music industry set up a studio to make world-class records. Regardless of what gear you are using, the information you'll find here will allow you to take advantage of decades of expert knowledge. Order now $39.95

Mastering Cubase 4

Electronic Musician magazine and Thomson Course Technology PTR have joined forces again to create the second volume in their Personal Studio Series, Mastering Steinberg's Cubase(tm). Edited and produced by the staff of Electronic Musician, this special issue is not only a must-read for users of Cubase(tm) software, but it also delivers essential information for anyone recording/producing music in a personal-studio. Order now $12.95



Delivered straight to your inbox every other week, MixLine takes you straight into the studio, with new product announcements, industry news, upcoming events, recent recording/post projects and much more. Click here to read the latest edition; sign up here.

MixLine Live

Delivered straight to your inbox every other week, MixLine Live takes you on the road with today's hottest tours, new sound reinforcement professional products, recent installs, industry news and much more. Click here to read the latest edition; sign up here.

[an error occurred while processing this directive]

The Wire, a virtual press conference offering postings of the latest gear and music news, direct from the source. Visit the The Wire for the latest press postings.