Audio at Electronic Arts

Mar 1, 2004 12:00 PM, By Blair Jackson

Polls


Mix Regional

The Mix Regional section for Mix's September 2014 issue focuses on Miami. Send us your studio news: updates, sessions, new rooms, plus club performances and installations. Let the Mix audience know what is going on! Send photos and descriptions to mixeditorial@nbmedia.com.

AS LITTLE AS 10 YEARS AGO, sound and music for games were practically afterthoughts, usually handled by the same person, generated on MIDI synths and only occasionally using samplers and libraries for effects, and wedged into the game at the end, usually at 8-bit resolution. How things have changed! The video game industry has become a multi-billion-dollar worldwide phenomenon that now rivals (and often exceeds) the film and record industries when it comes to product volume, profitable companies and user loyalty. With millions to be made, the stakes have gotten higher and the pressure more intense to have the most amazing visuals and the best sound possible on every new game. Today, audio designers are working in studios rivaling today's top music facilities in 96 kHz to deliver 44.1k playback. We've gone from those 8-bit mono footsteps feebly rendered at 11 kHz to sound effects and music delivered in 5.1 CD-quality audio. With that in mind, Mix brings you this special section, a close look at the audio side of video games.

If you are a serious gamer or, like me, have children or teens in the house who fit that description, chances are you own a few games made by Electronic Arts (EA), the Redwood City, Calif. — based software company responsible (along with a number of top development partners) for some of the best-selling sports games (Madden Football, NCAA College Football, FIFA Soccer, NBA Live, Tiger Woods PGA Tour, etc.), simulation games (SimCity and the incredibly popular series, The Sims), war games (the Battlefield and Medal of Honor series) and games based on films or film characters, from James Bond to Looney Tunes to Harry Potter to the company's skyrocketing The Lord of the Rings franchise. My household spent a lot more on games this past year than on CDs, and EA took a big chunk of that money. With revenues of $2.5 billion in 2003, a whopping 22 titles that sold more than a million copies each, and offices in Northern California, London, Tokyo, Austin, Orlando, Vancouver and Montreal that employ some 4,400 employees, EA is definitely the industry's top dog. On a rainy afternoon in early December, I visited EA's headquarters to tour its facilities and talk to some members of the audio staff, including the team responsible for the sound on the recent hit game, The Lord of the Rings: The Return of the King.

EA made it clear that it was committed to high-quality audio a decade ago when it hired former recording industry giant Murray Allen to be the company's director of audio production. (He has since moved up the ladder to become VP of post-production.) Allen helped equip the audio rooms in the company's former facility in nearby Foster City, as well as the lovely new multibuilding “campus” (as they call it) just south of San Francisco. Now, another studio veteran, Fred Jones, is EA's audio facilities director. Like his predecessor, Jones has his work cut out for him.

THE QUEST FOR HIGH-QUALITY SOUND

Though set in a somewhat anonymous business park, EA's sprawling headquarters is everything one would hope it might be. Aside from such amenities as a soccer field, full-court indoor basketball gym and a lovely restaurant/food court that serves a wide variety of nicely prepared foods, the campus boasts a number of specially equipped rooms where games can be played and/or tested, and cubicle after cubicle of state-of — the-art computers used for everything from animation to sound design and realization. In The Lord of the Rings wing of one of the nondescript high-rise buildings, artwork, models, fantasy game miniatures and props from the film can be found on the walls and on desks, and the rows of cubicles — quiet the day I visited because the game was finished and work hadn't quite begun on the next EA LOTR game — have “street” names pulled from Middle Earth locales. These people clearly dig their work.

Down in one of the two adjoining ground-floor studios — a THX-certified room equipped with a Pro Tools|HD 3 system, a Pro Control (replacing the studio's former console, an SSL Avant) and two different models of Genelec speakers (1038s in front, 1032s as rear surrounds) — I sit down with Fred Jones, LOTR audio director Don Veca and lead sound designer Charlie Stockley to talk about how video game sound design is both similar and quite different from its feature-film counterpart. (The other lead sound designer, Paul Gorman, was not available that day.) And The Lord of the Rings: The Return of the King game is particularly apt, as it incorporates extensive footage and sound effects from Peter Jackson's films amidst the multiple levels of sophisticated third-person animated game play. In its first month of release alone, the game sold half-a-million copies, and it has been widely hailed as one of the best of 2003. With Ringsmania at an all-time high, the game almost couldn't lose, but its creators — who are also now co-perpetuators of an ongoing LOTR game franchise at EA — embrace the task of creating a mind-blowing game with the same determination and seriousness as Frodo's quest to reach Mount Doom. Most of the sound work was done on the designers' individual Pro Tools MIXPlus systems in small offices that are scattered throughout the complex, with the big high-end studio being used later in the process for intensive mixing and surround work.

IT'S LIKE THE MOVIES, BUT…

Whereas a video game can often have a development time of several years from conception to release, The Return of the King's gestation period was considerably shorter because of the annual release of each LOTR film. For marketing reasons, it was paramount to have the game come out shortly before the film itself, or roughly a year after The Two Towers, which is a completely different game. “A lot of people might think The Return of the King is a sequel to the last game the way the movie is, but to us it wasn't, because everything was in-house for this game, and we built it completely from scratch,” Veca notes.

Veca has been with EA for 12 years. Originally a multi-instrumentalist who went to San Jose State for the university's music program, he found himself spending much of his free time in the college's recording studios programming electronic music, and eventually got his degree in computer science. He got a job at nearby Apple Computer, where he worked on early versions of Sound Manager, MIDI Manager and QuickTime, and then was hired by EA to do audio tools and ad libs programming and authoring.

Everyone agrees that it was the eventual widespread adoption of the CD-ROM format for video games that allowed for the rapid improvement of game audio. “It's still a video game, not an audio game,” Veca says. “But audio has been slowly catching up, and now we're getting to the point where we're expected to sound like a movie. So if we're expected to sound like a movie, then we have to have the resources and the horsepower. Fortunately, we've had lots of support. Neil Young [executive producer of the game; no relation to the rock star] is one of the first executive producers I've encountered who really understands the value of video game audio and its importance in the overall game experience. We were the first THX-approved game. Why did he do that? Because he thought sound was a real priority.”

On the simplest level, creating sound for video games these days is similar to working on a Hollywood film: It usually involves original effects creation, Foley, the integration of score and, in many cases, voice-over. For The Return of the King, the game's sound crew had the added advantage of having actual 5.1 audio from the film and music stems of Howard Shore's score delivered to them. But because the footage they used from the film in different sections of the game had to be completely re-edited to fit the game, so did the audio. And, typical of Hollywood films, the finished audio for the movie was not available at the time EA was working on the game. As a result, the soundtrack is a combination of film elements and effects created specifically for the game.

“We have this cool starting place,” Veca says. “We get to use the music, we get to use some of the sound effects, but we have to chop it up, fill in the holes and create brand-new stuff. But then the hardest part, the most important part, is making it play back correctly, implementing it correctly.

“The analogy one of the guys who works here has used is, you can give me a Stradivarius and it will sound like crap, but you can give Isaac Stern a student model and he's gonna rip it — it's gonna sound great! The point being, you can create the coolest samples and have the most pristine audio, but it's not going to be worth a damn unless it's in the game and implemented correctly.” Veca says that for this game, he had the authority and tools that he needed to place the sounds in the game as intended. “It's a double-edged sword because, sure, we get the power and we get to make it sound way better than normal, but it's that much more work — implementing is a whole other job.”

Games are, by definition, interactive, so just as the player controls the movements of the game's characters, the sound must reflect the action, too. A game like The Return of the King might have a dozen different levels for a player to navigate through, with each level presenting tasks that the player must complete to move on. There are, needless to say, many sound variables that need to be programmed into the game. This requires another level of technical expertise far removed from the traditional sound designer's role.

“Usually, there are game programmers who write the code that lets you see everything and lets everything happen when it happens,” Veca says. “Then there are programmers who write the authoring tools that let an implementer take these assets — our sounds and [film sound designer] Dave Farmer's sounds — put them into this game format, massage them, play them back and integrate them into the game. Those are the tools people. In this case, we [the sound team] were implementers for our own audio. We took the raw assets, edited them and formatted them for the game so they're at the right sample rate or whatever. Then we take those assets and we plug them into the tools. We add triggers that will reference these assets and play them back correctly.

“Charlie [Stockley] was in charge of all RAM-based interactive sounds, including all the characters,” Veca continues. “Every character that you see running around, he put the sound where it's supposed to go at exactly the right volume, using tools that our audio programmer gave us to give us full control. And the cool thing about our audio programmer — Laurent Betbeder — is he's a sound designer and he understands what we need, so we can define how we want things. A lot of it is sequencing: We'll write a specification for the program that says, ‘Okay, they want a footstep here. It's supposed to pick from these three samples, randomize it this much, et cetera.’ So we give it a spec and tell when and where it goes in the game, and then, ultimately, we'll hit the Enter key and it'll grind away and create a big hunk of data that includes the new information in the whole game.” Veca explains that this trial-and-error process is getting faster with new technology. “In the old days, when I was a programmer, you could change some code, hit the Compile button and literally go to lunch and come back before it was finished. If you made one little mistake, you had to go through the whole thing again and wait another couple of hours. But now, like on the effects end, with Pro Tools, you do a crossfade, it isn't quite right, you do it again and it's simple. That improves the process dramatically.”

“The sound design is probably 20 or 30 percent of the process, and the implementation is probably a good 70 percent of what we're doing,” adds Stockley. “It's kind of like a pre-mix in a way. We're adjusting all these minute elements, like how far are we going to hear it if the character walks by that fire, and how does it change as you get closer to it or farther away? Ultimately, the player is the mixer, in a way, because we don't know how close the guy is going to walk to that fire. But we do know that the timbre is going to change along with the volume as the player walks closer or farther away.”

The music, too, is spurred by specific events in the game. “There were six hours of music from the movie, and for this three-minute scene, we might want a high-energy fight thing, or for a sneaky level something completely different. We'd have to scour this score, find this mood, maybe match a key, put it together and make a new piece out of it that you then could say, ‘Now trigger this when you go into this area.’ More often than not, it is not just one piece of Howard Shore's music. It might be a montage that stays within a particular vibe for a certain amount of time. Then you go into this other area and you find this other piece of music that's been edited.

“In the old days,” he continues, “audio couldn't be streamed from disk. Anything you heard was from RAM and it was very limited. So you had these tiny little loops, and we used MIDI because it used even less RAM because you're triggering instruments. We did a lot of cool interactive things with that, but one day, somebody decided to take some of that bandwidth that the rest of the gamers were using for their video streaming. Now, no one uses MIDI; it's almost exclusively streaming. Now, not only do we get a stereo music track at 44.1 — still compressed, but pretty high-quality — we're also streaming ambience, so when you go into a battle, you don't just have the characters making sounds, you have an ambience track that somebody has created in a 5.1 studio. We even have one more stream that doubles as streamed sound effects and for VO.” (For The Return of the King game, Veca went to New Zealand for three weeks to record the principal actors for game voice-overs — a rare privilege.)

RAM BUDGETS

Even with sound gaining importance in video game design, there are still major decisions — and concessions — to be made at every step of the audio process. “There are trade-offs all the time,” Veca says. “There's a limited budget for RAM; Charlie's in charge of that. You've got all these characters to work with — where are we going to use a high-quality sound, where are you going to use the low-quality sound.”

Stockley adds, “I'll look at footsteps or some low thunder rumble and I can either sample rate-convert them lower or higher or massage sounds to fit in my RAM budget. The timbre of the sound and the length are what I focus on most. If I have breaking ice or glass or something, that's going to be a higher sample rate, and, obviously, I want to keep the highs and keep it sounding good. The big huge catapult launching with all the wood creaking and everything — I'll want that to be special. But then a footstep or a clothing Foley can be down-sampled because you're not going to hear it clearly anyway in a massive battle of 10,000 Orcs [the evil manufactured creatures in the LOTR trilogy].

“Then there were the siege towers. These things are huge pieces of wood and metal and stone and everything else, and there's no way I could fit that kind of sound in my RAM budget — if we implemented it as a streamed sound effect, it would step over everything else and streams would be cut off everywhere — so I had to think of a way to make this thing sound huge. So I took little snippets of metal and things like that, lowered the pitch and pitched down some squeaks to become this groany, metally sound, and then I sequenced maybe 20 or 30 of those sounds so it randomly creaked between metals and little pieces of woods. Then underneath all that, I had a one- or two-second loop of a continuous rumble. I mixed all that together inside of a sequencer and came up with a big sound that didn't take up too much [space].”

WHICH BOX?

Another factor is the format of the game: Each has its own demands, peculiarities and limitations. These days, everything is geared toward surround, and because it is the most popular format, “We do our main development on the [Sony] PlayStation 2,” Veca says. “But PlayStation doesn't play 5.1, it plays surround. So does [Nintendo] GameCube. You can get 5.1 from the PlayStation via a DTS software solution, and some of our EA games use a software solution for DTS, which is routed out of the digital/optical output. But it takes too much CPU horsepower for most games — another trade-off. [Microsoft] Xbox does 5.1, so we use 5.1 for our interactive sound effects. But to stream six discrete channels from disk on that box is just not feasible for the LOTR games.

“We have to do a lot of planning and say, ‘If this works on this box, what about that box?’ People are expecting this on this box, and this other box doesn't do it — what do you do? So there's a lot of pre-production work we have to go through to figure out how we're going to deal with this. Actually, we have fairly limited horsepower on the PlayStation. The audio hardware has been frankly lagging behind and, hopefully, the next generation will improve on that. But our biggest job is trying to fit all this cool stuff into this little tiny box and make it all sound good.”

Because of the compressed schedule required to make The Return of the King, the audio department's production timetable was, Veca, admits, “brutal. I think in the summertime, there were quite a few 90-hour weeks.” But no one seems to have any regrets. “We learn something new during every game and we get better at what we do each time out,” he says. Adds Stockley, a veteran of such game titles as Wing Commander IV and Nuclear Strike, “I've been here six years and this game in particular is the best game I've been on. The team is amazing.”

And they'll likely all be working together again this year on the next The Lord of the Rings game. Next up is a game centered on the battle for Middle Earth, and who knows what other adventures await eager game players? Whatever emerges from this EA team — just one of dozens working all over the world — you can count on the audio being more sophisticated and more prominent with each future release.


Blair Jackson is Mix's senior editor.

The Precious Vocals

The Return of the King audio director Don Veca on recording character voices for the game:

“I spent three weeks in New Zealand basically ‘camped out,’ waiting for the actors to be available between their movie work. I actually recorded them myself on a Mac laptop system, which worked out great. For the first Lord of the Rings game I worked on, The Two Towers, I wound up traveling to about five different studios around the world to record the actors, each with different recording studios and gear. Some of the studios and engineers were great, some were not so great. This caused consistency issues, which were very difficult to deal with when the dialog was implemented into the game. I overcame that this time by doing it all with my own portable rig. I used a hand-picked DPA 4035 headset mic for most of the recording, because most of what we recorded was loud yelling in a battle context and the 4035 is great for this [144dB SPL]. Also, many of the actors tend to ‘act’ — move around a lot — during their delivery. In fact, Andy Serkis [Gollum] actually did the entire four-hour recording session on his hands and knees, constantly jumping around all over the booth! The headset mic totally saved us there, but the headphones were soaked after the session!

“The actors themselves were pretty fun to work with, too, especially the Hobbits: Elijah Wood [Frodo], Billy Boyd [Pippin] and Dominic Monaghan [Merry], who are all avid gamers. As it turned out, I was expecting to have to miss my daughter's tenth birthday due to the three-week recording schedule, so the Hobbits and Andy each allowed me to record a special birthday ‘audiograph’ for her. Needless to say, she was thrilled. On top of the three weeks in New Zealand, I also had to fly to the UK for Gandalf; the rest were recorded in the U.S. Overall, it was a great experience.”






Acceptable Use Policy
blog comments powered by Disqus

Mix Books

Modern Recording and Mixing

This 2-DVD set will show you how the best in the music industry set up a studio to make world-class records. Regardless of what gear you are using, the information you'll find here will allow you to take advantage of decades of expert knowledge. Order now $39.95

Mastering Cubase 4

Electronic Musician magazine and Thomson Course Technology PTR have joined forces again to create the second volume in their Personal Studio Series, Mastering Steinberg's Cubase(tm). Edited and produced by the staff of Electronic Musician, this special issue is not only a must-read for users of Cubase(tm) software, but it also delivers essential information for anyone recording/producing music in a personal-studio. Order now $12.95

Newsletters

MixLine

Delivered straight to your inbox every other week, MixLine takes you straight into the studio, with new product announcements, industry news, upcoming events, recent recording/post projects and much more. Click here to read the latest edition; sign up here.

MixLine Live

Delivered straight to your inbox every other week, MixLine Live takes you on the road with today's hottest tours, new sound reinforcement professional products, recent installs, industry news and much more. Click here to read the latest edition; sign up here.

[an error occurred while processing this directive]

The Wire, a virtual press conference offering postings of the latest gear and music news, direct from the source. Visit the The Wire for the latest press postings.