In last month’s discussion about game audio, I focused on the challenges of making soundtracks for PC-based games. But PCs, of course, are only part of the story. “The PC used to be capable of so much more than the original Nintendo and the other early consoles, but that gap is smaller now,” says producer and audio supervisor Mike Verrette of Ironlore Entertainment, a mid-sized developer in Massachusetts. Microsoft itself has recognized this reality: “They shifted to the Xbox, supporting the console market at the expense of the PC market,” Verrette says. “For one thing, it’s faster to play: You just buy the disc and pop it in. On a PC, it can take up to 25 minutes to install a game since along with everything else, you have to download new drivers for the sound and graphics cards. So maybe by Tuesday you’re playing the game you bought on Saturday. But now they’re trying to revitalize the PC game market. There’s a new game initiative in Vista for auto-loading and auto-installing game software.”
Whichever way Microsoft swings, however, the company has heavy competition from powerful new formats: Sony’s PlayStation 3 and Nintendo’s Wii. And there’s a new thrust among game developers to make their products multiplatform-capable. Because the cost of developing many games is now well up into six figures, it makes economic sense to have them sellable in as many markets as possible. There are obvious economies in this strategy: Different platforms can use the same art, the same code base and the same music. But there are plenty of headaches, too.
For composers, sound designers and audio producers, each platform offers its own challenges. As for the dedicated consoles, the most important difference between them and a PC is that all of the data for a console game comes off of a CD- or DVD-ROM with its relatively slow access time, whereas on a PC, it gets preloaded onto a much faster hard disk. Therefore, streaming speed becomes a real issue. “In a typical game, the sound designer usually gets five to 10 percent of the system resources,” explains Jason Booth, a designer at Harmonix, the MIT Media Lab spin-off that makes the Guitar Hero Series, and which was recently bought by MTV for $175 million. “On PlayStation 2, you have two megs of RAM for audio. All of the sound for wherever you are in the game has to reside in that memory. Sometimes you have to leave room in there for a streaming buffer, but you also have to worry about the speed of the disc spin. Data on the outside of the disc reads faster than data on the inside, so you have to take that into account — data for loading a new level might be on the inside, while data used during run-time is placed on the outside. A common technique is to load in the sound effects ahead of time and stream the music. But, of course, you have to be okay with the rest of the team with that: They can’t be loading textures at the same time you’re streaming music. So everyone has to work together and play nice.”
“The trend is toward never loading at all,” says Verrette. “You want to never force the player to watch a load screen. We’re not there yet, but some games are doing it better than others.”
On the plus side, however, says Verrette, “At least with the console, you know that everyone’s got the same box. You have the same sound and graphics hardware. And piracy is less of an issue — it’s a closed architecture, which makes it harder to get in and crack the copy protection. It’s also not a standard DVD format, so to pirate a game, someone would have to write the software to create the disk images.”
The tools for developing soundtracks are still very much computer-based. “You build the sounds in Pro Tools,” says Alexander Brandon, the audio director for Midway Home Entertainment, and who writes the “Audio-Next” column for Mix, “and all the tools for integrating the sounds into the game are on a PC. You test what you’re doing by ‘baking’ a build of the game and making a simulation of it. Ideally, you want to hear what it sounds like in the console itself, and with an Xbox, you can stream in the sound over Ethernet.”
Because every game engine handles audio differently, developers have to work with a different set of tools for each platform. PlayStation 2, for example, uses a toolset called SCREAM. “Everyone has something for their hardware,” says Booth, “but if you’re developing games for all three consoles, you can’t use tools from the console companies since they don’t work on the others. You have to build your own.”
“For example, within Microsoft,” adds Brandon, “the Xbox has a different set of ‘filters’ [DSP functions] from Windows, and the data is organized differently for the hardware. Even the audio compression format is different: Xbox uses XMA, which is its own version of Windows Media.”
A number of enterprising companies, many of whom are game developers themselves, offer cross-platform tool sets derived from their own in-house development. Id Software, makers of Quake, sells its engine to developers working in Windows, Mac and Linux, while Criterion Software’s RenderWare suite and Epic Games’ Unreal engine can generate code for both PCs and consoles. An audio-only tool set, now in its third iteration, is available from Canadian company Audiokinetic. Its WaveWorks Interactive Sound Engine (Wwise, see “Technology Spotlight” in October 2006 issue of Mix) started out as a PC/Xbox system, and the company recently added PlayStation 3 compatibility.
Wwise offers a slew of helpful tools for game sound design, including prioritization systems that keep track of how many layers are available at any particular moment; multi-listener support for split-screen multiplayer games; linking reverb settings with particular areas in the game and allowing up to four such environments to exist simultaneously for smooth transitions (I wish some of the software reverbs in my DAW would do that!); allowing the designer to decide whether effects will be added in real time or rendered to help manage CPU power; ducking inaudible voices without forcing them to restart their loops; and a clever Occlusion/Obstruction feature that changes sounds’ apparent relative positions using volume and lowpass filtering.
“About 60 percent of developers use middleware engines,” says Brandon. “The complexities of the systems, in terms of graphics and AI, have increased tenfold over the last generation, and that makes it much more difficult to roll your own engines. And the engines have a long way to go in terms of stabilization, the way that the audio industry is settled on Pro Tools and Nuendo.”
“Usually, the engines get customized for each game,” notes Booth. “The interesting part of the job is designing the systems for what they’re going to do and how you’re going to use them. Features like being able to mix layers of sound in real time are built on top of the API [Application Programming Interface]. You build an abstraction layer that writes to the API and interfaces to whatever sound engine you have. If you do that well, then to the sound designer there are no real differences among the platforms. But in some cases, there are features that one console has — but not the others — so you have to ask, ‘Do we spend time on this since it only works on one platform, and only that part of our user base will see it? Do we go for the lowest common denominator or for the best features?’
“There’s a long learning curve for working with new systems, and it’s rare to find a company that has a really well-developed sound toolkit,” Booth continues. “If a development team is 30 guys, the sound staff is one or two. Sound often gets the shaft: Designers have to work with text files and arcane interfaces. So part of the sound designer’s job is to fight, and say, ‘My work would go a lot faster if I could have better tools.’”
On the other hand, there is some progress in raising awareness of the role of sound, both in the game and in the development process. “The companies are learning the lesson about finalizing the script before they finish the music,” says Brandon. “Since people are now using live orchestras and big Hollywood actors, it’s gotten very expensive to do anything over.”
The best part is that the quality of game sound is better than ever. “We’re in a gratifying time when you can use 16-bit samples and do DSP on the fly,” says Booth. Brandon adds, “It’s much more possible to have the equivalent DSP power of a last-generation Pro Tools rig running on a black box alongside your game.”
Occasionally, though, in an odd way, that high quality might work against a company, especially one that’s been around for a while. “Games are about invoking Pavlovian responses in the players,” says Booth. “You’re creating signals and getting people to react to them. You’re showing people a pattern and asking them to break it to win. If you change a sound from one version to another — even if it’s better — people may not like it since they’ve built a reaction to the original sound.
“In Asheron’s Call 1,” which Booth helped design for his former employer, Turbine Entertainment, “when you pushed a button, it made the character more powerful. That’s the main reward in a role-playing game. But the original sound was 8-bit, and when we did Asheron’s Call 2, we re-created it all in 16-bit and couldn’t find the original source. We spent about a week re-creating that sound for the second version, trying to make it better. But everyone was so keyed in to the old sound that they objected to any changes, and we had to go back and carefully re-create the original sound at a higher quality.”
Game development has become one of the hottest fields to get into for young artists, musicians and sound designers, not just because it can be creative or because of the potential money and fame (at least within the gamer community), but also for the sheer intellectual challenge. “It’s complex stuff, and frankly I think that’s what attracts people to this industry,” says Booth, who studied music production at Berklee College. “People that like working under constraints can engender great creativity. When I was working with multitrack tape, my 8-tracks always sounded better than my 24-tracks. With the smaller format, I had to make decisions earlier — I was forced to make choices and take risks.”
Like any young and exploding industry — think office software in the mid-’80s or Internet portals in the late-’90s — there is a tremendous amount of human energy going into game development these days and that creates problems. Like the “Microserfs” of decades past, engineers and designers are being asked to work long hours on projects over extended timelines, and there is a human cost. “It’s not like any other industry I’ve worked in,” says Verrette. “In TV post, a long project was a week. Now, projects take years. There are some real horror stories. We interviewed someone for a job here who had been working 12-hour days, seven days a week, for 28 days.
“Part of it is the generation: The average age is 25 or 26, and a lot of these people have never worked anywhere else,” Verrette continues. “You have a culture of people who have unstructured days working in a creative environment. We have guys who insist they like to work that way — ‘I work best through the weekend.’ The perception for a lot of younger people is that this is the price you pay for working in the game industry: ‘This is cool and this is what I want to do.’ It doesn’t make it right, but it’s how it is.
“It’s simple to get very behind on a project if you’re not focused. People don’t schedule themselves very well at the beginning and end up working 12 to 16 hours at the end. We’re still trying to put in place better project management to avoid that. There aren’t very many common practices yet. And technology moves so fast that every project is different. You can’t specify the right pipeline to produce widget A because the pipeline changes on each project.”
Established strategies from other industries often don’t apply to deadline issues in the game-development world. “The publisher’s solution to the problem may not be the same as the developer’s solution,” says Verrette. “If you want to make the schedule shorter, throwing more money and more people at it may not solve the problem, because those people may be sitting around not doing anything, while the key people end up working much harder.”
“The model of the game industry is startup-centric,” says Booth. “It’s hard to predict how long things are going to take. You have to build it, see what’s wrong with it, change it and keep iterating on it. It’s very easy for the schedule to get behind and then we have to crunch. Over the years, that’s been baked into the industry. Companies that are not well managed have to constantly crunch to get the game done, because if they get too far behind, they’ll go out of business. Other companies that aren’t under that pressure sometimes adopt this, as well. But if you sustain a crunch for a long time, more than a week or so, you burn out your employees and they become useless employees. It generates lots of bad will, then people stop caring and the games end up being not as good. [To] people who find themselves in situations like that, I say they should get out. They’re hurting their own creativity.”
Companies that don’t carefully manage their human assets can end up paying a price. In late 2004, the wife of an engineer at Electronic Arts posted to a blog a long, detailed complaint about how the long hours her spouse was working, and the stress he was working under, were ruining their family life. The post triggered a huge outpouring of sympathy from the community — as well as a couple of lawsuits, which Electronic Arts settled this past April for $15 million.
“At Harmonix, we’ll crunch around once a year, for a week or two,” says Booth. “Management views this as a mistake in scheduling, that it shouldn’t be happening, and they take great effort to make people feel comfortable. They bring in food, masseuses — they try to make people feel like they’re being taken care of.” At Verrette’s company, Ironlore, “We try to anticipate. We implement crunch periods early on in the project, consistently picking up slack as it goes along, so at the end we don’t have a huge, long crunch.
“It’s a real problem when you’ve got an industry that’s doubling in size every few years,” Booth continues. “How do you keep this growth going and keep yourselves in check so you treat your employees well?”
And there’s another result of this huge growth that may ultimately be even more disturbing. “A friend of mine who’s a game designer used to be the lead singer of a band,” says Brandon, “and he’s always telling me horror stories about the record industry. He likes the game industry much better. He says people tell you the truth and are forthcoming. But the more money that goes [into the] industry, the more commercialism is involved. Something’s beginning to change: There’s more money, and there are also now A&R people.”
Paul Lehrman prefers games where you move around a lot and take a shower afterward. His book, The Insider Audio Bathroom Reader, is published by Thomson Course PTR and is available fromwww.mixbooks.com, insideraudio.comand the usual suspects.
