Oceans of Audio, Rivers of Bits
Mar 1, 2004 12:00 PM, By Ron Franklin
It used to be that keeping track of audio or video projects meant pulling the appropriate tape reel out of the vault and cueing up the next session. But let's face it, in a digital age, audio is data. Tape vaults have been replaced by stacks of hard drives. These drives not only hold the raw digital bits that our workstations turn into audio and video, they also hold zillions of MIDI tracks, sampled sound collections, libraries of music or sound effects, and all of the metadata — the data about the data — such as log files, track sheets, databases, client billing records and much more.
When I first started working in Hollywood in the mid-1980s, hard drives came in denominations of 10 and 20 megabytes. If you could string several together (a risky proposition given the delicate nature of the early SCSI interface), then you could even get several whole songs (in stereo) on a single system! Fast forward to today when 24-bit, 48kHz audio is standard, alongside other bit-voracious rates such as 96 kHz, 192 kHz or even DSD. Now add high track counts for most projects and integrated digital video for post. Clearly, storage needs have become a major consideration in planning any kind of creative media environment, whether a home project studio or a multiroom facility.
With hard drives now ranging up to 250 GB in capacity, available storage on individual drives has certainly been keeping up with the demands of contemporary projects. But storage attached directly to a single workstation still belongs in the category of local attached storage. This means that other computers in a network can't really take full advantage of all that storage, even if they can “see” it over a local area network (LAN). To be really useful, each workstation has to be able to access shared media storage drives over a high-speed connection that can reliably deliver multiple tracks of data without choking.
The “gold standard” in speedy drive access for many years was SCSI (Small Computer Systems Interface), an interface that provided enough speed to handle the demands of multiple channels of real-time digital media. Unfortunately, with SCSI there are three or four points of potential failure: SCSI interface cards can fail, cables can fail, termination can be a problem, etc.
The types of data bus by which drives can be attached have now grown to include not only SCSI, but also FireWire (IEEE 1394), Serial ATA (a more efficient replacement for the parallel ATA used for IDE drives on most PCs), Fibre Channel, iSCSI (SCSI over Internet Protocol, a rapidly emerging technology) and even USB. All of them can deliver data, but some such as USB will not have the speed you need to deliver high track counts of digital audio. For most audio applications, the choice comes down to SCSI, FireWire or Fibre Channel. There is also a choice to be made between using single drives or RAIDs. In network parlance, a collection of single drives is called JBOD, or Just a Bunch Of Disks. A RAID (Redundant Array of Independent Disks) can bring an extra amount of data security, as data is written redundantly to more than one drive, and any information lost due to failure of any single disk in the array can usually be recovered from the other disks.
And whichever technology one chooses for the drives and data bus, the two biggest problems plaguing local attached storage solutions are scalability and networkability.
Enter the storage area network (SAN). The primary purpose of a SAN is to provide a dedicated network to transfer data between computer systems and storage elements, and among storage elements. While local attached storage may provide a way for a few servers on a network to share a few hundred gigabytes of data over relatively modest connections (Ethernet) or via “sneaker net,” the solution provided by a SAN can scale to allow for terabytes of information delivered reliably at high speed to large numbers of stations.
The most common implementation of SANs today uses a high-speed serial I/O bus called Fibre Channel, which is capable of transferring data between ports at 100 MB/second or more. To be clear: You can use Fibre Channel drives as local attached storage, but you can also upgrade and integrate the FC drives into a SAN configuration when more workstations need to access that same storage.
There are networked data solutions for every application, whether it's as simple as hooking up two FireWire drives in your project studio or managing terabytes of data in a huge corporation. Here's a look at some of the more popular solutions currently available to digital music and post facilities.DIGITAL AUDIO/VIDEO SPECIALISTS
Lucky for us audio people there are a number of companies that specialize in serving the unique needs of digital audio/video production. In many cases, the software solutions created by these specialty companies are designed to work seamlessly in the context of popular audio/video workstations.
One of the earliest companies in the professional digital media world to offer networking solutions for digital media data was Avid Technology. The company's Avid Unity MediaNetwork is designed specifically for sharing high-bandwidth digital media files between networked workstations. Unity's software takes advantage of Avid's file system architecture, allowing multiple users to access the same media files in a collaborative work group. The system includes a file server, the Avid Unity MEDIA-Switch, Avid Unity File manager software and other software for system administration and performance monitoring. Configurations can be customized with storage and interface cards. According to Avid, the Unity system can handle more than 60 simultaneous real-time clients (200 simultaneous DV25 streams) with 200 48kHz audio tracks, all sharing the same media at the same time. The lower-cost Avid Unity LANShare EX is designed for small to mid-size facilities as a cost-effective way to implement a SAN over Fibre Channel and Ethernet. For more information, see www.avid.com/products/unitymedianetwork.
Glyph Technologies (www.glyphtech.com) is familiar to many workstation users in the pro audio industry. The company had an early storage system called CobaSAN, and although a SAN solution is still available, Glyph has switched its main focus to providing “hot-swap” storage for the small to medium-sized operation. Glyph's line of hot-swap FireWire systems ranges from the Glyph GT 051 single-drive enclosure with an MSRP of $499 to the Glyph GT 308, a three-space eight-bay enclosure that comes with the choice of two to six hot-swappable GT Key drives. Its SCSI/FireWire expansion bay offers the option of AIT backup, SCSI hot-swap receiving bay and DVD-R/RW and/or CD-R/RW. (Price varies depending on configuration.) Addressing concerns about the speed of FireWire compared to Fibre Channel, Glyph director of marketing Peter Glanville says, “The FireWire 400 bus offers 40 megabytes per second. In our experience, this easily handles the track counts most audio workstations use.” Many of Glyph's products include its QuietMetal™ technology, which blocks the vibration of the hard drive and fan from reaching the enclosure by using a trilaminate construction.
Another longtime force in storage for digital media users is Rorke Data (www.rorke.com), a division of Bell Microproducts. Rorke products range from desktop storage and SCSI hot-swap systems to Fibre Channel SANs, incorporating the latest Serial ATA technology in the context of a SAN. Director of product development Pam Moeller says, “While we're selling a lot of Fibre Channel — based systems, we've also recently launched some new Serial ATA — based drives. These are lower-cost RAID systems that use the SATA drive technologies.” Rorke has not pursued FireWire drive technology in any of its products, leaning instead toward the Fibre Channel and SATA technologies. Moeller notes that “we prefer the Fibre Channel connectivity and have tended toward this solution for bigger SAN facilities.”
Rorke's Galaxy-i product line, targeted toward post-production, offers a choice of high-performance 2-gigabits-per-second Fibre Channel or SCSI-160 controller technology with high-capacity, cost-effective Serial ATA drive technology. Products include the Galaxy-16i Series with up to 4.8 terabytes in a single rack with single- or dual-controller and JBOD expansion capability available with either Fibre Channel or SCSI storage. The Galaxy-12LPi has similar capabilities with capacity to 3.6 TB using a single controller with Fibre Channel or SCSI in a 2U rackspace server, while the Galaxy-8LPi provides up to 2.4 TB with the same specifications.
A key component of Rorke's line is ImageSAN file-level SAN-management software, which lets users share files without causing any corruption of the data traffic. “We have volume-level as well as file-level sharing products,” says Moeller. “While our current implementation doesn't support cross-platform file sharing, this is in our development plans for the future of our Image SAN product.”
Studio Network Solutions (www.studionetworksolutions.com) provides Fibre Channel SAN technology for post-production audio and video clients. According to CEO Gary Holladay, the industry is moving toward Fibre Channel and away from IDE, SCSI or FireWire as media drives due to concerns over reliability and capability (track count available from a drive). Holladay says, “People look into our solutions when they can no longer afford downtime or the instability of hot-swap. The new Pro Tools|HD Accel requires eight FireWire drives to operate to full track count right now. We can put together a four-seat system that guarantees full track count on every workstation for less than $20k.”
Fibre Channel SAN improves the number of tracks that can be played simultaneously from a single drive. For example, the A/V SAN can deliver 128 tracks of 24-bit/48k audio from a single drive. Featuring up to 400 MB/s (200 MB/s per port) of throughput and concurrent access for multi-users, the unit also allows simultaneous data backup while using the drives.
Even an individual user with a single workstation can benefit: “Studio Network Solutions Fibredrive II starts at less than $2,000 MSRP and offers everything that you need from a drive system,” says Holladay. “It is the first step to a multi-user SAN because you already have purchased the FC card, cable and drive(s) and, like the A/V SAN, you can use this as part of a larger system as you grow.”
SNS' SANmp cross-platform network software offers features such as auto-sync, database redundancy and hardware RAID support for Pro Tools and other DAWs, plus SANscan, which lets editors find sessions on the drive array without mounting the storage on the desktop.
With all of the recent improvements in the price/performance ratio, a multi-use SAN is now less expensive and adds considerably more value than buying local storage for each machine in a multi-workstation environment. Holladay points out the advantages that the SAN offers in this case: “You get the benefits of sharing the drives in multiple rooms at the same time, eliminating ‘sneaker net’ and eliminating the need to move FireWire/SCSI drives all around the studio and having to spread the session data across multiple drives. You also get the benefit of simplified backup since all data is centralized. Since the cost is more realistic these days, SAN is a pretty easy decision. In the past, when the drives were twice the cost of SCSI, you had to invest quite a bit in the whole system, but like anything else in the computer realm, prices go down and you get more — as it should be.”BIG BUSINESS, BIG NETWORKS
Networked storage is big business, so it's not surprising that some very big companies have extensive solutions going well beyond typical audio/video needs. For users interested in putting together a robust, large-scale SAN, here are a few examples.
ADTX (www.adtx.com) is a Japanese company with a U.S. office in Irvine, Calif. The company's RAID systems include desktop and rackmount units using Fibre Channel or FireWire connectivity. The Array MasStore FC Series is a scalable Fibre Channel RAID solution available in configurations for small and medium installations, while the Array MasStore FCII is designed for larger data centers. The Array MasStor L Series incorporates Serial ATA RAID technology. The company also provides interface cards, including SCSI-to-IDE and FireWire-to-SCSI converters. The DV MasStor system allows direct, computerless recording and playback of high-quality digital video. ADTX is an investor in the U.S. company mSoft Inc., a supplier of sound library servers, and mSoft delivers its systems using ADTX RAIDs.
Advanced Digital Information Corporation (www.adic.com) makes tape library products; disk-based backup systems; software for backup, file sharing and archiving; and SAN appliances. Its SNC 5000 expands storage network functionality by linking devices that use different protocols, letting dissimilar operating systems share the same fabric, dynamically mapping diskless servers to disk volumes and providing access security.
Atto Technology Inc. (www.attotech.com) makes a wide variety of host adapter products for connecting storage and specialized software for controlling RAIDs and SANs. In addition to a variety of SCSI host adapters, notable products include the ATTO FibreBridge
EMC Corporation (www.emc.com/products) offers a full range of networked storage products and services. The company's Automated Network Storage system manages the storage and distribution of programming and advertising for Turner Broadcasting's 19 entertainment cable TV networks, including TNT, TBS, The Cartoon Network and Turner Classic Movies. Products include the Symmetrix DMX series of high-end SAN solutions and the CLARiiON Series of entry-level networked storage systems.
Emulex Corporation (www.emulex.com) is the world's largest supplier and developer of storage networking host bus adapters. The company has a vast range of HBAs for Fibre Channel and also offers an iSCSI interface. The Emulex GN9000/SI host bus adapter is a high-performance iSCSI HBA featuring 64-bit PCI v2.2 and 1Gb/s data rate using industry-standard Gigabit Ethernet networks.
Hewlett Packard offers a wide variety of SAN hardware components, including Fibre Channel switches, host bus adapters, and iSCSI routers and gateways. The HP StorageWorks SAN is its flagship SAN offering. HP also has a wide range of storage-management software and servers. To find out more, go to http://www.hewlett-packard.com.
IBM's (www.storage.ibm.com/ibmsan) SAN Integration Server is designed to help integrate IBM virtualization technology, Fibre Channel switches and RAID technologies into a preconfigured, comprehensive solution. The SAN Integration Server will initially be capable of scaling to more than 100 terabytes of storage capacity and connecting up to 42 hosts.
Network Appliance (www.netapp.com) offers enterprise storage systems and near-line storage solutions for backup and archiving applications. The company's Net-App
Nexsan (www.nexsan.com) is a technology company focused on providing ATA drive — based RAID and iSCSI storage solutions. The InfiniSAN ATAboy2 RAID array combines ATA disk drives with SCSI or FC connections, receiving the highest rating among storage products tested by Infoworld in 2003.
Omneon Media Server Systems (www.omneon.com) focuses on the broadcast and video industry. The company's Omneon Spectrum
RAID Inc. (www.raidinc.com) acquired the assets of Jems Data in 2002, and the combined company now offers both direct attached storage and storage area network products. The Falcon Series uses Serial ATA drives, includes both U160 SCSI and 2GB Fibre Channel interfaces in 8, 12 and 16-bay configurations and can accommodate up to 4.8 TB of storage capacity in a compact 3U form factor.
Seachange International (www.schange.com) focuses on the broadcast market and is a major provider of servers and networks for television. Its products are well known as digital advertising spot-insertion systems.
Silicon Graphics' (www.sgi.com) SGI InfiniteStorage solutions are designed to work flexibly to share data across file systems, including Windows, Unix, Macintosh, Irix, Linux, etc. The company's InfiniteStorage SAN 2000 is the entry-level system using the SGI
Xiotech Corporation (www.xiotech.com) offers the Magnitude and Magnitude 3D line of SAN products. The company's patented technology is used to create fiber-distributed storage clusters that emphasize scalability and resilience. Major media customers include Warner Brothers Entertainment, which estimates that its storage system needs are growing at about 1 terabyte a quarter.FINAL WORD
How do you know when a technology has really arrived? There are no hard and fast rules, but a good indication might be when it reaches such a level of maturity that it actively assists in the creative process, not bogging it down with needless complexity and maintenance problems. By that measure, it seems fair to say that the era of the storage area network has arrived.
The vast storage capabilities and extreme reliability of a Fibre Channel SAN can radically change the way a project can be approached so that it becomes a creative tool in its own right. As an example, a Fibre Channel SAN proved a major factor in the creation of the latest album release of legendary heavy metal band Metallica. The band's digital audio engineer Mike Gillies describes what he believes to be “a first” in the recording process:
“For the St. Anger sessions, not one bandmember showed up to the studio with a single riff or lyric written,” he states. “They would jam out for hours, sometimes for eight- to nine-hour sessions, and anytime anyone had an instrument in their hands, I recorded it. Then we went through and found the little bits that we liked, looped them and that's how the songs were written. This just isn't the way records are made, and there is no way this could be done before the Fibre Channel technology of the [SNS] A/V SAN.”
Another recent example of a Fibre Channel SAN providing solid support in a difficult creative environment was Oscar-winning picture and sound editor Walter Murch's use of a storage area network from Rorke Data during the editing of Anthony Minghella's Civil War film, Cold Mountain. (At press time, Murch was nominated for Best Film Editing for Cold Mountain.) The system included four Apple Final Cut Pro workstations running on a SAN loaded with 1.2 terabytes of storage set up near the film's shooting location in Bucharest, Romania. The workflow and scale for Cold Mountain were enormous, comprising 4,900 takes, 830 daily rolls, 225 video tapes and 190 sound rolls, all used for recording the film's 211 total scenes shot over 113 days. Having this vast amount of media data available to each of the workstations on the SAN contributed to the ease and flexibility with which the editorial team was able to work their creative magic.
The variety of audio and video mixing and editing suites at Angel Mountain Studios in Bethlehem, Pa. [see “On the Cover,” September 2003], makes for a complex network of attached workstations. Simultaneous multiproject demands were taxing the facility's SCSI-based storage infrastructure, so they upgraded to a Studio Network Solutions SAN. Manager Carl James summarizes the studio's systems:
“We have five Pro Tools|HD3 rooms running on Apple G4s. The computers are rackmounted in machine rooms with the monitors connected at the mix location by Geffen extenders. There are also two Apple Final Cut Pro stations running uncompressed on Cinewave hardware installed on G4s and an Apple XServe in our central machine room.”
Bringing these computers into the network requires installation of a PCI Fibre Channel card that connects to the SAN switch in the CMR. The SAN has a terabyte of storage dedicated to audio, divided among 72GB and 144GB Fibre Channel drives, and a 500GB Fibre Channel striped array (RAID) for video.
“The XServe in the central machine room functions as the SAN administrator computer,” James continues. “It also houses an Atto dual SCSI card that ties to a set of Jems Data SCSI hot-swap bays, and LTO, AIT, VXA and DDS tape drives for output and archival purposes. In addition, there is a 3-D animation G4, a compositing and DVD-authoring workstation G4 and a render farm of four Xserves — one of which has the seat on the SAN — that all interconnect with the two video workstations via a Gigabit Ethernet LAN.”
Upgrading to a SAN has streamlined Angel Mountain's workflow. “A project can now migrate from one room to another with little effort and attention to asset management,” says James. “Secondly, the speed of the Fibre Channel storage running over the SAN reduced the drive count per project: We can now run 128 tracks at 48k or 64 tracks at 96k from one drive, a reduction of four to one over our previous setup.”