COSC 414 Audio in Games

1. COSC 414 Audio in Games History of Audio in Games Source: http://www.computernostalgia.net/articles/HistoryofSoundCards.htm

2. History of Sound Cards • A sound card is a computer expansion card that can input and output sound under control of computer programs. • Typical uses of sound cards include providing the audio component for multimedia applications such as music composition, editing video or audio, presentation/education, and entertainment (games). • Many computers have sound capabilities built in, while others require these expansion cards if audio capability is desired.

3. General characteristics • A typical sound card includes • sound chip, usually featuring a digital-to-analog converter, that converts recorded or generated digital waveforms of sound into an analog format. • This signal is led to a (typically 1/8-inch earphone-type) connector where an amplifier, headphones, or similar sound destination can be plugged in. • More advanced designs usually include more than one sound chip to separate duties between digital sound production and synthesized sounds (usually for real-time generation of music and sound effects utilizing little data and CPU time).

4. External Connectors • Most sound cards have a line in connector where the sound signal from a cassette tape recorder or similar sound source can be input. • The sound card can digitize this signal and store it (controlled by the corresponding computer software) on the computer's hard disk for editing or further reproduction. • Another typical external connector is the microphone connector, for connecting to a microphone or other input device that generates a relatively lower voltage than the line in connector. • Input through a microphone jack is typically used by speech recognition software or Voice over IP applications.

5. Digital Sound Reproduction • Digital sound reproduction is usually achieved by multi-channel DACs (digital to analog converters), able to play multiple digital samples at different pitches and volumes, optionally applying real-time effects like filtering or distortion. • Multi-channel digital sound playback can also be used for music synthesis if used with a digitized instrument bank of some sort, typically a small amount of ROM or Flash memory containing samples corresponding to the standard MIDI instruments. • (A contrasting way to synthesize sound on a PC uses "audio codecs", which rely heavily on software for music synthesis, MIDI compliance and even multiple-channel emulation. This approach has become common as manufacturers seek to simplify the design and the cost of the sound card itself).

6. Voices vs channels • Another important characteristic of any sound card is the number of distinct voices (intended as the number of sounds that can be played back simultaneously and independently) and the number of channels (intended as the number of distinct electrical audio outputs). • For example, many older sound chips had three voices, but only one audio channel (mono) where all the voices were mixed into, while the AdLib sound card had 9 voice and 1 mono channel. • Today, a sound card having hardware support for more than the two standard stereo voices, is likely to referred at as "providing hardware audio acceleration".

7. Early Sound Generation • Sound cards for computers based on the IBM PC were uncommon until 1988, leaving the internal PC speaker as the only way early PC software could produce sound and music. • The speaker was limited to square wave production, leading to the common nickname of "beeper" and the resulting sound described as "beeps and boops". • Several companies, most notably Access Software, developed techniques for digital sound reproduction over the PC speaker; the resulting audio, while functional, suffered from distorted output and low volume, and usually required all other processing to halt while sounds were played.

8. Innovation in Hardware • Other home computer models of the 1980s included hardware support for digital sound playback or music synthesis (or both), leaving the IBM PC at a disadvantage when it came to multimedia applications such as music composition or gaming. • It is important to note that the initial design and marketing focuses of sound cards for the IBM PC platform were not based on gaming, but rather on specific audio applications such as • music composition • speech synthesis • It took the involvement of Sierra and other game companies in 1988 to switch the focus toward gaming.

9. Hardware manufacturers - AdLib • One of the first manufacturers of sound cards for the IBM PC was AdLib • The AdLib had two modes: • A 9-voice mode where each voice could be fully programmed, • A lesser-used "percussion" mode that used 3 regular voices to produce 5 independent percussion-only voices for a total of 11. (The percussion mode was considered inflexible by most developers, so it was used mostly by AdLib's own composition software.)

10. Hardware manufacturers – Creative Labs • Creative Labs also marketed a sound card at the same time called the Creative Music System. • twelve voices • stereo card • Sounded like twelve simultaneous PC speakers • never caught on the way the AdLib did, even after Creative marketed it a year later through Radio Shack as the Game Blaster. • The Game Blaster retailed for under $100 and included the hit game title Silpheed.

11. Breakthrough! Sound Blaster Live! • Probably the most significant historical change in the history of sound cards came when Creative Labs produced the Sound Blaster card. • The Sound Blaster • cloned the AdLib, • added a sound coprocessor to record and play back digital audio • a game port for adding a joystick • the ability to interface to MIDI equipment • With more features at nearly the same price point, and compatibility with existing AdLib titles, most first-time buyers chose the Sound Blaster. A Sound Blaster Live! Value card, a typical present-day (2006) PCI sound card.

12. Industry adoption • When game company Sierra On-Line opted to support add-on music the concept of what sound and music could be on the IBM PC changed dramatically. • Two of the companies Sierra partnered with were Roland and Adlib, opting to produce in-game music for King's Quest 4 that supported the Roland MT-32 and Adlib Music Synthesizer. • The MT-32 had superior output quality, due in part to its method of sound synthesis as well as built-in reverb.

13. Game Innovation in sound • Being the most sophisticated synthesizer they supported, Sierra chose to use most of the MT-32's custom features and unconventional instrument patches to produce background sound effects (birds chirping, horses clopping, etc.) before the Sound Blaster brought playing real audio clips to the PC entertainment world. • Many game companies would write for the MT-32, but support the Adlib as an alternative due to the latter's higher market base. • The adoption of the MT-32 led the way for the creation of the MPU-401/Roland Sound Canvas and General MIDI standards as the most common means of playing in-game music until the mid-1990s.

14. Feature evolution • Most ISA (Industry Standard Architecture ) bus soundcards could not record and play digitized sound simultaneously, mostly due to inferior card DSPs (digital signal processing). • Later PCI bus cards fixed these limitations and are mostly full-duplex. • For years, soundcards had only one or two channels of digital sound (most notably the Sound Blaster series and their compatibles) with the notable exception of the Gravis Ultrasound family, which had hardware support for up to 32 independent channels of digital audio. • Early games and MOD-players needing more channels than the card could support had to resort to mixing multiple channels in software.

15. Increased Support • Today, most good quality sound cards have hardware support for at least 16 channels of digital audio, but others, like those that utilize cheap audio codecs, still rely partially or completely on software to mix channels, through either device drivers or the operating system itself to perform a software downmix of multiple audio channels.

16. Sound devices other than expansion cards • Integrated sound on the PC • In 1984, the IBM PCjr debuted with a rudimentary 3-voice sound synthesis chip, the SN76489, capable of generating three square-wave tones with variable amplitude, and a pseudo white noise channel that could generate primitive percussion sounds. The Tandy 1000, initially being a clone of the PCjr, duplicated this functionality, with the Tandy TL/SL/RL line adding digital sound recording/playback capabilities. • In the late 1990s, many computer manufacturers began to replace plug-in soundcards with a "codec" (actually a combined audio AD/DA-converter) integrated into the motherboard. Many of these used Intel's AC97 specification. Others used cheap ACR slots.

17. Needs Drive Innovation • As of 2005, these "codecs" usually lack the hardware for direct music synthesis or even multi-channel sound, with special drivers and software making up for these lacks, at the expense of CPU speed (for example, MIDI reproduction takes away 10-15% CPU time on an Athlon XP 1600+ CPU). • Nevertheless, some manufacturers offered (and offer, as of 2006) motherboards with integrated "real" (non-codec) soundcards usually in the form of a custom chipset providing e.g. full ISA or PCI Soundblaster compatibility, thus saving an expansion slot while providing the user with a (relatively) high quality soundcard.

18. Integrated sound on other platforms • Various computers which do not use the IBM PC architecture, such as Apple's Macintosh, and workstations from manufacturers like Sun have had their own motherboard integrated sound devices. In some cases these provide very advanced capabilities (for the time of manufacture), in most they are minimal systems. Some of these platforms have also had sound cards designed for their bus architectures which of course cannot be used in a standard PC.

19. USB sound cards • While not literally sound cards (since they don't plug into slots inside of a computer, and usually are not card-shaped (rectangular)), there are devices called USB sound cards. • These attach to a computer via USB cables. • The USB specification defines a standard interface, the USB audio device class, allowing a single driver to work with the various USB sound devices on the market.

20. Other outboard sound devices • USB Sound Cards are far from the first external devices allowing a computer to record or synthesize sound. Virtually any method that was once common for getting an electrical signal in or out of a computer has probably been used to attempt to produce sound.

21. Driver architecture • To use a sound card, the operating system typically requires a specific device driver. • Some operating systems include the drivers for some or all cards available, in other cases the drivers are supplied with the card itself, or are available for download. • Microsoft Windows uses proprietary drivers generally written by the sound card manufacturers.