Sound / Audio / Music

1. Sound / Audio / Music

2. How do we obtain sounds?

3. How do we obtain sounds? • We sample them!

4. Sampling: – analog to digital – continuous to discrete Must be done carefully!

5. stereo

6. Problem: Correct and incorrect sampling rates (sampling frequency; x direction).

7. Problem: Quantization (not enough bits in our ADC; y direction).

8. Audio File Formats • Unity supported audio file formats • MP(EG)1/2/3, OggVorbis, wav, and AIFF • mono or stereo • Torque supported audio file formats • .wav files & .ogg files (OggVorbis) • mono or stereo

9. Audio File Formats • Compression (AKA CODECS) types: • lossy vs. lossless • Ex. • 6.1M mp3 file converted to a 5.1M .ogg file and a 33.9M .wav file • 7.6M mp3 file converted to a 8.6M .ogg file and a 83.2M .wav file • But game engines may pause while decompressing large files.

10. Audio File Formats • Compressed • Compress sounds by choosing the compressed option from inside the Editor. The audio data will be small, but will cost CPU cycles to decode at play time. Depending on the target, Unity will encode the audio to either OggVorbis (Mac/PC/Consoles) or MP3 (Mobile platforms). • For the best sound quality supply the audio in an uncompressed format such as WAV or AIFF (containing PCM data) and let Unity do the encoding. • If only targeting Mac/PC (standalones, webplayers) importing an OggVorbis file will not degrade the quality. But converting to mobile platforms will force a re-encode to MP3 and cause a slight quality degradation. Same goes for importing MP3, when only targeting mobile platforms. • Native • Use Native (WAV, AIFF) audio for short sound effects. The audio data will be larger, but sounds won't need to be decoded at play time.

11. Audio File Editor: Audacity (audacity.sourceforge.net)

12. Audacity: record, play, etc.

13. Audacity: select and cut

14. Audacity: after cut

15. Audacity: Fit-in-Window

16. Audacity: after Fit-in-Window

17. Audacity: add reverb effect to selection

18. Audacity: add reverb effect to selection

19. Add a cheesy wahwah effect to your selection (but preview it first). http://www.youtube.com/watch?v=Ovld0LDOBg0 http://www.youtube.com/watch?v=OEJh2FFUUoU

20. Audio in Unity Audio Components • Audio Clip • Audio Listener • Audio Source • Audio Effects (PRO only) • Audio Low Pass Filter (PRO only) • Audio High Pass Filter (PRO only) • Audio Echo Filter (PRO only) • Audio Distortion Filter (PRO only) • Audio Reverb Filter (PRO only) • Audio Chorus Filter (PRO only) • Reverb Zones

21. Audio in Unity Audio Components The following slides were compiled from information available from the excellent Unity reference manual (http://unity3d.com/support/documentation/Components/index.html).

22. Audio Clip

23. Audio in Unity Audio Components: Audio Clip • Audio Clips are audio data used by Audio Sources. Unity supports mono, stereo and multi (up to 8) channels audio assets. Unity supports importing the following audio file formats: .aif, .wav, .mp3, and .ogg.

24. Audio in Unity Audio Components: Audio Clip

25. Audio in Unity Audio Components: Audio Clip • Audio Format The specific format that will be used for the sound at runtime. • Native Larger file size, higher quality. Best for very short sound effects. • Compressed Smaller file size, lower/variable quality. Best for medium length sound effects and music. • 3D Sound If enabled, the sound will play back in 3D space. Both Mono and Stereo sounds can be played in 3D. • Force to mono If enabled, the audio clip will be down-mixed to a single channel sound.

26. Audio in Unity Audio Components: Audio Clip (cont’d.) • Load Type The way Unity load audio assets runtime. • Decompress on load Decompress sounds upon load. Use this for smaller compressed sounds to avoid the performance overhead of decompressing on the fly. Beware that a compressed sound will use ~10x more memory that when decompressed, so don't use this for large files. • Compressed in memory Keep sounds compressed in memory and decompress while playing. This have a slight performance overhead (esp. for Ogg/Vorbis compressed files) so only use this for bigger files. • Stream from disc Stream audio data directly from disc. This uses a fraction of the original sounds size of memory. Use this for your music or very long tracks. Depending on the hardware, a general advice is to keep this down to 1-2 simultaneously streams. • Compression Amount of Compression to be applied to a Compressed clip. Statistics about file size can be seen beneath the slider. It is suggested to drag the slider to a place that leaves the playback "good enough" but still at a small enough size to meet your file size/distribution needs.

27. Audio Listener

28. Audio in Unity Audio Components: Audio Listener • The Audio Listener acts as a microphone-like device. It receives input from any given Audio Source in the scene and plays sounds through the computer speakers. • For most applications it makes the most sense to attach the listener to the Main Camera. • If an audio listener is within the boundaries of a Reverb Zone reverberation is applied to all audible sounds in the scene. (PRO only) Furthermore, Audio Effects can be applied to the listener and it will be applied to all audible sounds in the scene.

29. Audio Source

30. Audio in Unity Audio Components: Audio Source • The Audio Source plays back an Audio Clip in the scene. If the Audio Clip is a 3D clip, the source is played back at a given position and will attenuate over distance. The audio can be spread out between speakers (stereo to 7.1) (Spread) and morphed between 3D and 2D (PanLevel). This can be controlled over distance with falloff curves. Also, if the listener is within 1 or multiple Reverb Zones, reverberations is applied to the source. (PRO only) Individual filters can be applied to each audio source for an even richer audio experience. See Audio Effects for more details.

31. Audio in Unity Audio Components: Audio Source The Audio Source gizmo in the Scene View and its settings in the inspector.

32. Audio in Unity Audio Components: Audio Source • Rolloff mode: How fast the sound fades. The higher the value, the closer the Listener has to be before hearing the sound.

33. Audio in Unity Audio Components: Audio Source • Rolloff mode types: • Logarithmic rolloff: The sound is loud when you are close to the audio source, but when you get away from the object it decreases significantly fast. • Linear rolloff: The more far away from the audio source you go, the less you can hear it. • Custom rolloff: The sound from the audio source behaves accordingly to how you set the graph of roll offs.

34. Audio in Unity Audio Components: Audio Source • Rolloff mode types: • Logarithmic Rolloff • Linear Rolloff • Custom Rolloff

35. Audio in Unity Audio Components: Audio Source • Distance Functions • There are several properties of the audio that can be modified as a function of the distance between the audio source and the audio listener. • Volume: Amplitude(0.0 - 1.0) over distance. • Pan: Left(-1.0) to Right(1.0) over distance. • Spread: Angle (degrees 0.0 - 360.0) over distance. • Low-Pass (only if LowPassFilter is attached to the AudioSource): Cutoff Frequency (22000.0-10.0) over distance.

36. Audio in Unity Audio Components: Audio Source properties • Audio Clip Reference to the sound clip file that will be played. • Mute If enabled the sound will be playing but muted. • Bypass Effects This Is to quickly "by-pass" filter effects applied to the audio source. An easy way to turn all effects on/off. • Play On Awake If enabled, the sound will start playing the moment the scene launches. If disabled, you need to start it using the Play() command from scripting. • Loop Enable this to make the Audio Clip loop when it reaches the end. • Priority Determines the priority of this audio source among all the ones that coexist in the scene. (Priority: 0 = most important. 256 = least important. Default = 128.). Use 0 for music tracks to avoid it getting occasionally swapped out. • Volume How loud the sound is at a distance of 1 world unit (1 meter) from the Audio Listener. • Pitch Amount of change in pitch due to slowdown/speed up of the Audio Clip. Value 1 is normal playback speed. • 3D Sound Settings Settings that are applied to the audio source if the is a 3D Sound.

37. Audio in Unity Audio Components: Audio Source properties (cont’d.) • Pan Level Sets how much the 3d engine has an effect on the audio source. • Spread Sets the spread angle to 3d stereo or multichannel sound in speaker space. • Doppler Level Determines how much doppler effect will be applied to this audio source (if is set to 0, then no effect is applied). • Min Distance Within the MinDistance, the sound will stay at loudest possible. Outside MinDistance it will begin to attenuate. Increase the MinDistance of a sound to make it 'louder' in a 3d world, and decrease it to make it 'quieter' in a 3d world. • Max Distance The distance where the sound stops attenuating at. Beyond this point it will stay at the volume it would be at MaxDistance units from the listener and will not attenuate any more. • Rolloff Mode How fast the sound fades. The higher the value, the closer the Listener has to be before hearing the sound. • 2D Sound Settings Settings that are applied to the audio source if the is a 3D Sound. • Pan 2D Sets how much the engine has an effect on the audio source.

38. Audio effects

39. Audio in Unity Audio Components • Audio Effects (PRO only) • Audio Low Pass Filter (PRO only) • Audio High Pass Filter (PRO only) • Audio Echo Filter (PRO only) • Audio Distortion Filter (PRO only) • Audio Reverb Filter (PRO only) • Audio Chorus Filter (PRO only)

40. Audio in Unity Audio Components • Audio Effects (PRO only) • Audio Low Pass Filter (PRO only) • Audio High Pass Filter (PRO only) • Audio Echo Filter (PRO only) • Audio Distortion Filter (PRO only) • Audio Reverb Filter (PRO only) • Audio Chorus Filter (PRO only)

41. Reverb zones

42. Audio in Unity Audio Components: Reverb Zones • Reverb Zones take an Audio Clip and distortionates it depending where the audio listener is located inside the reverb zone. • They are used when you want to gradually change from a point where there is no ambient effect to a place where there is one. For example when you are entering a cavern.

43. Audio in Unity Audio Components: Reverb Zones • The Audio Reverb Zone gizmo seen in the inspector:

44. Audio in Unity Audio Components: Reverb Zones Properties • Min Distance - Represents the radius of the inner circle in the gizmo, this determines the zone where there is a gradually reverb effect and a full reverb zone. • Max Distance - Represents the radius of the outer circle in the gizmo, this determines the zone where there is no effect and where the reverb starts to get applied gradually. • Reverb Preset - Determines the reverb effect that will be used by the reverb zone.

45. Examples • Player sounds (footsteps, utterances) • Weapon sounds • Vehicle sounds • Environmental sounds • Interface sounds

46. Converting stereo to mono with Audacity (Many game engines require mono sources for 3D sound, i.e. sound with volume based upon listener distance.) • Split stereo track. • Make both tracks mono. • Use the level control for each track, found on the left of the track, to set to: "Gain: -6 dB". Then when you quick mix, it is guaranteed to not clip. • Select both tracks • Project -> Quick Mix. • Save mono audio as .wav or .ogg.

47. Sounds on the internet • The altavista.com search engine no longer allows you to search for audio files (and specific audio file formats).

48. Sound/audio example Demo Audacity and “Good Times.” • file:///j://pub_html/csc343/slides/gt.aup • (actually an XML file) • Isolate the bass track. • lowpass filter • slow tempo but maintain pitch • Unity Sound exercise.

49. Sound/audio exercise Unity Sound exercise.