DSP Everywhere… Applications of DSP in Audio and Digital Communications

DSP Everywhere…Applications of DSP in Audio and Digital Communications Simon Doclo Dept. Elec. Engineering, K.U.Leuven simon.doclo@esat.kuleuven.ac.be www.esat.kuleuven.ac.be/~doclo/

Wireless systems: GSM, WLAN • 3G-systems: UMTS, CDMA • Modems: Cable, ADSL, VDSL • Line echo cancellation • Satellite communications • Optical communication Introduction • DSP in Digital Communications Simon Doclo DSP Everywhere…

Audio and speech coding • Audio effects • Tele-conferencing • Voice-controlled systems Introduction • DSP in Audio Applications • Hands-free telephony • Hearing aids / cochlear implants Simon Doclo DSP Everywhere…

Image and video processing • Medical applications • Cryptography • Process control in chemical, pharmaceutical, energy plants • … Introduction • Other applications Anywhere (digital) signals are present, DSP-techniques are required! Simon Doclo DSP Everywhere…

Overview • Introduction • DSP in digital communications systems: • xDSL-modems: modulation, equalisation • DSP in audio applications: • Hands-free communication: echo, noise and reverberation • Basic techniques: • Acoustic echo cancellation (AEC) • Multi-microphone beamforming • Application: hearing aids • Conclusion Simon Doclo DSP Everywhere…

x 1000 Telephone Line modems • High-speeddata communication: • optical, cable, wireless, telephone line • Telephone Line Modems • voice-band modems : up to 56kbits/sec in 0…4kHz band • ADSL modems : up to 6Mbits/sec in 30kHz…1MHz band • VDSL modems : up to 52Mbits/sec in …10MHz band • Time to download 10 Mbyte-file: Simon Doclo DSP Everywhere…

xDSL Modems • ADSL : ‘Asymmetric Digital Subscriber Line’ • HDSL : ‘High Speed Digital Subscriber Line’ • VDSL : ‘Very High Speed Digital Subscriber Line’ …-1993: ADSL spurred by interest in video-on-demand (VOD) 1995 : ADSL/VOD interest decline 1996 : ADSL technology trials prove viability. 1997-... : ADSL deployment, reoriented to data applications, as telco’s reaction to cable operators offering high- speed internet access with cable modems 2000-… : VDSL Simon Doclo DSP Everywhere…

Downstream Bandwidth Copper wire 1.1 MHz Central office Subscriber Upstream 12 MHz Down Up Length ADSL 6 Mbps 640 Kbps 3 km VDSL 52 Mbps 6.4 Mbps 300 m xDSL Modems • Analog/digital telephone network: BW  3 kHz, SNR  35 dB Shannon capacity • ADSL/VDSL: higher bandwidth, lower SNR + impairments • Bitrate depends on length of copper line vb. Simon Doclo DSP Everywhere…

SNR bits/toon frequentie frequentie DMT Principles: IFFT/FFT-based modulation • Modulation-technique : DMT (Discrete Multitone) • Basic idea: • Decompose frequency into ‘tones’ (FFT/IFFT) • Assign bits according to SNR per tone • ADSL spec (=ANSI standard): • 256 tones, 512-point (I)FFTs • carrier spacing fo= 4.3215 kHz, basic sampling rate 2.21 MHz (=512*4.3215 kHz) • VDSL (=proposal): up to 4096 tones, same carrier spacing Simon Doclo DSP Everywhere…

ADSL Spectrum • ADSL spectrum : Simon Doclo DSP Everywhere…

useful signal NEXT FEXT Communication impairments (1) • Frequency-dependent channel attenuation introduces inter-symbol interference (ISI)  equalization • Coupling between wires in same or adjacent binders introduces crosstalk (XT) • Near-end Xtalk (NEXT) • Far-end Xtalk (FEXT) • Other systems (e.g. HPNA) • Radio Frequency Interference (RFI): e.g. AM broadcast, amateur radio • Noise: e.g.impulsive noise (=high bursts of short duration) • Echo: due to hybrid impedance mismatch echo cancellation Conclusion: Need advanced modulation, DSP,etc. ! Simon Doclo DSP Everywhere…

Communication impairments (2) • ADSL channel attenuation, crosstalk, noise Simon Doclo DSP Everywhere…

4-QAM IFFT P/S S/P Discrete equivalent channel FFT 8-QAM FEQ Modulation (IFFT) Time domain signal Demodulation (FFT) Equalisation Bitstream (freq domain) Modulation - Demodulation (1) • DMT-transmission block scheme: Simon Doclo DSP Everywhere…

real real Modulation - Demodulation (2) • Transmission: modulation is realized by means of 2N-point Inverse Discrete Fourier Transform (IFFT)(example N=4 ) • Receiver: demodulation with inverse operation, i.e. FFT Simon Doclo DSP Everywhere…

The Magic Prefix Trick (1) Additional feature : before transmission, a ‘prefix’ is added to each time-domain symbol, i.e. the last samples are copied and put up front : Simon Doclo DSP Everywhere…

The Magic Prefix Trick (2) Prefix insertion : • in the receiver, the samples corresponding to the prefix are removed (=unused) : IFFT P/S S/P Discrete equivalent channel FFT FEQ Simon Doclo DSP Everywhere…

Tone 3 Tone 2 Tone 1 Tone 0 Tone 3 Tone 2 Tone 1 Tone 0 The Magic Prefix Trick (3) • if channel impulse response has length L (= L non-zero taps) and ( is prefix length), then all ‘transient effects’ between symbols are confined to the prefix period : Tx-side Rx-side Channel ch(t) * s(t) r(t) Prefix From IFFT Guardband To FFT Simon Doclo DSP Everywhere…

The Magic Prefix Trick (4) • Magic trick fails if , resulting in • inter-symbol-interference (ISI) = interference from previous symbol(s) (same carrier) • inter-carrier interference (ICI) = interference from other carriers • In the receiver, after removing the samples corresponding to the prefix, the i-th tone is observed, multiplied by a factor H(i.fo), i.e. the channel response for frequency f=i.fo • ‘Prefix trick’ is based on a linear convolution (filtering by channel impulse response) being turned into a circular convolution, which corresponds to component-wise multiplication in frequency domain  easy equalization ! Simon Doclo DSP Everywhere…

Hands-free communication • Recorded microphone signals are corrupted by: • Far-end echoes  acoustic echo cancellation • Acoustic background noise  noise suppression • Room reverberation  dereverberation • Application: hands-free telephony, hearing aids, voice control Simon Doclo DSP Everywhere…

unknown known (=loud-speaker signal) Signal model: some maths… • Multi-microphone signal enhancement algorithms: • Extract clean speech/audio signal from microphone recordings • Exploit spatial and frequency diversity between speech and noise • Microphone signals (m=1…M): • Output signal: compute filters g[k] : • echo cancellation: • noise reduction/dereverberation: • gm[k] cancels noise components • gm[k] focuses on speech s[k] Simon Doclo DSP Everywhere…

Acoustic echo cancellation (AEC) Suppress acoustic and line echo: • to guarantee normal conversation conditions : users do not like to hear a delayed and filtered version of their own voice • to prevent the closed-loop system from becoming unstable if amplification is too high Simon Doclo DSP Everywhere…

Room Acoustics • Propagation of sound waves in an acoustic environment results in • signal attenuation • spectral distortion • The attenuation and distortion can be modeled quite well as a linear filtering operation • Non-linear distortion mainly stems from the loudspeakers. Its effect is typically of second order, therefore (often) not taken into account • The linear filter h[k] modeling the acoustic path between loudspeaker and microphone is represented by the acoustic impulse response Simon Doclo DSP Everywhere…

For typical applications the impulse reponse is between 100 and 400 ms long  several 100 to 1000 taps @ 8-16 kHzmemory requirement for circular buffers in DSP Acoustic Impulse Response (1) • dead time Different parts: • direct path impulse and early reflections, which depend on the geometry of the room • an exponentially decaying tail called reverberation, coming from multiple reflections • Because people move around in the recording room, the acoustic impulse response is highly time-varying Simon Doclo DSP Everywhere…

Acoustic Impulse Response (2) ESAT speech laboratory : T60 120 ms Paleis voor Schone Kunsten : T60 1500 ms Original speech signal : Simon Doclo DSP Everywhere…

Acoustic Impulse Response : FIR or IIR ? • If the acoustic impulse response is modeled as • an FIR filter  many hundreds to several thousands of filter taps are required • an IIR filter  filter order can be reduced, but still several hundreds of filter coefficients are required (=‘bad’ model for acoustic impulse response) • Remark:IIR-filters good model for classical filters (LP,HP,BP,BS) • hence FIR models are typically used in practice • as they are guaranteed to be stable • as adaptive filtering techniques are called for: • FIR adaptive filters are easier than IIR adaptive filters Simon Doclo DSP Everywhere…

AEC based on Adaptive Filtering • Goal: Identify acoustic impulse response h[k] and subtract filtered loudspeaker signal from microphone signal • Thanks to the adaptivity • time-varying acoustics can be tracked • AEC is ‘self-learning’ • performance superior to performance of conventional techniques Simon Doclo DSP Everywhere…

filtered signal desired signal error signal Adaptive Filtering Algorithms • Algorithm: 2 steps • Filter loudspeaker signal  error signal indicates how close this signal is to recorded microphone signal • Update filter: update depends on error signal Simon Doclo DSP Everywhere…

Data filtering Filter update Normalized Least Mean Square (NLMS) with L is the adaptive filter length,  is the adaptation stepsize,  is a regularization parameter and k is the discrete-time index Circular data buffer Filter coefficients Simon Doclo DSP Everywhere…

Control Algorithm • ‘AEC is more than just an adaptive filter’ : • adaptive filter is supplemented with control software, which mainly controls the adaptation speed (e.g. no adaptation during double-talk) • In practice echo suppression is limited to 30 dB due to time-variance, non-linearities, finite filterlength  postprocessing (e.g. center-clipping) Simon Doclo DSP Everywhere…

Real-time DSP Implementation (1) • AEC-implementation on DSP (lab equipment): • TMS320C44 @ 50 MHz : data acquisition (ADC/DAC) • TMS320C40 @ 50 MHz : acoustic echo cancellation (AEC) AEC ADC/DAC Simon Doclo DSP Everywhere…

Real-time DSP Implementation (2) • Adaptive filtering part : several algorithms can be selected • NLMS : time-domain algorithm • PB-FDAF : frequency-domain algorithm (better performance) • Control software • double-talk detection • non-linear postprocessing algorithm • Variable sampling rate • Common sampling rates for speech applications: 8 kHz, 16 kHz • for audio applications: 22.05 kHz, 44.1 kHz, 48 kHz • Echo paths up to 325 ms can be modeled and tracked with the FDAF based on LMS at 8 kHz sampling frequency and 16 ms delay Simon Doclo DSP Everywhere…

Real-time DSP Implementation (3) Execution times for the most important blocks of the DSP code were measured : N=768 FFT-size=128 fs=8000 Hz block 64 samples = 8 ms Simon Doclo DSP Everywhere…

Output AEC without Double-talk Detection Demo Local speaker Output AEC Far-end signal Near-end signal Simon Doclo DSP Everywhere…

Beamforming basics • Background/history: antenna array design for RADAR • Array elements are combined electronically such that: • array can be steered towards specific direction  higher directivity • beam shaping is possible • Beamforming for hands-free communication : • focus beam on speech source(s)  speech enhancement and dereverberation • put spatial nulls in direction of noise sources noise reduction • Classification: • fixed beamforming: data-independent  fixed filters gm[k]e.g. delay-and-sum, weighted-sum, filter-and-sum • adaptive beamforming: data-dependent  adaptive filters gm[k]e.g. LCMV-beamformer, Generalized Sidelobe Canceller Simon Doclo DSP Everywhere…

Delay-and-sum beamforming (1) • Microphone signals are delayed and summed together array can be virtually steered to angle  • Angular selectivity is obtained, based on constructive ( =) and destructive () interference • Uniform delay-and-sum beamforming implies • Uniform array  equal inter-microphone distance • Uniformly distributed delays Simon Doclo DSP Everywhere…

Delay-and-sum beamforming (2) • Spatial directivity pattern H(,) for uniform DS-beamformer • H(,) has sinc-like shape and is frequency-dependent M=5 microphones d=3 cm inter-microphone distance =60 steering angle fs=16 kHz sampling frequency Simon Doclo DSP Everywhere…

Spatial aliasing Delay-and-sum beamforming (3) • For an ambiguity, called spatial aliasing, occurs.This is analogous to time-domain aliasing where now the spatial sampling (=d) is too large. M=5, =60, fs=16 kHz, d=8 cm Simon Doclo DSP Everywhere…

M=8 Logarithmic array L=50 =90 fs=8 kHz Filter-and-sum beamformer • Better directivity patterns than DS-beamformer are obtained with weighted-sum and filter-and-sum beamformers • e.g. Frequency-independent directivity pattern Simon Doclo DSP Everywhere…

Adaptive beamforming • Adaptive filter-and-sum structure: • Minimize noise output power, while maintaining a chosen frequency response in look direction (and/or other linear constraints) • LCMV = Linearly Constrained Minimum Variance • minimize variance of output z[k] • in order to avoid desired signal to be distorted or cancelled out,J linear constraints are added Simon Doclo DSP Everywhere…

Postproc Generalized Sidelobe Canceller (1) • GSC consists of three parts: • Fixed (delay-and-sum) beamformer, in order to achieve spatial alignment of speech source speech reference • Blocking matrix, placing spatial nulls in the direction of the speech source noise references • Multi-channel adaptive filter with delay Simon Doclo DSP Everywhere…

Problems of GSC: • impossible to reduce noise from look-direction • reverberation effects cause signal leakage in noise reference • adaptive filter is only updated when no speech is present ! Generalized Sidelobe Canceller (2) • Blocking matrix Ca : • creating maximum M-1 independent noise references by placing spatial nulls in look-direction • different possibilities: e.g. Griffiths-Jim, Walsh broadside Simon Doclo DSP Everywhere…

Application: Hearing Aids (1) • Hearing problems are very common nowadays • Most of the users are dissatisfied with the performance of their hearing aid in noisy environments (cocktail party effect) increase speech intelligibility by reducing background noise • Traditional hearing aids: • one microphone, analog, limited signal processing • amplification of all incoming sound without distinction between different sound sources • Enabling technologies: • microphone miniaturisation  integrate multiple microphones into one hearing aid • micro-electronics: size ASIC < 10 mm2,low power consumption • advanced DSP techniques (noise reduction, feedback suppression) Simon Doclo DSP Everywhere…

Application: Hearing Aids (2) • Improvement of speech intelligibility by reduction of background noise • BTE hearing aid with 2 (or more) closely-spaced microphones • GSC in switched mode: • Beamfomer : weights can be adapted during speech • Noise suppression (ANC) : only adaptation during noise • Speech detection : determine when speech is present Simon Doclo DSP Everywhere…

Conclusion • DSP-techniques can be found in many every-day products: • audio applications: CD, MiniDisc, hands-free telephony • communications: GSM, modems, WLAN • medical applications: hearing aids, cochlear implants • Implementation differences: • sampling rate, memory requirements, complexity • Basic techniques: • filters, filterbanks, FFT/IFFT  frequency filtering • adaptive filters  track changing systems • multi-sensor systems  spatial filtering Simon Doclo DSP Everywhere…

DSP Everywhere… Applications of DSP in Audio and Digital Communications