ADVANCED SIGNAL PROCESSING TECHNIQUES FOR WIRELESS COMMUNICATIONS

1. ADVANCED SIGNAL PROCESSING TECHNIQUES FOR WIRELESS COMMUNICATIONS Erdal Panayırcı Electronics Engineering DepartmentIŞIK University

2. OUTLINE • Introduction • Knowledge Gaps in General • The essential of EM algorithm • The Sage algorithm • Some Application Areas • Sequential Monte Carlo Method (SMC) • Knowledge Gaps in SMC

3. INTRODUCTION • Future generation wireless commun. systems are confronted with new challenges mainly due to • Hostile channel characteristics • Limited bandwidth • Very high data rates

4. Advanced Signal Proc. techniques such as The Expectation-Maximization algorithm The SAGE algorithm The Baum-Welch algorithm Sequential Monte Carlo Techniques Kalman filters and their extensions Hidden Markov modeling Stochastic approximation algorithms

5. In collaboration with Inexpensive and Rapid computational power provide powerful tools to overcome the limitations of current technologies.

6. Applications of advanced signal processing algorithms, include, but are not limited to • Joint/Blind/Adaptive • Sequence(data) detection • Frequency, Phase ,timing synchronization • Equalization • Channel Estimation techniques.

7. These techniques are employed in advanced wireless communication systems such as OFDM/OFDMA CDMA MIMO,Space-time-frequency Coding Multi-User detection

8. Especially, development of the suitable algorithms for wireless multiple access systems in Non-stationary Interference-rich environments presents major challenges to us.

9. Optimal solutions to these problems mostly can not be implemented in practice mainly due to high computational complexity

10. Advanced signal processing tools, I mentioned before, have provided a promising route for the design of low complexity algorithms with performances approaching the theoretical optimum for Fast, and Reliable communication in highly severe and dynamic wireless environment

11. Over the past decade, such methods have been successfully applied in several communication problems. But many technical challenges remain in emerging applications whose solutions will provide the bridge between the theoretical potential of such techniques and their practical utility.

12. The Key Knowledge Gaps Theoretical performance and convergence analysis of these Algorithms Some new efficient algorithms need to be worked out and developed for some of the problems mentioned above Computational complexity problems of these algorithms when applied to on-line implementations of some algorithms running in the digital receivers must be handled.

13. Implementation of these algorithms based on batch processing and sequential (adaptive) processing depending on how the data are processed and the inference is made has not been completely solved for some of the techniques mentioned above.

14. Some class of algorithms requires efficient generation of random samples from an arbitrary target probability distribution, known up to a normalizing constant. So far two basic types of algorithms, Metropolis algorithm and Gibbs sampler have been widely used in diverse fields. But it is known that they are substantially complex and difficult to apply for on-line applications like wireless communications. There are gaps for devising new types of more efficient algorithms that can be effectively employed in wireless applications.

15. THE EM ALGORITHM • The EM algorithm was popularized in 1977 • An iterative “algorithm” for obtaining ML parameter estimates • Not really an algorithm, but a procedure • Same problem has different EM formulations • Based on definition of complete and incompletedata

16. Main References • L. E. Baum, T. Petrie, G. Soules and N. Weiss, A Maximization Technique in Statistical Estimation for Probabilistic Functions of Markov Chains, Annals of Mathematical Statistics, pp. 164-171, 970. • A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum-Likelihood from Incomplete Data Via the EM Algorithm, Journal, Royal Statistical Society, Vol. 39, pp. 1-17, 1977. • C. F. Wu, On the Convergence Properties of the EM Algorithm, Annals of Statistics, Vol. 11, pp. 95-103, 1983.

17. The Essential EM Algorithm Consider estimating parameter vector s from data y(“incomplete” data): Parameters to be estimated Random parameters Then, the ML estimate of s is:

18. Thus, obtaining ML estimates may require: Often analytically intractable An Expectation Computationally intensive A Maximization

19. The EM Iteration Define the complete data x Many-to-one mapping having conditional density The EM iteration at the i-th step: E-step: M-step:

20. Convergence Properties • At each iteration the likelihood-function is monotonically non-decreasing • If the likelihood-function is bounded, then the algorithm converges • Under some conditions, the limit point coincides with the ML estimate

21. EM Algorithm Extensions • J. A. Fessler and A. O. Hero, Complete-data spaces and generalized EM algorithms, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-93), Vol. 4, pp. 1-4, 1993. • J. A. Fessler and A. O. Hero, Space-alternating generalized EM algorithm, IEEE Transactions and Signal Processing, October 1994.

22. The SAGE Algorithm • The SAGE algorithm is an extension of EM algorithm • It provides much faster convergence than EM • Algorithm alternates several hidden data spaces rather than just using one complete data space, and • Updates only a subset of elements of the parameters in each itteration

23. Some Application Areas • Positron-Emission-Tomography (PET) • Genetics • Neural Networks • Radar Imaging • Image / Speech processing • Communications • Channel Estimation / Equalization • Multiuser detection • Squence estimation • Interference rejection

24. SEQUENTIAL MONTE CARLO TECHNIQUE (SMC) • Emerged in the field of statistics, • J. S. Liu and R. Chen, “Sequential Monte Carlo Methods for Dynamics Systems”, J. American Stat. Assoc., Vol. 93, pp. 1032-1044, 1998.

25. Recently, SMC has been successfully applied to several problems in wireless communications, such as, Blind equalization Detection/decoding in fading channels It is basically based on approximating theexpectation operation by means of sequentially generated Monte Carlo samples from either unknow state variables or system parameters.

26. Main Advantages • SMC is self adaptive and no training/pilot symbols or decision feedback are needed • Tracking of fading channels and the estimation of the data sequence are naturally in integrated • Channel noise can be either Gaussian on Non-Gaussian • It is suitable for MAP receiver design

27. If the system employs channel coding, the coded signal structure can be easily exploited to improve the accuracy of both channel and data estimation • SMC is suitable for high-speed parallel implementation using VLSI • Does not require iterations like in EM algorithm • Updating with new data can be done more efficiently

28. denotes in the incomplete data or unobservable or missing data. SMC Method • Let denote the parameter vector of interest • Let denote the complete data so that is assumed to be simple • is partially observed • It can be partitioned as where denotes the observed part

29. Example 1. Fading channel Problem: Joinly estimate the data signal and the unknown channel parameters

30. 2. Joint Phase Offset and SNR Estimation is unknown phase offset is unknown noise variance is the data to be transmitted

31. Problem: Estimate based on complete data where observed part incomplete data

32. MAP solution of the unknown parameter vector q is Where p(q |Yt) can be computed by means of incomplete data sequence as MAP SOLUTION USING SMC METHOD

33. Substituting this in the above, we have

34. To implement SMS, we need to draw m independent samples (Monte Carlo samples) from the conditional distribution of • Usually, directly drawing samples from this distribution is difficult. • But, drawing samples from some trial-distribution is easy.

35. is drawn Suppose a set of samples from the trial distribution By associating weight to the sample • In this case, we can use the idea of importance sampling as follows:

36. We can now estimate as follows; where, The pair is called a properlyweighted sample. w. r. t. distribution

37. By properly choosing the trial distribution q(.), the weighted samples can be generated sequentially. That is, suppose a set of properly weighted samples at time t-1 is given. Then SMC algorithm generates from this set, a new one at time t.

38. As a summary SMC algorithm is given as follows for j = 1, 2,..., m • Draw samples from the trial distribution q(.) and let • Compute the important weight from sequentially. • Compute the MAP estimate

39. KNOWLEDGE GAPS IN SMC • Coosing the effective sample size m (empirically usually ,20 < m < 100). • The sampling weights measures the “quality” of the corresponding drawn data sequence • Small weights implies that these samples do not really represent the distribution from which they are drawn and have small contribution in the final estimation • Resampling procedure was developed for it. It needs to be improved for differential applications

40. Delay Estimation Problem: • Since the fading process is highly correlated, the future received signals contain information about current data and channel state. • A delay estimate seems to be more efficient and promising than the present estimate summarized above.

41. In delay estimation: Instead of making inference on (St, q) with posterior density p(q, St|Yt), we delay this inference to a later time (t+D) with the distribution p(q, St|Yt+D)

42. Note: Such a delay estimation method does not increase computational cost but it requires some extra memory. Knowledge Gap: Develop computationally efficient delayed-sample estimation techniques which will find applications in channel with strong memory (ISI channel).

43. Turbo Coding Applications • Because, SMC is soft-input and soft-output in nature, the resulting algorithms is capable of exchanging extrinsic information with the MAP outher channel decoder and sucessively improving the overall receiver performance. Therefore blind MAP decoder in turbo receivers can be worked out.