John J. Shynk

Probability, Random Variables, and Random Processes

Theory and Signal Processing Applications

• Gebundenes Buch

Produktbeschreibung

Probability is ubiquitous in every branch of science and engineering. This text on probability and random processes assumes basic prior knowledge of the subject at the undergraduate level. Targeted for first- and second-year graduate students in engineering, the book provides a more rigorous understanding of probability via measure theory and fields and random processes, with extensive coverage of correlation and its usefulness. The book also provides the background necessary for the study of such topics as digital communications, information theory, adaptive filtering, linear and nonlinear estimation and detection, and more.

---

Probability, Random Variables, and Random Processes is a comprehensive textbook on probability theory for engineers that provides a more rigorous mathematical framework than is usually encountered in undergraduate courses. It is intended for first-year graduate students who have some familiarity with probability and random variables, though not necessarily of random processes and systems that operate on random signals. It is also appropriate for advanced undergraduate students who have a strong mathematical background.

The book has the following features:
Several appendices include related material on integration, important inequalities and identities, frequency-domain transforms, and linear algebra. These topics have been included so that the book is relatively self-contained. One appendix contains an extensive summary of 33 random variables and their properties such as moments, characteristic functions, and entropy.
Unlike most books on probability, numerous figures have been included to clarify and expand upon important points. Over 600 illustrations and MATLAB plots have been designed to reinforce the material and illustrate the various characterizations and properties of random quantities.
Sufficient statistics are covered in detail, as is their connection to parameter estimation techniques. These include classical Bayesian estimation and several optimality criteria: mean-square error, mean-absolute error, maximum likelihood, method of moments, and least squares.
The last four chapters provide an introduction to several topics usually studied in subsequent engineering courses: communication systems and information theory; optimal filtering (Wiener and Kalman); adaptive filtering (FIR and IIR); and antenna beamforming, channel equalization, and direction finding. This material is available electronically at the companion website.

Probability, Random Variables, and Random Processes is the only textbook on probability for engineers that includes relevant background material, provides extensive summaries of key results, and extends various statistical techniques to a range of applications in signal processing.

Produktdetails

• Verlag: Wiley & Sons
• 1. Auflage
• Seitenzahl: 794
• Erscheinungstermin: 6. November 2012
• Englisch
• Abmessung: 249mm x 196mm x 43mm
• Gewicht: 1556g
• ISBN-13: 9780470242094
• ISBN-10: 0470242094
• Artikelnr.: 35259558

Autorenporträt

JOHN J. SHYNK, PhD, is Professor of Electrical and Computer Engineering at the University of California, Santa Barbara. He was a Member of Technical Staff at Bell Laboratories, and received degrees in systems engineering, electrical engineering, and statistics from Boston University and Stanford University.

Inhaltsangabe

PREFACE xxi NOTATION xxv 1 Overview and Background 1 1.1 Introduction 1 1.2 Deterministic Signals and Systems 19 1.3 Statistical Signal Processing with MATLAB(r) 35 Problems 39 Further Reading 45 PART I Probability, Random Variables, and Expectation 2 Probability Theory 49 2.1 Introduction 49 2.2 Sets and Sample Spaces 50 2.3 Set Operations 54 2.4 Events and Fields 58 2.5 Summary of a Random Experiment 64 2.6 Measure Theory 64 2.7 Axioms of Probability 68 2.8 Basic Probability Results 69 2.9 Conditional Probability 71 2.10 Independence 73 2.11 Bayes' Formula 74 2.12 Total Probability 76 2.13 Discrete Sample Spaces 79 2.14 Continuous Sample Spaces 83 2.15 Nonmeasurable Subsets of R 84 Problems 87 Further Reading 90 3 Random Variables 91 3.1 Introduction 91 3.2 Functions and Mappings 91 3.3 Distribution Function 96 3.4 Probability Mass Function 101 3.5 Probability Density Function 103 3.6 Mixed Distributions 104 3.7 Parametric Models for Random Variables 107 3.8 Continuous Random Variables 109 3.9 Discrete Random Variables 151 Problems 173 Further Reading 176 4 Multiple Random Variables 177 4.1 Introduction 177 4.2 Random Variable Approximations 177 4.3 Joint and Marginal Distributions 183 4.4 Independent Random Variables 186 4.5 Conditional Distribution 187 4.6 Random Vectors 190 4.7 Generating Dependent Random Variables 201 4.8 Random Variable Transformations 205 4.9 Important Functions of Two Random Variables 218 4.10 Transformations of Random Variable Families 226 4.11 Transformations of Random Vectors 229 4.12 Sample Mean X and Sample Variance S2 232 4.13 Minimum, Maximum, and Order Statistics 234 4.14 Mixtures 238 Problems 240 Further Reading 243 5 Expectation and Moments 244 5.1 Introduction 244 5.2 Expectation and Integration 244 5.3 Indicator Random Variable 245 5.4 Simple Random Variable 246 5.5 Expectation for Discrete Sample Spaces 247 5.6 Expectation for Continuous Sample Spaces 250 5.7 Summary of Expectation 253 5.8 Functional View of the Mean 254 5.9 Properties of Expectation 255 5.10 Expectation of a Function 259 5.11 Characteristic Function 260 5.12 Conditional Expectation 265 5.13 Properties of Conditional Expectation 267 5.14 Location Parameters: Mean, Median, and Mode 276 5.15 Variance, Covariance, and Correlation 280 5.16 Functional View of the Variance 283 5.17 Expectation and the Indicator Function 284 5.18 Correlation Coefficients 285 5.19 Orthogonality 291 5.20 Correlation and Covariance Matrices 294 5.21 Higher Order Moments and Cumulants 296 5.22 Functional View of Skewness 302 5.23 Functional View of Kurtosis 303 5.24 Generating Functions 304 5.25 Fourth-Order Gaussian Moment 309 5.26 Expectations of Nonlinear Transformations 310 Problems 313 Further Reading 316 PART II Random Processes, Systems, and Parameter Estimation 6 Random Processes 319 6.1 Introduction 319 6.2 Characterizations of a Random Process 319 6.3 Consistency and Extension 324 6.4 Types of Random Processes 325 6.5 Stationarity 326 6.6 Independent and Identically Distributed 329 6.7 Independent Increments 331 6.8 Martingales 333 6.9 Markov Sequence 338 6.10 Markov Process 350 6.11 Random Sequences 352 6.12 Random Processes 359 Problems 375 Further Reading 379 7 Stochastic Convergence, Calculus, and Decompositions 380 7.1 Introduction 380 7.2 Stochastic Convergence 380 7.3 Laws of Large Numbers 388 7.4 Central Limit Theorem 390 7.5 Stochastic Continuity 394 7.6 Derivatives and Integrals 404 7.7 Differential Equations 414 7.8 Difference Equations 422 7.9 Innovations and Mean-Square Predictability 423 7.10 Doob-Meyer Decomposition 428 7.11 Karhunen-Lo`eve Expansion 433 Problems 441 Further Reading 444 8 Systems, Noise, and Spectrum Estimation 445 8.1 Introduction 445 8.2 Correlation Revisited 445 8.3 Ergodicity 448 8.4 Eigenfunctions of RXX(tau ) 456 8.5 Power Spectral Density 457 8.6 Power Spectral Distribution 463 8.7 Cross-Power Spectral Density 465 8.8 Systems with Random Inputs 468 8.9 Passband Signals 476 8.10 White Noise 479 8.11 Bandwidth 484 8.12 Spectrum Estimation 487 8.13 Parametric Models 500 8.14 System Identification 513 Problems 515 Further Reading 518 9 Sufficient Statistics and Parameter Estimation 519 9.1 Introduction 519 9.2 Statistics 519 9.3 Sufficient Statistics 520 9.4 Minimal Sufficient Statistic 525 9.5 Exponential Families 528 9.6 Location-Scale Families 533 9.7 Complete Statistic 536 9.8 Rao-Blackwell Theorem 538 9.9 Lehmann-Scheff
e Theorem 540 9.10 Bayes Estimation 542 9.11 Mean-Square-Error Estimation 545 9.12 Mean-Absolute-Error Estimation 552 9.13 Orthogonality Condition 553 9.14 Properties of Estimators 555 9.15 Maximum A Posteriori Estimation 561 9.16 Maximum Likelihood Estimation 567 9.17 Likelihood Ratio Test 569 9.18 Expectation-Maximization Algorithm 570 9.19 Method of Moments 576 9.20 Least-Squares Estimation 577 9.21 Properties of LS Estimators 582 9.22 Best Linear Unbiased Estimation 586 9.23 Properties of BLU Estimators 590 Problems 592 Further Reading 595 A Note on Part III of the Book 595 APPENDICES Introduction to Appendices 597 A Summaries of Univariate Parametric Distributions 599 A.1 Notation 599 A.2 Further Reading 600 A.3 Continuous Random Variables 601 A.4 Discrete Random Variables 623 B Functions and Properties 634 B.1 Continuity and Bounded Variation 634 B.2 Supremum and Infimum 640 B.3 Order Notation 640 B.4 Floor and Ceiling Functions 641 B.5 Convex and Concave Functions 641 B.6 Even and Odd Functions 641 B.7 Signum Function 643 B.8 Dirac Delta Function 644 B.9 Kronecker Delta Function 645 B.10 Unit-Step Functions 646 B.11 Rectangle Functions 647 B.12 Triangle and Ramp Functions 647 B.13 Indicator Functions 648 B.14 Sinc Function 649 B.15 Logarithm Functions 650 B.16 Gamma Functions 651 B.17 Beta Functions 653 B.18 Bessel Functions 655 B.19 Q-Function and Error Functions 655 B.20 Marcum Q-Function 659 B.21 Zeta Function 659 B.22 Rising and Falling Factorials 660 B.23 Laguerre Polynomials 661 B.24 Hypergeometric Functions 662 B.25 Bernoulli Numbers 663 B.26 Harmonic Numbers 663 B.27 Euler-Mascheroni Constant 664 B.28 Dirichlet Function 664 Further Reading 664 C Frequency-Domain Transforms and Properties 665 C.1 Laplace Transform 665 C.2 Continuous-Time Fourier Transform 669 C.3 z-Transform 670 C.4 Discrete-Time Fourier Transform 676 Further Reading 677 D Integration and Integrals 678 D.1 Review of Riemann Integral 678 D.2 Riemann-Stieltjes Integral 681 D.3 Lebesgue Integral 684 D.4 Pdf Integrals 688 D.5 Indefinite and Definite Integrals 690 D.6 Integral Formulas 692 D.7 Double Integrals of Special Functions 692 Further Reading 696 E Identities and Infinite Series 697 E.1 Zero and Infinity 697 E.2 Minimum and Maximum 697 E.3 Trigonometric Identities 698 E.4 Stirling's Formula 698 E.5 Taylor Series 699 E.6 Series Expansions and Closed-Form Sums 699 E.7 Vandermonde's Identity 702 E.8 Pmf Sums and Functional Forms 703 E.9 Completing the Square 704 E.10 Summation by Parts 705 Further Reading 706 F Inequalities and Bounds for Expectations 707 F.1 Cauchy-Schwarz and H
older Inequalities 707 F.2 Triangle and Minkowski Inequalities 708 F.3 Bienaym
e, Chebyshev, and Markov Inequalities 709 F.4 Chernoff's Inequality 711 F.5 Jensen's Inequality 713 F.6 Cram
er-Rao Inequality 714 Further Reading 718 G Matrix and Vector Properties 719 G.1 Basic Properties 719 G.2 Four Fundamental Subspaces 721 G.3 Eigendecomposition 722 G.4 LU, LDU, and Cholesky Decompositions 724 G.5 Jacobian Matrix and the Jacobian 726 G.6 Kronecker and Schur Products 728 G.7 Properties of Trace and Determinant 728 G.8 Matrix Inversion Lemma 729 G.9 Cauchy-Schwarz Inequality 730 G.10 Differentiation 730 G.11 Complex Differentiation 731 Further Reading 732 GLOSSARY 733 REFERENCES 743 INDEX 755 PART III Applications in Signal Processing and Communications Chapters at the Web Site www.wiley.com/go/randomprocesses 10 Communication Systems and Information Theory 771 10.1 Introduction 771 10.2 Transmitter 771 10.3 Transmission Channel 783 10.4 Receiver 786 10.5 Information Theory 803 Problems 821 Further Reading 824 11 Optimal Filtering www.wiley.com/go/randomprocesses 825 11.1 Introduction 825 11.2 Optimal Linear Filtering 825 11.3 Optimal Filter Applications 827 11.4 Noncausal Wiener Filter 829 11.5 Causal Wiener Filter 831 11.6 Prewhitening Filter 837 11.7 FIR Wiener Filter 839 11.8 Kalman Filter 844 11.9 Steady-State Kalman Filter 851 11.10 Linear Predictive Coding 857 11.11 Lattice Prediction-Error Filter 861 11.12 Levinson-Durbin Algorithm 865 11.13 Least-Squares Filtering 868 11.14 Recursive Least-Squares 872 Problems 876 Further Reading 879 12 Adaptive Filtering www.wiley.com/go/randomprocesses 880 12.1 Introduction 880 12.2 MSE Properties 880 12.3 Steepest Descent 889 12.4 Newton's Method 894 12.5 LMS Algorithm 895 12.6 Modified LMS Algorithms 911 12.7 Adaptive IIR Filtering 923 Problems 936 Further Reading 939 13 Equalization, Beamforming, and Direction Finding www.wiley.com/go/randomprocesses 940 13.1 Introduction 940 13.2 Channel Equalization 941 13.3 Optimal Bussgang Algorithm 943 13.4 Blind Equalizer Algorithms 949 13.5 CMA Performance Surface 952 13.6 Antenna Arrays 958 13.7 Beampatterns 960 13.8 Optimal Beamforming 962 13.9 Adaptive Beamforming 970 13.10 Direction Finding 981 Problems 985 Further Reading 989