About the Author xv
Series Preface xix
Preface xxiii
Volume 1
Part 1 Basic Concepts of Algebra 1
1 Scalars, Vectors, Matrices, and Determinants 3
2 Function Features 7
2.1 Series 17
2.1.1 Arithmetic Series 17
2.1.2 Geometric Series 19
2.1.3 Arithmetic/Geometric Series 22
2.2 Multiplication and Division of Polynomials 26
2.2.1 Product 27
2.2.2 Quotient 28
2.2.3 Factorization 31
2.2.4 Splitting 35
2.2.5 Power 43
2.3 Trigonometric Functions 52
2.3.1 Definition and Major Features 52
2.3.2 Angle Transformation Formulae 57
2.3.3 Fundamental Theorem of Trigonometry 73
2.3.4 Inverse Functions 79
2.4 Hyperbolic Functions 80
2.4.1 Definition and Major Features 80
2.4.2 Argument Transformation Formulae 85
2.4.3 Euler's Form of Complex Numbers 89
2.4.4 Inverse Functions 90
3 Vector Operations 97
3.1 Addition of Vectors 99
3.2 Multiplication of Scalar by Vector 101
3.3 Scalar Multiplication of Vectors 103
3.4 Vector Multiplication of Vectors 111
4 Matrix Operations 119
4.1 Addition of Matrices 120
4.2 Multiplication of Scalar by Matrix 121
4.3 Multiplication of Matrices 124
4.4 Transposal of Matrices 131
4.5 Inversion of Matrices 133
4.5.1 Full Matrix 134
4.5.2 Block Matrix 138
4.6 Combined Features 140
4.6.1 Symmetric Matrix 141
4.6.2 Positive Semidefinite Matrix 142
5 Tensor Operations 145
6 Determinants 151
6.1 Definition 152
6.2 Calculation 157
6.2.1 Laplace's Theorem 159
6.2.2 Major Features 161
6.2.3 Tridiagonal Matrix 177
6.2.4 Block Matrix 179
6.2.5 Matrix Inversion 181
6.3 Eigenvalues and Eigenvectors 185
6.3.1 Characteristic Polynomial 186
6.3.2 Cayley-Hamilton's Theorem 190
7 Solution of Algebraic Equations 199
7.1 Linear Systems of Equations 199
7.1.1 Jacobi's Method 203
7.1.2 Explicitation 212
7.1.3 Cramer's Rule 213
7.1.4 Matrix Inversion 216
7.2 Quadratic Equation 220
7.3 Lambert's W Function 224
7.4 Numerical Approaches 228
7.4.1 Double-initial Estimate Methods 229
7.4.1.1 Bisection 229
7.4.1.2 Linear Interpolation 232
7.4.2 Single-initial Estimate Methods 242
7.4.2.1 Newton and Raphson's Method 242
7.4.2.2 Direct Iteration 250
Further Reading 255
Volume 2
Part 2 Basic Concepts of Calculus 259
8 Limits, Derivatives, Integrals, and Differential Equations 261
9 Limits and Continuity 263
9.1 Univariate Limit 263
9.1.1 Definition 263
9.1.2 Basic Calculation 267
9.2 Multivariate Limit 271
9.3 Basic Theorems on Limits 272
9.4 Definition of Continuity 280
9.5 Basic Theorems on Continuity 282
9.5.1 Bolzano's Theorem 282
9.5.2 Weierstrass' Theorem 286
10 Differentials, Derivatives, and Partial Derivatives 291
10.1 Differential 291
10.2 Derivative 294
10.2.1 Definition 294
10.2.1.1 Total Derivative 295
10.2.1.2 Partial Derivatives 300
10.2.1.3 Directional Derivatives 307
10.2.2 Rules of Differentiation of Univariate Functions 308
10.2.3 Rules of Differentiation of Multivariate Functions 325
10.2.4 Implicit Differentiation 325
10.2.5 Parametric Differentiation 327
10.2.6 Basic Theorems of Differential Calculus 331
10.2.6.1 Rolle's Theorem 331
10.2.6.2 Lagrange's Theorem 332
10.2.6.3 Cauchy's Theorem 334
10.2.6.4 L'Hôpital's Rule 337
10.2.7 Derivative of Matrix 349
10.2.8 Derivative of Determinant 356
10.3 Dependence Between Functions 358
10.4 Optimization of Univariate Continuous Functions 362
10.4.1 Constraint-free 362
10.4.2 Subjected to Constraints 364
10.5 Optimization of Multivariate Continuous Functions 367
10.5.1 Constraint-free 367
10.5.2 Subjected to Constraints 371
11 Integrals 373
11.1 Univariate Integral 374
11.1.1 Indefinite Integral 374
11.1.1.1 Definition 374
11.1.1.2 Rules of Integration 377
11.1.2 Definite Integral 386
11.1.2.1 Definition 386
11.1.2.2 Basic Theorems of Integral Calculus 393
11.1.2.3 Reduction Formulae 396
11.2 Multivariate Integral 400
11.2.1 Definition 400
11.2.1.1 Line Integral 400
11.2.1.2 Double Integral 403
11.2.2 Basic Theorems 404
11.2.2.1 Fubini's Theorem 404
11.2.2.2 Green's Theorem 409
11.2.3 Change of Variables 411
11.2.4 Differentiation of Integral 414
11.3 Optimization of Single Integral 416
11.4 Optimization of Set of Derivatives 424
12 Infinite Series and Integrals 429
12.1 Definition and Criteria of Convergence 429
12.1.1 Comparison Test 430
12.1.2 Ratio Test 431
12.1.3 D'Alembert's Test 432
12.1.4 Cauchy's Integral Test 434
12.1.5 Leibnitz's Test 436
12.2 Taylor's Series 437
12.2.1 Analytical Functions 451
12.2.1.1 Exponential Function 451
12.2.1.2 Hyperbolic Functions 458
12.2.1.3 Logarithmic Function 459
12.2.1.4 Trigonometric Functions 463
12.2.1.5 Inverse Trigonometric Functions 466
12.2.1.6 Powers of Binomials 476
12.2.2 Euler's Infinite Product 479
12.3 Gamma Function and Factorial 488
12.3.1 Integral Definition and Major Features 489
12.3.2 Euler's Definition 494
12.3.3 Stirling's Approximation 499
13 Analytical Geometry 505
13.1 Straight Line 505
13.2 Simple Polygons 508
13.3 Conical Curves 510
13.4 Length of Line 516
13.5 Curvature of Line 525
13.6 Area of Plane Surface 530
13.7 Outer Area of Revolution Solid 536
13.8 Volume of Revolution Solid 552
14 Transforms 559
14.1 Laplace's Transform 559
14.1.1 Definition 559
14.1.2 Major Features 571
14.1.3 Inversion 583
14.2 Legendre's Transform 590
15 Solution of Differential Equations 597
15.1 Ordinary Differential Equations 597
15.1.1 First Order 598
15.1.1.1 Nonlinear 598
15.1.1.2 Linear 600
15.1.2 Second Order 602
15.1.2.1 Nonlinear 603
15.1.2.2 Linear 613
15.1.3 Linear Higher Order 650
15.2 Partial Differential Equations 660
16 Vector Calculus 667
16.1 Rectangular Coordinates 667
16.1.1 Definition and Representation 667
16.1.2 Definition of Nabla Operator, ¿ 668
16.1.3 Algebraic Properties of ¿ 673
16.1.4 Multiple Products Involving ¿ 676
16.1.4.1 Calculation of (¿.¿)¿ 676
16.1.4.2 Calculation of (¿.¿)u 676
16.1.4.3 Calculation of ¿.(¿u) 677
16.1.4.4 Calculation of ¿.(¿ × u) 679
16.1.4.5 Calculation of ¿.(¿¿¿) 680
16.1.4.6 Calculation of ¿.(uu) 682
16.1.4.7 Calculation of ¿ × (¿ ¿) 684
16.1.4.8 Calculation of ¿(¿.u) 685
16.1.4.9 Calculation of (u.¿)u 690
16.1.4.10 Calculation of ¿.(¿.u) 693
16.2 Cylindrical Coordinates 695
16.2.1 Definition and Representation 695
16.2.2 Redefinition of Nabla Operator, ¿ 700
16.3 Spherical Coordinates 705
16.3.1 Definition and Representation 705
16.3.2 Redefinition of Nabla Operator, ¿ 715
16.4 Curvature of Three-dimensional Surfaces 729
16.5 Three-dimensional Integration 737
17 Numerical Approaches to Integration 741
17.1 Calculation of Definite Integrals 741
17.1.1 Zeroth Order Interpolation 743
17.1.2 First- and Second-Order Interpolation 750
17.1.2.1 Trapezoidal Rule 751
17.1.2.2 Simpson's Rule 754
17.1.2.3 Higher Order Interpolation 768
17.1.3 Composite Methods 771
17.1.4 Infinite and Multidimensional Integrals 775
17.2 Integration of Differential Equations 777
17.2.1 Single-step Methods 779
17.2.2 Multistep Methods 782
17.2.3 Multistage Methods 790
17.2.3.1 First Order 790
17.2.3.2 Second Order 790
17.2.3.3 General Order 793
17.2.4 Integral Versus Differential Equation 801
Part 3 Basic Concepts of Statistics 807
18 Continuous Probability Functions 809
18.1 Basic Statistical Descriptors 810
18.2 Normal Distribution 815
18.2.1 Derivation 816
18.2.2 Justification 821
18.2.3 Operational Features 826
18.2.4 Moment-generating Function 829
18.2.4.1 Single Variable 829
18.2.4.2 Multiple Variables 835
18.2.5 Standard Probability Density Function 842
18.2.6 Central Limit Theorem 845
18.2.7 Standard Probability Cumulative Function 855
18.3 Other Relevant Distributions 858
18.3.1 Lognormal Distribution 858
18.3.1.1 Probability Density Function 858
18.3.1.2 Mean and Variance 859
18.3.1.3 Probability Cumulative Function 862
18.3.1.4 Mode and Median 863
18.3.2 Chi-square Distribution 865
18.3.2.1 Probability Density Function 865
18.3.2.2 Mean and Variance 869
18.3.2.3 Asymptotic Behavior 870
18.3.2.4 Probability Cumulative Function 872
18.3.2.5 Mode and Median 873
18.3.2.6 Other Features 874
18.3.3 Student's t-distribution 876
18.3.3.1 Probability Density Function 876
18.3.3.2 Mean and Variance 879
18.3.3.3 Asymptotic Behavior 883
18.3.3.4 Probability Cumulative Function 886
18.3.3.5 Mode and Median 887
18.3.4 Fisher's F-distribution 888
18.3.4.1 Probability Density Function 888
18.3.4.2 Mean and Variance 893
18.3.4.3 Asymptotic Behavior 896
18.3.4.4 Probability Cumulative Function 899
18.3.4.5 Mode and Median 902
18.3.4.6 Other Features 903
19 Statistical Hypothesis Testing 915
20 Linear Regression 923
20.1 Parameter Fitting 924
20.2 Residual Characterization 927
20.3 Parameter Inference 931
20.3.1 Multivariate Models 931
20.3.2 Univariate Models 934
20.4 Unbiased Estimation 937
20.4.1 Multivariate Models 937
20.4.2 Univariate Models 940
20.5 Prediction Inference 949
20.6 Multivariate Correction 951
Further Reading 963
