Forewords xi
Preface xix
About the Author xxiii
1 Introduction 1
1.1 Autonomous Systems 3
1.2 The Role of Machine Learning in Autonomous Systems 4
1.3 System Identification - an Abstract Model of the Real World 6
1.4 Online versus Offline Identification 9
1.5 Adaptive and Evolving Systems 10
1.6 Evolving or Evolutionary Systems 11
1.7 Supervised versus Unsupervised Learning 13
1.8 Structure of the Book 14
PART I FUNDAMENTALS
2 Fundamentals of Probability Theory 19
2.1 Randomness and Determinism 20
2.2 Frequentistic versus Belief-Based Approach 22
2.3 Probability Densities and Moments 23
2.4 Density Estimation - Kernel-Based Approach 26
2.5 Recursive Density Estimation (RDE) 28
2.6 Detecting Novelties/Anomalies/Outliers using RDE 32
2.7 Conclusions 36
3 Fundamentals of Machine Learning and Pattern Recognition 37
3.1 Preprocessing 37
3.2 Clustering 42
3.3 Classification 56
3.4 Conclusions 58
4 Fundamentals of Fuzzy Systems Theory 61
4.1 Fuzzy Sets 61
4.2 Fuzzy Systems, Fuzzy Rules 64
4.3 Fuzzy Systems with Nonparametric Antecedents (AnYa) 69
4.4 FRB (Offline) Classifiers 73
4.5 Neurofuzzy Systems 75
4.6 State Space Perspective 79
4.7 Conclusions 81
PART II METHODOLOGY OF AUTONOMOUS LEARNING SYSTEMS
5 Evolving System Structure from Streaming Data 85
5.1 Defining System Structure Based on Prior Knowledge 85
5.2 Data Space Partitioning 86
5.3 Normalisation and Standardisation of Streaming Data in an Evolving Environment 96
5.4 Autonomous Monitoring of the Structure Quality 98
5.5 Short- and Long-Term Focal Points and Submodels 104
5.6 Simplification and Interpretability Issues 105
5.7 Conclusions 107
6 Autonomous Learning Parameters of the Local Submodels 109
6.1 Learning Parameters of Local Submodels 110
6.2 Global versus Local Learning 111
6.3 Evolving Systems Structure Recursively 113
6.4 Learning Modes 116
6.5 Robustness to Outliers in Autonomous Learning 118
6.6 Conclusions 118
7 Autonomous Predictors, Estimators, Filters, Inferential Sensors 121
7.1 Predictors, Estimators, Filters - Problem Formulation 121
7.2 Nonlinear Regression 123
7.3 Time Series 124
7.4 Autonomous Learning Sensors 125
7.5 Conclusions 131
8 Autonomous Learning Classifiers 133
8.1 Classifying Data Streams 133
8.2 Why Adapt the Classifier Structure? 134
8.3 Architecture of Autonomous Classifiers of the Family AutoClassify 135
8.4 Learning AutoClassify from Streaming Data 139
8.5 Analysis of AutoClassify 140
8.6 Conclusions 140
9 Autonomous Learning Controllers 143
9.1 Indirect Adaptive Control Scheme 144
9.2 Evolving Inverse Plant Model from Online Streaming Data 145
9.3 Evolving Fuzzy Controller Structure from Online Streaming Data 147
9.4 Examples of Using AutoControl 148
9.5 Conclusions 153
10 Collaborative Autonomous Learning Systems 155
10.1 Distributed Intelligence Scenarios 155
10.2 Autonomous Collaborative Learning 157
10.3 Collaborative Autonomous Clustering, AutoCluster by a Team of ALSs 158
10.4 Collaborative Autonomous Predictors, Estimators, Filters and AutoSense by a Team of ALSs 159
10.5 Collaborative Autonomous Classifiers AutoClassify by a Team of ALSs 160
10.6 Superposition of Local Submodels 161
10.7 Conclusions 161
PART III APPLICATIONS OF ALS
11 Autonomous Learning Sensors for Chemical and Petrochemical Industries 165
11.1 Case Study 1: Quality of the Products in an Oil Refinery 165
11.2 Case Study 2: Polypropylene Manufacturing 172
11.3 Conclusions 178
12 Autonomous Learning Systems in Mobile Robotics 179
12.1 The Mobile Robot Pioneer 3DX 179
12.2 Autonomous Classifier for Landmark Recognition 180
12.3 Autonomous Leader Follower 193
12.4 Results Analysis 196
13 Autonomous Novelty Detection and Object Tracking in Video Streams 197
13.1 Problem Definition 197
13.2 Background Subtraction and KDE for Detecting Visual Novelties 198
13.3 Detecting Visual Novelties with the RDE Method 203
13.4 Object Identification in Image Frames Using RDE 204
13.5 Real-time Tracking in Video Streams Using ALS 206
13.6 Conclusions 209
14 Modelling Evolving User Behaviour with ALS 211
14.1 User Behaviour as an Evolving Phenomenon 211
14.2 Designing the User Behaviour Profile 212
14.3 Applying AutoClassify0 for Modelling Evolving User Behaviour 215
14.4 Case Studies 216
14.5 Conclusions 221
15 Epilogue 223
15.1 Conclusions 223
15.2 Open Problems 227
15.3 Future Directions 227
APPENDICES
Appendix A Mathematical Foundations 231
Appendix B Pseudocode of the Basic Algorithms 235
References 245
Glossary 259
Index 263
