This book is the outcome of a decade’s research into a speci?c architecture and associated learning mechanism for an arti?cial neural network: the - chitecture involves negative feedback and the learning mechanism is simple Hebbian learning. The research began with my own thesis at the University of Strathclyde, Scotland, under Professor Douglas McGregor which culminated with me being awarded a PhD in 1995 [52], the title of which was “Negative Feedback as an Organising Principle for Arti?cial Neural Networks”. Naturally enough, having established this theme, when I began to sup- vise PhD students of my own, we continued to develop this concept and this book owes much to the research and theses of these students at the Applied Computational Intelligence Research Unit in the University of Paisley. Thus we discuss work from • Dr. Darryl Charles [24] in Chapter 5. • Dr. Stephen McGlinchey [127] in Chapter 7. • Dr. Donald MacDonald [121] in Chapters 6 and 8. • Dr. Emilio Corchado [29] in Chapter 8. We brie?y discuss one simulation from the thesis of Dr. Mark Girolami [58] in Chapter 6 but do not discuss any of the rest of his thesis since it has already appeared in book form [59]. We also must credit Cesar Garcia Osorio, a current PhD student, for the comparative study of the two Exploratory Projection Pursuit networks in Chapter 8. All of Chapters 3 to 8 deal with single stream arti?cial neural networks. Contents......Page 3 Acronyms......Page 11 Preface......Page 12 Introduction......Page 14 Artificial Neural Networks......Page 15 The Organisation of this Book......Page 16 --- Single Stream Networks......Page 19 Hebbian Learning......Page 22 Quantification of Information......Page 24 Principal Component Analysis......Page 27 Weight Decay in Hebbian Learning......Page 29 ANNs and PCA......Page 32 Anti-Hebbian Learning......Page 34 Independent Component Analysis......Page 36 Conclusion......Page 40 Introduction......Page 41 Model......Page 51 Using Distance Differences......Page 56 Minor Components Analysis......Page 58 Conclusion......Page 64 Peer-Inhibitory Neurons......Page 67 Analysis of Differential Learning Rates......Page 69 Differential Activation Functions......Page 80 Emergent Properties of the Peer-Inhibition Network......Page 92 Conclusion......Page 93 Multiple Cause Data......Page 95 Non-negative Weights......Page 98 Factor Analysis......Page 102 Conclusion......Page 118 Exploratory Data Analysis......Page 120 Exploratory Projection Pursuit......Page 121 The Data and Sphering......Page 122 The Projection Pursuit Network......Page 123 Other Indices......Page 132 Using Exploratory Projection Pursuit......Page 136 Independent Component Analysis......Page 142 Conclusion......Page 145 Background......Page 146 The Classification Network......Page 149 The Scale Invariant Map......Page 152 The Subspace Map......Page 158 The Negative Feedback Coding Network......Page 167 Conclusion......Page 176 The Negative Feedback Network and Cost Functions......Page 178 -Insensitive Hebbian Learning......Page 180 The Maximum Likelihood EPP Algorithm......Page 185 A Combined Algorithm......Page 190 Conclusion......Page 195 --- Dual Stream Networks......Page 196 Statistical Canonical Correlation Analysis......Page 199 The First Canonical Correlation Network......Page 200 Experimental Results......Page 202 A Second Neural Implementation of CCA......Page 210 Simulations......Page 212 Linear Discriminant Analysis......Page 214 Discussion......Page 215 A Probabilistic Perspective......Page 217 Robust CCA......Page 219 A Model Derived from Becker’s Model 1......Page 220 Discussion......Page 223 Nonlinear Correlations......Page 225 The Search for Independence......Page 229 Kernel Canonical Correlation Analysis......Page 233 Relevance Vector Regression......Page 242 Appearance-Based Object Recognition......Page 245 Mixtures of Linear Correlations......Page 248 Exploratory Correlation Analysis......Page 255 Experiments......Page 259 Connection to CCA......Page 261 FastECA......Page 262 Local Filter Formation From Natural Stereo Images......Page 264 Twinned Maximum Likelihood Learning......Page 274 Unmixing of Sound Signals......Page 278 Conclusion......Page 279 Multicollinearity & Partial Least Squares......Page 282 Application to CCA......Page 283 Extracting Multiple Canonical Correlations......Page 287 Experiments on Multicollinear Data......Page 288 A Neural Implementation of Partial Least Squares......Page 291 Conclusion......Page 295 Twinned Principal Curves......Page 297 Properties of Twinned Principal Curves......Page 299 Twinned Self-Organising Maps......Page 311 Discussion......Page 313 Review......Page 314 Current and Future Work......Page 317 A.1 The Interneuron Model......Page 319 A.2 Other Models......Page 321 A.3 Related Biological Models......Page 324 B.1 F ̈oldi ́ak’s Sixth Model......Page 326 B.2 Competitive Hebbian Learning......Page 329 B.3 Multiple Cause Models......Page 330 B.4 Predictability Minimisation......Page 333 B.5 Mixtures of Experts......Page 335 B.6 Probabilistic Models......Page 337 C.1 Jutten and Herault......Page 344 C.2 Nonlinear PCA......Page 347 C.3 Information Maximisation......Page 348 C.4 Penalised Minimum Reconstruction Error......Page 353 C.5 FastICA......Page 354 D.1 The I-Max Model......Page 356 D.2 Stone’s Model......Page 358 D.3 Kay’s Neural Models......Page 359 D.4 Borga’s Algorithm......Page 361 E.1 Artificial Data Sets......Page 366 E.2 Real Data Sets......Page 369 Refs......Page 373 Index......Page 382
The central idea of Hebbian Learning and Negative Feedback Networks is that artificial neural networks using negative feedback of activation can use simple Hebbian learning to self-organise so that they uncover interesting structures in data sets. Two variants are considered: the first uses a single stream of data to self-organise. By changing the learning rules for the network, it is shown how to perform Principal Component Analysis, Exploratory Projection Pursuit, Independent Component Analysis, Factor Analysis and a variety of topology preserving mappings for such data sets.
The second variants use two input data streams on which they self-organise. In their basic form, these networks are shown to perform Canonical Correlation Analysis, the statistical technique which finds those filters onto which projections of the two data streams have greatest correlation.
The book encompasses a wide range of real experiments and displays how the approaches it formulates can be applied to the analysis of real problems.
"The central idea of Hebbian Learning and Negative Feedback Networks is that artificial neural networks using negative feedback of activation can use simple Hebbian learning to self-organise in such a way that they uncover interesting structures in data sets." "The book encompasses a wide range of real experiments and displays how the approaches it formulates can be applied to the analysis of real problems. It will be of particular interest to postgraduates and academic or industrial researchers in the neuroscience community."--Jacket
