Variational Bayesian Learning Theory

Variational Bayesian learning is one of the most popular methods in machine learning. Designed for researchers and graduate students in machine learning, this book summarizes recent developments in the non-asymptotic and asymptotic theory of variational Bayesian learning and suggests how this theory can be applied in practice. The authors begin by developing a basic framework with a focus on conjugacy, which enables the reader to derive tractable algorithms. Next, it summarizes non-asymptotic theory, which, although limited in application to bilinear models, precisely describes the behavior of the variational Bayesian solution and reveals its sparsity inducing mechanism. Finally, the text summarizes asymptotic theory, which reveals phase transition phenomena depending on the prior setting, thus providing suggestions on how to set hyperparameters for particular purposes. Detailed derivations allow readers to follow along without prior knowledge of the mathematical techniques specific to Bayesian learning. Cover Front Matter Variational Bayesian Learning Theory Copyright Contents Preface Nomenclature Part I: Formulation 1 Bayesian Learning 2 Variational Bayesian Learning Part II: Algorithm 3 VB Algorithm for Multilinear Models 4 VB Algorithm for Latent Variable Models 5 VB Algorithm under No Conjugacy Part III: Nonasymptotic Theory 6 Global VB Solution of Fully Observed Matrix Factorization 7 Model-Induced Regularization and Sparsity Inducing Mechanism 8 Performance Analysis of VB Matrix Factorization 9 Global Solver for Matrix Factorization 10 Global Solver for Low-Rank Subspace Clustering 11 Efficient Solver for Sparse Additive Matrix Factorization 12 MAP and Partially Bayesian Learning Part IV: Asymptotic Theory 13 Asymptotic Learning Theory 14 Asymptotic VB Theory of Reduced Rank Regression 15 Asymptotic VB Theory of Mixture Models 16 Asymptotic VB Theory of Other Latent Variable Models 17 Unified Theory for Latent Variable Models Appendix A. James–Stein Estimator Appendix B. Metric in Parameter Space Appendix C. Detailed Description of Overlap Method Appendix D. Optimality of Bayesian Learning Bibliography Subject Index Designed for researchers and graduate students in machine learning, this book introduces the theory of variational Bayesian learning, a popular machine learning method, and suggests how to make use of it in practice. Detailed derivations allow readers to follow along without prior knowledge of the specific mathematical techniques. This Introduction To The Theory Of Variational Bayesian Learning Summarizes Recent Developments And Suggests Practical Applications.