Algebraic and Geometric Methods in Statistics

This up-to-date account of algebraic statistics and information geometry explores the emerging connections between the two disciplines, demonstrating how they can be used in design of experiments and how they benefit our understanding of statistical models, in particular, exponential models. This book presents a new way of approaching classical statistical problems and raises scientific questions that would never have been considered without the interaction of these two disciplines. Beginning with a brief introduction to each area, using simple illustrative examples, the book then proceeds with a collection of reviews and some new results written by leading researchers in their respective fields. Part III dwells in both classical and quantum information geometry, containing surveys of key results and new material. Finally, Part IV provides examples of the interplay between algebraic statistics and information geometry. Computer code and proofs are also available online, where key examples are developed in further detail. Cover......Page 1 Half-title......Page 3 Title......Page 5 Copyright......Page 6 Dedication......Page 7 Contents......Page 9 List of contributors......Page 11 Preface......Page 15 Frequently used notations and symbols......Page 18 1.1 Introduction......Page 19 1.2 Explicit versus implicit algebraic models......Page 20 Example 1.1......Page 21 1.2.1 Design......Page 22 1.3.1 Model structure......Page 24 1.3.2 Inference......Page 27 1.3.3 Cumulants and moments......Page 29 1.4 Information geometry on the simplex......Page 30 1.4.2 Paths on the simplex......Page 31 1.6 Fisher information......Page 32 1.6.1 The generalised Pythagorean theorem......Page 34 1.7 Appendix: a summary of commutative algebra (with Roberto Notari)......Page 35 Definition 1.3......Page 36 Proposition 1.4......Page 37 Corollary 1.1......Page 38 Definition 1.14......Page 39 Proposition 1.13......Page 40 References......Page 41 Part I Contingency tables......Page 43 2.1 Introduction......Page 45 2.2 Latent class models for contingency tables......Page 46 2.3 Geometric description of latent class models......Page 49 Example 2.1......Page 50 2.4.1 Effective dimension and polynomials......Page 52 Example 2.3......Page 53 2.4.2 The 100 Swiss Franc problem......Page 54 2.5.1 Example: Michigan influenza......Page 65 2.5.2 Data from the National Long Term Care Survey......Page 66 2.6.1 Introduction and motivation......Page 70 Question 2.2......Page 71 Proposition 2.1......Page 72 Question 2.3......Page 73 Proposition 2.2......Page 74 2.1 Theorem......Page 75 Proposition 2.3......Page 76 2.7 Conclusions......Page 77 References......Page 78 3.1 Introduction......Page 81 3.2 Definitions and notation......Page 82 3.3 Parameter surfaces and other loci for 2 × 2 tables......Page 84 3.3.1 Space of tables for fixed conditional probabilities......Page 85 Proposition 3.1......Page 86 3.4.1 Specification I......Page 87 Lemma 3.1......Page 88 Proposition 3.2......Page 89 3.4.4 Odds-ratio specification......Page 90 Lemma 3.2......Page 91 Proposition 3.3......Page 92 3.5.4 Two odds-ratios......Page 94 3.6 Extensions and discussion......Page 95 3.6.1 Simpson's paradox......Page 96 3.7 Generalisations and questions......Page 97 References......Page 98 4.1 Introduction......Page 101 4.2 Model selection......Page 102 4.3.1 Metropolis-Hastings algorithm......Page 104 4.3.2 Diaconis-Sturmfels algorithm......Page 105 Example 4.1......Page 106 4.4 Reduction of computational costs......Page 108 Theorem 4.1......Page 109 4.5 Simulation results......Page 110 4.5.2 A simulation study of p-values......Page 111 4.5.3 Results for AZT data set......Page 112 References......Page 114 5.1 Introduction......Page 117 5.2 Arbitrary margins and toric ideals......Page 118 5.3 Survey of computational methods......Page 120 Example 5.1......Page 121 Proposition 5.2......Page 122 5.5 Additional examples......Page 123 Example 5.4......Page 124 Example 5.6......Page 125 5.6 Conclusions......Page 126 References......Page 127 6.1 Introduction......Page 129 6.2 Background and definitions......Page 131 Definition 6.2......Page 132 Example 6.1......Page 133 Proposition 6.1......Page 134 6.4 Geometric description of the models......Page 135 6.5 Adding symmetry......Page 136 Example 6.2......Page 137 6.6 Final example......Page 138 References......Page 139 7.1 Introduction......Page 141 7.2 Bivariate normal random variables......Page 143 Theorem 7.1......Page 144 Theorem 7.2......Page 146 Lemma 7.1......Page 147 Lemma 7.3......Page 148 Lemma 7.4......Page 149 References......Page 150 8.1 Introduction......Page 153 8.2 Terminology and notation......Page 155 8.3 The generalised shuttle algorithm......Page 156 Proposition 8.1......Page 158 Proposition 8.2......Page 159 8.5 Calculating bounds in the decomposable case......Page 160 Lemma 8.1......Page 161 Proposition 8.4......Page 162 8.5.1 Example: Bounds for the Czech autoworkers data......Page 163 8.6 Computing sharp bounds......Page 164 8.7 Large contingency tables......Page 165 8.8 Other examples......Page 166 Example 8.3......Page 167 Example 8.4......Page 168 Example 8.5......Page 169 8.9 Conclusions......Page 170 References......Page 171 Part II Designed experiments......Page 175 9.1 Introduction......Page 177 Example 9.1......Page 179 Example 9.3......Page 180 Theorem 9.1......Page 181 Example 9.6......Page 182 Example 9.7......Page 183 Conjecture 9.1......Page 184 9.4 Interpolation over varieties......Page 185 9.5 Becker-Weispfenning interpolation......Page 186 9.6 Reduction of power series by ideals......Page 187 Example 9.12......Page 188 9.7 Discussion and further work......Page 189 References......Page 190 10.1 Introduction......Page 193 Definition 10.1......Page 195 10.3 Biochemical network inference......Page 196 Example 10.1......Page 197 Example 10.2......Page 198 Example 10.3......Page 199 Example 10.4......Page 200 10.4 Polynomial dynamical systems......Page 201 10.5 Discussion......Page 202 References......Page 203 11.1 Introduction......Page 205 11.1.1 Outline of the chapter......Page 206 Example 11.2 (Example 11.1 cont.)......Page 207 Definition 11.1......Page 208 Example 11.6......Page 209 Definition 11.4......Page 210 Example 11.9......Page 211 Theorem 11.4......Page 212 Theorem 11.5......Page 213 Example 11.13......Page 214 Proposition 11.2......Page 215 Definition 11.6......Page 216 Example 11.14 (Example 11.13 cont.)......Page 217 Theorem 11.10......Page 218 11.6 Further comments......Page 219 References......Page 220 12.1 Introduction......Page 221 12.2.1 Full factorial design......Page 222 Definition 12.1......Page 223 Example 12.2......Page 224 12.2.4 Regular fractions......Page 225 Example 12.4 (Regular fraction)......Page 226 Example 12.5 (Permutation of levels - Example 12.4 cont.)......Page 227 Proposition 12.5 (Sudoku fractions)......Page 228 Definition 12.5 (Sudoku fraction)......Page 229 Proposition 12.7......Page 230 Definition 12.7......Page 231 12.4.1 Polynomial form of M1 and M2 moves......Page 232 Proposition 12.8......Page 233 12.4.2 Polynomial form of M3 moves......Page 234 Proposition 12.10......Page 235 Example 12.10 (Example 12.8 cont.)......Page 236 Proposition 12.12......Page 237 Proposition 12.14......Page 238 Example 12.12......Page 239 12.6 Conclusions......Page 240 References......Page 241 13.1 Introduction......Page 243 13.2.1 Conditional tests for discrete observations......Page 244 13.2.2 How to define the covariate matrix......Page 247 13.3.1 Models for the full factorial designs......Page 250 Proposition 13.1......Page 251 13.3.2 Models for the regular fractional factorial designs......Page 252 13.4 Discussion......Page 254 References......Page 255 Part III Information geometry......Page 257 14.1 Parametric estimation; the Cramér-Rao inequality......Page 259 14.2 Manifolds modelled by Orlicz spaces......Page 262 Example 14.1......Page 264 14.3 Efron, Dawid and Amari......Page 265 Definition 14.1......Page 266 Definition 14.2......Page 267 Definition 14.3......Page 268 Definition 14.4......Page 269 14.4.1 Quantum Cramér-Rao inequality......Page 270 Theorem 14.1......Page 271 14.5 Perturbations by forms......Page 272 References......Page 273 15.1 The work of Pistone and Sempi......Page 275 15.2.1 The underlying set of the information manifold......Page 277 15.2.2 The quantum Cramér class......Page 278 15.3 The Orlicz norm......Page 279 Theorem 15.1......Page 280 References......Page 281 16.1 Introduction......Page 283 16.2.1 Young functions and associated norms......Page 284 16.2.3 The quantum exponential Orlicz space and its dual......Page 285 16.3 The spaces......Page 286 Theorem 16.1......Page 287 Lemma 16.3......Page 288 Lemma 16.5......Page 289 Theorem 16.5......Page 290 Lemma 16.6......Page 291 Theorem 16.7......Page 292 Theorem 16.10......Page 293 References......Page 294 17.1 Introduction......Page 295 17.2 The role of geometry in statistical modelling......Page 296 17.3.1 Geometry elicitation......Page 298 17.3.2 Estimating geometry from data......Page 299 17.4 Congruent embeddings and simplicial geometries......Page 300 Proposition 17.1 ((Cencov .1982))......Page 302 Proposition 17.2 ((Campbell 1986))......Page 304 17.5 Text documents......Page 305 17.6 Discussion......Page 307 References......Page 308 18.1.1 Reproducing kernel Hilbert space......Page 309 Proposition 18.1......Page 310 18.1.2 Exponential manifold associated with a RKHS......Page 311 Lemma 18.2......Page 312 Theorem 18.1......Page 313 Proposition 18.2......Page 314 18.1.3 Mean and covariance on reproducing kernel exponential manifolds......Page 315 Theorem 18.4......Page 316 18.2.1 Likelihood equation on a reproducing kernel exponential manifold......Page 317 18.2.2 √n-consistency of the mean parameter......Page 318 18.2.3 Pseudo maximum likelihood estimation......Page 319 Theorem 18.6......Page 320 References......Page 322 19.1 A general framework......Page 325 Proposition 19.2......Page 326 Proposition 19.4......Page 327 Proposition 19.5......Page 328 Example 19.1 (Parametric exponential model)......Page 329 19.3.2 MLE for exponential models......Page 330 Proposition 19.6......Page 331 19.3.4 The compound Poisson density model......Page 332 Theorem 19.1......Page 333 Example 19.4......Page 334 Proposition 19.7......Page 335 Example 19.5......Page 336 Theorem 19.4......Page 337 Definition 19.6......Page 338 Proposition 19.11......Page 339 Lemma 19.4......Page 340 Definition 19.7......Page 341 Theorem 19.5......Page 342 Acknowledgements......Page 343 References......Page 344 20.1.1 Aspects of classical Fisher information......Page 345 20.1.2 Quantum counterparts......Page 347 20.1.3 Quantum statistics......Page 348 20.2 Metric adjusted skew information......Page 350 Definition 20.2 (metric adjusted skew information)......Page 351 Theorem 20.1......Page 352 Theorem 20.2......Page 353 Definition 20.4......Page 354 References......Page 355 Part IV Information geometry and algebraic statistics......Page 357 21.1.1 Differential geometry......Page 359 21.1.2 Commutative algebra......Page 360 21.2 A first example: the Gibbs model......Page 361 Example 21.1......Page 363 21.3 Charts......Page 365 21.3.1 e-Manifold......Page 366 21.3.3 Sub-models and splitting......Page 367 21.3.4 Velocity......Page 368 21.3.5 General Gibbs model as sub-manifold......Page 369 21.3.6 Optimisation......Page 370 21.3.7 Exercise: location model of the Cauchy distribution......Page 371 21.4 Differential equations on the statistical manifold......Page 372 Example 21.5 (Heat equation)......Page 373 21.4.1 Deformed exponentials......Page 374 Example 21.8 (Continuous white noise)......Page 376 21.5.1 Polynomial random variables......Page 377 Example 21.9......Page 378 Example 21.11 (Quadratic exponential models)......Page 379 Example 21.12 (Polynomial density with two parameters)......Page 380 References......Page 381 Part V On-line supplements......Page 385 Coloured figures for Chapter 2......Page 387 Definition 22.1 (Ring)......Page 391 Definition 22.4 (Degree Reverse Lexicographic Ordering)......Page 392 Definition 22.7 (Reduced Grobner basis)......Page 393 Definition 22.9 (Ideal of variety)......Page 394 Definition 22.10 (Segre map)......Page 395 Theorem 22.3......Page 396 Theorem 22.6......Page 397 22.2.1 Computing the dimension of the image variety......Page 398 22.2.2 Solving Polynomial Equations......Page 400 22.2.3 Plotting Unidentifiable Space......Page 402 22.3 Proof of the Fixed Points for 100 Swiss Franks Problem......Page 404 22.4 Matlab Codes......Page 405 Bibliography......Page 412 Proposition 8.2......Page 413 Lemma 8.2......Page 415 Lemma 8.3......Page 416 Proposition 8.4......Page 417 24.2 Proofs Proposition 12.8......Page 426 Proposition 12.10......Page 428 Proposition 12.11......Page 430 Proposition......Page 431 24.3 Generation and classification of all the 4 × 4 sudoku......Page 432 24.3.1 CoCoA code for 4 × 4 sudoku......Page 433 24.3.2 4 × 4 sudoku regular fractions......Page 435 24.3.3 4 × 4 non-regular sudoku fractions......Page 437 25.1 Proofs Theorem 11.3......Page 442 Theorem 11.8......Page 443 Theorem 11.9......Page 444 Proposition 19.4......Page 445 Theorem 19.2......Page 446 Corollary 19.3......Page 448 This Up-to-date Account Of Algebraic Statistics And Information Geometry Explores The Emerging Connections Between The Two Disciplines, Demonstrating How They Can Be Used In Design Of Experiments And How They Benefit Our Understanding Of Statistical Models And, In Particular, Exponential Models. This Book Presents A New Way Of Approaching Classical Statistical Problems And Raises Scientific Questions That Would Never Have Been Considered Without The Interaction Of These Two Disciplines. Beginning With A Brief Introduction To Each Area, Using Simple Illustrative Examples, The Book Then Proceeds With A Collection Of Reviews And Some New Results By Leading Researchers In Their Respective Fields. Parts I And Ii Are Mainly On Contingency Table Analysis And Design Of Experiments, Part Iii Dwells On Both Classical And Quantum Information Geometry. Finally, Part Iv Provides Examples Of The Interplay Between Algebraic Statistics And Information Geometry. Computer Code And Some Proofs Are Also Available Online, Where Key Examples Are Also Developed In Further Detail.--jacket. Algebraic And Geometric Methods In Statistics -- Contingency Tables -- Maximum Likelihood Estimation In Latent Class Models For Contingency Table Data / S.e. Fienberg [and Others] -- Algebraic Geometry Of 2x2 Contingency Tables / A.b. Slavković And S.e. Fienberg -- Model Selection For Contingency Tables With Algebraic Statistics / A. Krampe And S. Kuhnt -- Markov Chains, Quotient Ideals And Connectivity With Positive Margins / Y. Chen, I. Dinwoodie And R. Yoshida -- Algebraic Modelling Of Category Distinguishability / E. Carlini And F. Rapallo -- The Algebraic Complexity Of Maximum Likelihood Estimation For Bivariate Missing Data / S. Hosten And S. Sullivant -- The Generalised Shuttle Algorithm / A. Dobra And S.e. Fienberg -- Designed Experiments -- Generalised Design: Interpolation And Statistical Modelling Over Varieties / H. Maruri-aguilar And H.p. Wynn -- Design Of Experiments And Biochemical Network Inference / R. Laubennacher And B. Stigler -- Replicated Measurements And Algebraic Statistics / R. Notari And E. Riccomagno -- Indicator Function And Sudoku Designs / R. Fontana And M.p. Rogantin -- Markov Basis For Design Of Experiments With Three-level Factors / S. Aoki And A. Takemura -- Information Geometry -- Introduction To Non-parametric Estimation / R.f. Streater -- The Banach Manifold Of Quantum States / R.f. Streater -- On Quantum Information Manifolds / A. Jencova -- Axiomatic Geometrics For Text Documents / G. Lebanon -- Exponenetial Manifold By Reproducing Kernel Hilbert Spaces / K. Fukumizu -- Geometry Of Extended Exponential Models / D. Imparato And B. Trivellato -- Quantum Statistics And Measures Of Quantum Information / F. Hansen -- Information Geometry And Algebraic Statistics -- Algebraic Varieties Vs. Differentiable Manifolds In Statistical Models / G. Pistone. Edited By Paolo Gibilisco ... [et Al.]. Includes Bibliographical References.