Algebraic Approach to Data Processing : Techniques and Applications

The series "Studies in Big Data" (SBD) publishes new developments and advances in the various areas of Big Data-quickly and with a high quality. The intent is to cover the theory, research, development, and applications of Big Data, as embedded in the fields of engineering, computer science, physics, economics and life sciences. The books of the series refer to the analysis and understanding of large, complex, and/or distributed data sets generated from recent digital sources coming from sensors or other physical instruments as well as simulations, crowd sourcing, social networks or other internet transactions, such as emails or video click streams and other. The series contains monographs, lecture notes and edited volumes in Big Data spanning the areas of computational intelligence including neural networks, evolutionary computation, soft computing, fuzzy systems, as well as artificial intelligence, data mining, modern statistics and Operations research, as well as self-organizing systems. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution, which enable both wide and rapid dissemination of research output. Preface 6 Contents 7 1 Introduction 14 1.1 What Is Data Processing and Why Do We Need It? 14 1.2 Why Algebraic Approach? 14 1.3 What We Do in This Book: An Overview 15 1.4 Thanks 16 References 16 2 What Are the Most Natural and the Most Frequent Transformations 17 2.1 Main Idea: Numerical Values Change When We Change a Measuring Unit and/or Starting Point 17 2.2 Scaling Transformations 17 2.3 Shifts 17 2.4 Linear Transformations 18 2.5 Geometric Transformations 18 2.6 Beyond Linear Transformations 18 2.7 Permutations 20 References 20 3 Which Functions and Which Families of Functions Are Invariant 21 3.1 Why Do We Need Invariant Functions 21 3.2 What Does It Mean for a Function to Be Invariant 21 3.3 Example: Scale-Invariant Functions of One Variable 22 3.4 What If We Have Both Shift- and Scale-Invariance? 22 3.5 Which Families of Functions Are Invariant: Case of Shift-Invariance 23 3.6 Which Families of Functions Are Invariant: Case of Scale-Invariance 24 3.7 What If We Have Both Shift- and Scale-Invariance 25 3.8 Which Linear Transformations Are Shift-Invariant 25 References 26 4 What Is the General Relation Between Invariance and Optimality 27 4.1 What Is an Optimality Criterion 27 4.2 We Need a Final Optimality Criterion 28 4.3 It Is Often Reasonable to Require That the Optimality Criterion Be Invariant 29 4.4 Main Result of This Chapter 29 5 General Application: Dynamical Systems 30 5.1 Problem: Why a Linear-Based Classification Often Works in Nonlinear Cases 30 5.2 Our Explanation 32 References 32 6 First Application to Physics: Why Liquids? 33 6.1 Two Applications to Physics: Summary 33 6.2 Problem: Why Liquids? 33 6.3 Towards a Formulation of the Problem in Precise Terms 34 6.4 Main Result of This Chapter 37 References 39 7 Second Application to Physics: Warping of Our Galaxy 40 7.1 Formulation of the Problem 40 7.2 Analysis of the Problem and the Resulting Explanation 41 References 43 8 Application to Electrical Engineering: Class-D Audio Amplifiers 44 8.1 Applications to Engineering: Summary 44 8.2 Problem: Why Class-D Audio Amplifiers Work Well? 45 8.3 Why Pulses 45 8.4 Why the Pulse's Duration Should Linearly Depend ... 46 References 49 9 Application to Mechanical Engineering: Wood Structures 50 9.1 Problem: Need for a Theoretical Explanation of an Empirical Fact 50 9.2 Our Explanation: Main Idea 51 9.3 Our Explanation: Details 51 9.4 Proof 54 References 56 10 Medical Application: Prevention 58 10.1 Problem: How to Best Maintain Social Distance 58 10.2 Towards Formulating This Problem in Precise Terms 58 10.3 Solution 59 Reference 59 11 Medical Application: Testing 60 11.1 Problem: Optimal Group Testing 60 11.2 What Was Proposed 60 11.3 Resulting Problem 60 11.4 Let Us Formulate This Problem in Precise Terms 61 11.5 Solution 61 References 62 12 Medical Application: Diagnostics, Part 1 63 12.1 Problem: Diagnosing Lung Disfunctions in Children 63 12.2 First Pre-processing Stage: Scale-Invariant Smoothing 67 12.3 Which Order Polynomials Should We Use? 68 12.4 Second Pre-processing Stage: Using the Approximating Polynomials to Distinguish Between Different Diseases 70 12.5 Third Pre-processing Stage: Scale-Invariant Similarity/Dissimilarity Measures 71 12.6 How to Select α: Need to Have Efficient and Robust Estimates 73 12.7 Scale-Invariance Helps to Take Into Account That Signal Informativeness Decreases with Time 74 12.8 Pre-processing Summarized: What Information Serves as An Input to a Neural Network 75 12.9 The Results of Training Neural Networks on These Pre-processed Data 77 References 79 13 Medical Application: Diagnostics, Part 2 81 13.1 Problem: Why Hierarchical Multiclass Classification Works Better Than Direct Classification 81 13.2 Our Explanation 83 References 84 14 Medical Application: Diagnostics, Part 3 86 14.1 Problem: Which Fourier Components Are Most Informative 86 14.2 Main Idea 87 14.3 First Case Study: Human Color Vision 88 14.4 Second Case Study: Classifying Lung Dysfunctions 89 References 90 15 Medical Application: Treatment 91 15.1 Problem: Geometric Aspects of Wound Healing 91 15.2 What Are Natural Symmetries Here and What Are the Resulting Cell Shapes: Case of Undamaged Skin 92 15.3 What If the Skin Is Damaged: Resulting Symmetries and Cell Shapes 93 15.4 Geometric Symmetries Also Explain Observed Cell Motions 94 References 95 16 Applications to Economics: How Do People Make Decisions, Part 1 97 References 99 17 Application to Economics: How Do People Make Decisions, Part 2 100 17.1 Problem: Need to Consider Multiple Scenarios 100 17.2 Our Explanation 101 References 101 18 Application to Economics: How Do People Make Decisions, Part 3 103 18.1 Problem: Using Experts 103 18.2 Towards an Explanation 103 References 104 19 Application to Economics: How Do People Make Decisions, Part 4 105 19.1 Why Should We Play Down Emotions 105 19.2 Towards Explanation 105 References 106 20 Application to Economics: Stimuli, Part 1 107 20.1 Problem: Why Rewards Work Better Than Punishment 107 20.2 Analysis of the Problem 108 20.3 Our Explanation 110 References 112 21 Application to Economics: Stimuli, Part 2 113 21.1 Problem: Why Top Experts Are Paid So Much 113 21.2 Our Explanation 114 References 115 22 Application to Economics: Investment 116 22.1 1/n Investment: Formulation of the Problem 116 22.2 Our Explanation 117 22.3 Discussion 118 References 119 23 Application to Social Sciences: When Revolutions Happen 120 23.1 Formulation of the Problem 120 23.2 Analysis of the Problem 121 References 125 24 Application to Education: General 126 24.1 Problem: Is Immediate Repetition Good for Learning? 126 24.2 Analysis of the Problem and the Resulting Explanation 127 References 131 25 Application to Education: Specific 132 25.1 Problem: Why Derivative 132 25.2 Invariance Naturally Leads to the Derivative 132 Reference 136 26 Application to Mathematics: Why Necessary Conditions Are Often Sufficient 137 26.1 Formulation of the Problem 137 26.2 Analysis of the Problem 138 26.3 How Can We Formalize What Is Not Abnormal 138 26.4 Resulting Explanation of the TONCAS Phenomenon 139 References 140 27 Data Processing: Neural Techniques, Part 1 141 27.1 Machine Learning Is Needed to Analyze Complex Systems 141 27.2 Neural Networks and Deep Learning: A Brief Reminder 142 27.3 Why Traditional Neural Networks 143 27.4 Why Sigmoid Activation Function: Idea 144 27.5 Why Sigmoid—Derivation 145 27.6 Limit Cases 148 27.7 We Need Multi-layer Neural Networks 149 27.8 Which Activation Function Should We Use 149 27.9 This Leads Exactly to Squashing Functions 150 27.10 Why Rectified Linear Functions 151 References 151 28 Data Processing: Neural Techniques, Part 2 153 28.1 Problem: Spiking Neural Networks 153 28.2 Analysis of the Problem and the First Result 154 28.3 Main Result: Spikes Are, in Some Reasonable Sense, Optimal 159 References 160 29 Data Processing: Fuzzy Techniques, Part 1 161 29.1 Why Fuzzy Techniques 161 29.2 Fuzzy Techniques: Main Ideas 161 29.3 Fuzzy Techniques: Logic 162 References 163 30 Data Processing: Neural and Fuzzy Techniques 164 30.1 Problem: Computations Should Be Fast and Understandable 164 30.2 Definitions and the Main Results 165 30.3 Auxiliary Result: What Can We Do with Two-Layer Networks 169 References 171 31 Data Processing: Fuzzy Techniques, Part 2 172 31.1 Problem: Which Fuzzy Techniques to Use? 172 31.2 Analysis of the Problem 174 31.3 Which Symmetric Membership Functions Should We ... 176 31.4 Which Hedge Operations and Negation Operations Should We Select 177 31.5 Proofs 177 References 183 32 Data Processing: Fuzzy Techniques, Part 3 185 32.1 Problem: Which Fuzzy Degrees to Use? 185 32.2 Definitions and the Main Result 188 32.3 How General Is This Result? 189 32.4 What If We Allow Unlimited Number of ``And''-Operations and Negations: Case Study 190 References 191 33 Data Processing: Fuzzy Techniques, Part 4 193 33.1 Problem: How to Explain Commonsense Reasoning 193 33.2 Our Explanation 195 33.3 Auxiliary Result: Why the Usual Quantifiers? 200 References 201 34 Data Processing: Probabilistic Techniques, Part 1 202 34.1 Problem: How to Represent Interval Uncertainty 202 34.2 Analysis of the Problem 205 34.3 Our Results 207 References 209 35 Data Processing: Probabilistic Techniques, Part 2 210 35.1 Problem: How to Represent General Uncertainty 210 35.2 Definitions and the Main Result 212 35.3 Consequence 215 References 216 36 Data Processing: Probabilistic Techniques, Part 3 217 36.1 Problem: Experts Don't Perform Well in Unusual Situations 217 36.2 Our Explanation 218 References 220 37 Data Processing: Beyond Traditional Techniques 222 37.1 DNA Computing: Introduction 222 37.2 Computing Without Computing—Quantum Version: A Brief Reminder 223 37.3 Computing Without Computing—Version Involving Acausal Processes: A Reminder 224 37.4 Computing Without Computing—DNA Version 226 37.5 DNA Computing Without Computing Is Somewhat Less ... 231 37.6 First Related Result: Security Is More Difficult to Achieve than Privacy 233 37.7 Second Related Result: Data Storage Is More Difficult Than Data Transmission 235 References 237 Appendix References 240 240 Index 241 The book explores a new general approach to selecting―and designing―data processing techniques. Symmetry and invariance ideas behind this algebraic approach have been successful in physics, where many new theories are formulated in symmetry terms. The book explains this approach and expands it to new application areas ranging from engineering, medicine, education to social sciences. In many cases, this approach leads to optimal techniques and optimal solutions. That the same data processing techniques help us better analyze wooden structures, lung dysfunctions, and deep learning algorithms is a good indication that these techniques can be used in many other applications as well. The book is recommended to researchers and practitioners who need to select a data processing technique―or who want to design a new technique when the existing techniques do not work. It is also recommended to students who want to learn the state-of-the-art data processing. The book explores a new general approach to selectingand designing data processing techniques. Symmetry and invariance ideas behind this algebraic approach have been successful in physics, where many new theories are formulated in symmetry terms. The book explains this approach and expands it to new application areas ranging from engineering, medicine, education to social sciences. In many cases, this approach leads to optimal techniques and optimal solutions. That the same data processing techniques help us better analyze wooden structures, lung dysfunctions, and deep learning algorithms is a good indication that these techniques can be used in many other applications as well. The book is recommended to researchers and practitioners who need to select a data processing techniqueor who want to design a new technique when the existing techniques do not work. It is also recommended to students who want to learn the state-of-the-art data processing