Cognitive Biases in Visualizations

This book brings together the latest research in this new and exciting area of visualization, looking at classifying and modelling cognitive biases, together with user studies which reveal their undesirable impact on human judgement, and demonstrating how visual analytic techniques can provide effective support for mitigating key biases. A comprehensive coverage of this very relevant topic is provided though this collection of extended papers from the successful DECISIVe workshop at IEEE VIS, together with an introduction to cognitive biases and an invited chapter from a leading expert in intelligence analysis. __Cognitive Biases in Visualizations__ will be of interest to a wide audience from those studying cognitive biases to visualization designers and practitioners. It offers a choice of research frameworks, help with the design of user studies, and proposals for the effective measurement of biases. The impact of human visualization literacy, competence and human cognition on cognitive biases are also examined, as well as the notion of system-induced biases. The well referenced chapters provide an excellent starting point for gaining an awareness of the detrimental effect that some cognitive biases can have on users’ decision-making. Human behavior is complex and we are only just starting to unravel the processes involved and investigate ways in which the computer can assist, however the final section supports the prospect that visual analytics, in particular, can counter some of the more common cognitive errors, which have been proven to be so costly. Preface 5 DECISIVe 2017 Workshop 6 Organizers 6 Program Committee Members 6 Contents 8 Contributors 10 1 So, What Are Cognitive Biases? 12 1.1 Introduction 12 1.1.1 Examples 13 1.2 A Brief History of Cognitive Biases 14 1.3 Impact of Biases 15 1.4 Cognitive Biases in Visualization 15 1.4.1 Interpretation of Visualizations 16 1.4.2 Visualization Tools 16 1.5 Debiasing 16 1.6 Conclusion 17 References 18 Bias Definitions, Perspectives and Modeling 22 2 Studying Biases in Visualization Research: Framework and Methods 23 2.1 Introduction 23 2.2 A Framework to Study Biases 26 2.2.1 Perceptual Biases 28 2.2.2 Action Biases 29 2.2.3 Social Biases 29 2.3 Methodological Considerations When Studying Biases 30 2.3.1 Perceptual Biases 30 2.3.2 Action Biases 32 2.3.3 Social Biases 33 2.3.4 Application of the Framework to Derive a Model 34 2.3.5 Threats to Validity 35 2.4 Conclusion 36 References 36 3 Four Perspectives on Human Bias in Visual Analytics 38 3.1 Introduction 38 3.2 Bias as a Cognitive Processing Error 39 3.3 Bias as a Filter for Information 40 3.4 Bias as a Preconception 42 3.5 Bias as a Model Mechanism 43 3.6 Discussion 45 3.6.1 Does Bias Endanger Mixed-Initiative Visual Analytics? 46 3.6.2 How to Keep the Machine ``Above the Bias''? 46 3.6.3 Could the Mixed-Initiative System Impart Bias to the User? 47 3.6.4 Is Bias Good or Bad? 48 3.7 Conclusion 49 References 49 4 Bias by Default? 52 4.1 Introduction 52 4.2 Relationship to Analytic Provenance 54 4.3 Gapminder 55 4.4 Markov Models 55 4.5 Interface Models 57 4.6 Application: Gapminder Analysis 60 4.7 Discussion 65 4.8 Conclusion 66 References 67 Cognitive Biases in Action 68 5 Methods for Discovering Cognitive Biases in a Visual Analytics Environment 69 5.1 Introduction 69 5.2 Theory-Driven Approaches 72 5.2.1 Design Recommendations 72 5.2.2 Systematic Tool Analysis 74 5.2.3 Process-Oriented Operationalization 75 5.3 Empirical Approaches – Behavioral Observation and Outcome-Oriented Operationalization 75 5.3.1 Behavioral Observation 75 5.3.2 Outcome-Oriented Operationalization 76 5.4 Automatic Cognitive Bias Detection Approach 78 5.5 Conclusion and Outlook 79 References 80 6 Experts' Familiarity Versus Optimality of Visualization Design: How Familiarity Affects Perceived and Objective Task Performance 82 6.1 Introduction 82 6.2 Manifestations of the Familiarity Heuristic 83 6.3 Effects on Visualization Based Judgments 87 6.4 Critical Reflection 90 6.5 Conclusion 92 References 92 7 Data Visualization Literacy and Visualization Biases: Cases for Merging Parallel Threads 94 7.1 Introduction 94 7.2 Background 95 7.2.1 Measuring Data Visualization Literacy and Quantifying Its Impact on Performance 96 7.2.2 Novices, Experts and Visualization Use 96 7.2.3 Biases and Data Visualization 98 7.3 Individual Differences and Bias: Guiding Results and Organizational Frameworks 98 7.4 Linking Data Visualization Literacy to Existing Studies of Visualization and Biases 100 7.4.1 Reversal: Augmenting Data Visualization Literacy Research with Biases 102 7.5 Conclusion 102 References 102 8 The Biases of Thinking Fast and Thinking Slow 104 8.1 Introduction 104 8.2 Heuristics and Biases 105 8.3 Case studies 106 8.3.1 Causes Trump Statistics 107 8.3.2 Tom W's Specialty 108 8.3.3 The 3-D Heuristic 110 8.4 Implications for Visualization and Visual Analytics 111 References 113 Mitigation Strategies 115 9 Experimentally Evaluating Bias-Reducing Visual Analytics Techniques in Intelligence Analysis 116 9.1 Introduction 116 9.2 Basics of Intelligence Analysis 117 9.2.1 Intelligence as a Cognitive Process 117 9.2.2 Assessing Analytic Quality 118 9.2.3 Judging “Correctness” 119 9.3 Impact of Heuristics and Biases on Intelligence Analysis 120 9.3.1 Confirmation Bias 120 9.3.2 Illusory Correlation 121 9.3.3 Absence of Evidence 121 9.3.4 Irrelevant Evidence 122 9.3.5 Overconfidence Bias 122 9.4 Mitigating Bias Effects 123 9.4.1 Training Interventions 124 9.4.2 Procedural Interventions 125 9.5 Experimental Methods 128 9.5.1 Considerations 129 9.5.2 Prior Studies 130 9.5.3 Experimental Design Framework 132 9.5.4 Cautions 136 9.6 Conclusion 137 References 137 10 Promoting Representational Fluency for Cognitive Bias Mitigation in Information Visualization 141 10.1 Introduction 141 10.2 Representational Fluency 143 10.3 Implications for Visualization Research and Practice 144 10.3.1 Developing Representational Fluency 145 10.3.2 Effect on Cognitive Processing 146 10.3.3 Preliminary Research Agenda 147 10.4 Summary 149 References 149 11 Designing Breadth-Oriented Data Exploration for Mitigating Cognitive Biases 152 11.1 Introduction 152 11.2 The Information Space Model of Breadth-Oriented Exploration 153 11.3 Three Considerations for Designing Breadth-Oriented Data Exploration 154 11.3.1 Unit of Exploration 155 11.3.2 User-Driven Versus System-Driven Exploration 156 11.3.3 Related Versus Systematic Exploration 157 11.4 Application of the Three Design Considerations 157 11.4.1 Task Analysis 157 11.4.2 Usage Scenario 158 11.4.3 Designing Based on the Three Considerations 160 11.5 Discussion 160 References 161 12 A Visualization Approach to Addressing Reviewer Bias in Holistic College Admissions 163 12.1 Introduction 163 12.2 Related Work 164 12.2.1 Reasoning Heuristics and Cognitive Biases 164 12.2.2 Using Visualizations to Mitigate Cognitive Biases 165 12.3 Characterization of the Holistic Review Process 165 12.4 System 1 and System 2 166 12.5 Accuracy of Expert Intuition in Holistic Reviews 167 12.6 Possible Reviewer Biases 168 12.6.1 Coherence, Causal Associations, and Narrative Fallacy 169 12.6.2 Anchoring as Adjustment 169 12.6.3 The Halo Effect 170 12.6.4 Confirmation Bias 170 12.6.5 Availability 170 12.6.6 Representativeness 171 12.6.7 The Avoidance of Cognitive Dissonance 172 12.6.8 Time-Induced and Stress-Induced Biases 172 12.7 Proposed Visualization Strategies to Mitigate Biases 172 12.7.1 Easing Cognitive Load 173 12.7.2 Supporting Sensemaking 173 12.7.3 Decorrelating Error 174 12.7.4 Mobilizing System 2 174 12.7.5 Combining Formulas with Intuition 175 12.8 Conclusion 175 References 176 13 Cognitive Biases in Visual Analytics—A Critical Reflection 178 13.1 Introduction 178 13.2 Puzzle Problem Approach Versus Everyday Thinking and Reasoning 179 13.3 Bias Mitigation Strategies 181 13.4 Conclusion 183 References 184 Front Matter ....Pages i-xii So, What Are Cognitive Biases? (Geoffrey Ellis)....Pages 1-10 Front Matter ....Pages 11-11 Studying Biases in Visualization Research: Framework and Methods (André Calero Valdez, Martina Ziefle, Michael Sedlmair)....Pages 13-27 Four Perspectives on Human Bias in Visual Analytics (Emily Wall, Leslie M. Blaha, Celeste Lyn Paul, Kristin Cook, Alex Endert)....Pages 29-42 Bias by Default? (Joseph A. Cottam, Leslie M. Blaha)....Pages 43-58 Front Matter ....Pages 59-59 Methods for Discovering Cognitive Biases in a Visual Analytics Environment (Michael A. Bedek, Alexander Nussbaumer, Luca Huszar, Dietrich Albert)....Pages 61-73 Experts’ Familiarity Versus Optimality of Visualization Design: How Familiarity Affects Perceived and Objective Task Performance (Aritra Dasgupta)....Pages 75-86 Data Visualization Literacy and Visualization Biases: Cases for Merging Parallel Threads (Hamid Mansoor, Lane Harrison)....Pages 87-96 The Biases of Thinking Fast and Thinking Slow (Dirk Streeb, Min Chen, Daniel A. Keim)....Pages 97-107 Front Matter ....Pages 109-109 Experimentally Evaluating Bias-Reducing Visual Analytics Techniques in Intelligence Analysis (Donald R. Kretz)....Pages 111-135 Promoting Representational Fluency for Cognitive Bias Mitigation in Information Visualization (Paul Parsons)....Pages 137-147 Designing Breadth-Oriented Data Exploration for Mitigating Cognitive Biases (Po-Ming Law, Rahul C. Basole)....Pages 149-159 A Visualization Approach to Addressing Reviewer Bias in Holistic College Admissions (Poorna Talkad Sukumar, Ronald Metoyer)....Pages 161-175 Cognitive Biases in Visual Analytics—A Critical Reflection (Margit Pohl)....Pages 177-184 "This book brings together the latest research in this new and exciting area of visualization, looking at classifying and modelling cognitive biases, together with user studies which reveal their undesirable impact on human judgement, and demonstrating how visual analytic techniques can provide effective support for mitigating key biases. A comprehensive coverage of this very relevant topic is provided though this collection of extended papers from the successful DECISIVe workshop at IEEE VIS, together with an introduction to cognitive biases and an invited chapter from a leading expert in intelligence analysis. Cognitive Biases in Visualizations will be of interest to a wide audience from those studying cognitive biases to visualization designers and practitioners. It offers a choice of research frameworks, help with the design of user studies, and proposals for the effective measurement of biases. The impact of human visualization literacy, competence and human cognition on cognitive biases are also examined, as well as the notion of system-induced biases. The well referenced chapters provide an excellent starting point for gaining an awareness of the detrimental effect that some cognitive biases can have on users' decision-making. Human behavior is complex and we are only just starting to unravel the processes involved and investigate ways in which the computer can assist, however the final section supports the prospect that visual analytics, in particular, can counter some of the more common cognitive errors, which have been proven to be so costly."--Publisher's description