Mitigating Bias in Machine Learning

Cover Title Page Copyright Page Dedication Contents Preface Acknowledgments 1 Beyond Algorithmic Bias Learning Objectives Chapter Overview 1.1 Introduction 1.2 Beyond Ethics in AI 1.2.1 Section Summary 1.3 What Is Algorithmic Justice? 1.3.1 The Main Causes of Injustice in Machine Learning 1.3.2 The Sources of Harm in a Machine Learning Life Cycle 1.3.3 Section Summary 1.4 Definitions of Algorithmic Fairness 1.4.1 Fairness Through Unawareness 1.4.2 Individual Fairness 1.4.3 Group Fairness 1.4.4 Counterfactual Fairness 1.4.5 Section Summary 1.5 Fairness Metrics 1.5.1 Statistical Parity Difference 1.5.2 Equal Opportunity Difference 1.5.3 Average Odds Difference 1.5.4 Disparate Impact 1.5.5 Theil Index 1.5.6 Section Summary 1.6 Methods for Fair Machine Learning 1.6.1 Preprocessing 1.6.2 In-processing 1.6.3 Postprocessing 1.6.4 Reweighing 1.6.5 Adversarial Debiasing 1.6.6 Reject Option-Based Classification 1.6.7 Section Summary 1.7 Tools to Help Detect and Mitigate Bias in Machine Learning Models 1.7.1 AI Fairness 360 1.7.2 Aequitas 1.7.3 Section Summary 1.8 Best Practices to Build a Fairer Application 1.8.1 Section Summary 1.9 Chapter Summary Chapter Glossary References End of Chapter Questions 2 Going Beyond the Technical: Exploring Ethical and Societal Implications of Machine Learning Learning Objectives Chapter Overview 2.1 Introduction 2.2 Programming Approaches 2.3 Societal and Cultural Implications of Algorithms 2.3.1 Racial Socialization 2.3.2 Racism in Its Many Forms 2.3.3 Why Does This Matter to Students? 2.4 Ethical Implications of Algorithms 2.5 Approaches to Mitigate Algorithmic Bias 2.6 Speak Up: Communicating Ideas with Digital Storytelling 2.6.1 Digital Storytelling 2.6.2 Other Considerations 2.7 Chapter Summary Chapter Glossary References End of Chapter Activities Reflective Exercises Algorithmic Bias and Search Algorithms Digital Storytelling Extensions Get Involved and Take Action! 3 Social Media and Health Information Dissemination Learning Objectives Chapter Overview 3.1 Introduction 3.1.1 Mitigating Bias in Health Information on Social Media 3.2 MyHealthImpactNetwork: For Students by Students 3.2.1 Why Consider Social Media? 3.2.2 Studying MyHealthImpact Twitter Feed 3.2.3 Data Preprocessing 3.2.4 Check Missing Values 3.2.5 Check Multicollinearity 3.3 Results of Data Inferential Analysis 3.3.1 Time Series Analysis 3.4 Chapter Summary Chapter Glossary References End of Chapter Questions 4 Comparative Case Study of Fairness Toolkits Learning Objectives Chapter Overview 4.1 Introduction 4.2 Bias 4.2.1 Label Bias 4.2.2 Sampling Bias 4.2.3 Representation Bias 4.3 Fairness 4.3.1 Statistical Measures 4.3.2 Similarity Measures 4.4 Applying Responsible AI 4.4.1 Checklists 4.4.2 Software Toolkits 4.4.3 Fairness Software Toolkits 4.4.4 Evaluating Fairness Toolkits 4.5 Results 4.6 What Are the Limitations of These Toolkits? 4.6.1 Diverse Range of Biases 4.6.2 Diverse Measures of Fairness 4.6.3 Bias Detection 4.6.4 Bias Mitigation 4.6.5 Intersectional Analysis 4.6.6 Applicable to Data Without Sensitive Information 4.7 Chapter Summary Acknowledgments Chapter Glossary References End of Chapter Questions 5 Bias Mitigation in Hate Speech Detection Learning Objectives Chapter Overview 5.1 Introduction 5.2 Background 5.2.1 Case Study of Hate Speech Detection 5.2.2 Section Summary 5.3 Bias in Hate Speech Detection Systems 5.4 Bias Mitigation in Hate Speech Detection Using Transfer Learning 5.4.1 Case Study of Transfer Learning 5.4.2 Section Summary 5.5 Bias Mitigation in Hate Speech Detection Using Transfer Learning 5.5.1 Case Study on Multitask Learning 5.5.2 Section Summary 5.6 Adversarial Methods for Bias Reduction in Hate Speech Detection 5.6.1 Case Study on Adversarial Training 5.6.2 Section Summary 5.7 Benefits and Pitfalls 5.7.1 Transfer Learning 5.7.2 Multitask Learning 5.7.3 Adversarial Methods 5.7.4 Section Summary 5.8 Other Methods 5.8.1 Section Summary 5.9 Hands-on Exercise 5.9.1 Model 5.9.2 Dataset 5.9.3 Data Preprocessing 5.9.4 Hate Speech Classification 5.9.5 Evaluation 5.9.6 Bias Visualization 5.10 Chapter Summary Chapter Glossary References Further Reading End of Chapter Problems and Questions Problems 6 Unveiling Unintended Systematic Biases in Natural Language Processing Learning Objectives Chapter Overview 6.1 Introduction 6.1.1 Pause Giant AI Experiment 6.1.2 Why Do We Trust AI? 6.1.3 Why Are There So Many Challenges? 6.2 Unfairness and Bias in NLP Applications 6.2.1 Recycling the Same Biases 6.2.2 AI Incident Repositories 6.3 Bias Taxonomy 6.3.1 Denied Opportunities and Preconceived Views 6.3.2 Biases Are Everywhere 6.4 Mitigating NLP Bias and Unfairness 6.4.1 Find and Neutralize 6.4.2 Measure and Evaluate 6.4.3 Examine Biases 6.5 Chapter Summary Chapter Glossary References End of Chapter Questions 7 Combating Bias in Large Language Models Learning Objectives Chapter Overview Prerequisites 7.1 Introduction 7.1.1 Bad Data In, Bad Data Out 7.2 Vectorization of Stochastic Parrots 7.2.1 Linear Analogies 7.2.2 Section Summary 7.3 Natural Language Processing: Linear Decision Making for Nonlinear Language 7.3.1 Attention Layer Mathematics 7.3.2 Section Summary: Outcome of the Attention Weights 7.4 Stage One: Data Collection 7.4.1 Dataset Nutrition Labels 7.4.2 Data Cards 7.4.3 Data Documentation 7.5 Stage Two: Fight Bad Math with Better Math 7.5.1 Counterfactuals 7.5.2 Parity 7.5.3 Stratified Sampling 7.6 Stage Three: Model Constraints/Operations 7.6.1 Flagging 7.6.2 Pruning 7.6.3 Nudging Case Study: The Limits of Better Training Data with No Constraint 7.7 Chapter Summary Chapter Glossary References End of Chapter Questions 8 Recognizing Bias in Medical Machine Learning and AI Models Learning Objectives Chapter Overview 8.1 Introduction 8.2 Defining Machine Learning 8.2.1 Supervised Machine Learning 8.2.2 Unsupervised Machine Learning 8.3 Building a Simple Machine Learning Model: Use Case 1 8.3.1 Preparing Our Development Environment 8.3.2 Downloading Anaconda by Going to www.anaconda.com/download 8.3.3 Updating Anaconda Packages from Terminals 8.3.4 Installing the Prospector Static Code Analysis Tool 8.3.5 Installing jupyter-matplotlib Visualization Library: ipympl 8.3.6 Standard Steps to Build a Machine Learning Model 8.3.7 Classifying Digits with the k-Nearest Neighbor Algorithm 8.4 Health Care Bias and Inequities: Use Case 2 8.4.1 Background 8.4.2 Are We There Yet? 8.4.3 The Use Case 8.4.4 Implementation Results and Brief Explanation 8.5 Chapter Summary Chapter Glossary References End of Chapter Questions 9 Toward Rectification of Machine Learning Bias in Health Care Diagnostics: A Case Study of Detecting Skin Cancer Across Diverse Ethnic Groups Learning Objectives Chapter Overview 9.1 Introduction 9.1.1 How Does ML Bias Occur, and How Do We Mitigate It? 9.2 Case Study: Mitigating Bias in ML for Melanoma 9.2.1 Retraining Melanoma Detection Algorithms for Diverse Skin Tones 9.2.2 The Diverse Dermatology Images Dataset 9.3 Defining Types of Biases and Mitigation Techniques in ML Life Cycles 9.4 Machine Learning Fairness 9.5 Chapter Summary Chapter Glossary References End of Chapter Questions 10 Applying the Wells-DuBois Protocol for Achieving Systemic Equity in Socioecological Systems Learning Objectives Chapter Overview 10.1 Introduction 10.1.1 Understanding AI and ML Use in Socioecological Systems 10.1.2 Examples of Socioecological Inequity and Bias 10.1.3 Clustering in ML for Model Outcome Assessment 10.1.4 Basic Concepts and Definitions 10.2 Equity Framework and Tool Application 10.2.1 The Systemic Equity Framework 10.2.2 The Wells-DuBois Protocol 10.2.3 Similarities and Differences with Other Equity Tools 10.2.4 Section Summary 10.3 Clustering Overview and Application 10.3.1 A Socioecological Example on Food Spending 10.3.2 Visualization and Initial Analysis 10.3.3 Section Summary 10.4 Applying the Wells-DuBois Protocol 10.4.1 Section Overview 10.5 Discussion and Future Directions 10.5.1 Other Clustering Activities to Help Achieve Systemic Equity 10.5.2 Further Discussion on Socioecological Systems 10.5.3 Other Benefits of Employing the Wells-DuBois Protocol and Systemic Equity 10.5.4 Section Summary 10.6 Chapter Summary Chapter Glossary References End of Chapter Problems 11 Community Engagement for Machine Learning Learning Objectives Chapter Overview 11.1 Introduction: Principles and Components of Community Engagement 11.1.1 Prerequisite Knowledge and Context: Case Study of Flint, Michigan 11.1.2 Brief Introduction to Environmental Justice and Environmental Data Justice 11.1.3 Data Justice 11.1.4 What Is a Community? 11.1.5 Community-Based Participatory Research 11.1.6 Citizen Science and Community Science 11.1.7 Section Summary 11.2 Project Initiation: Steps of Conducting Community-Driven Environmental Data Science 11.2.1 Phase 1: Building Partnerships 11.2.2 Phase 2: Preparation for Modeling: Done in Conversation with Stakeholders 11.2.3 Phase 3: Model Development 11.2.4 Phase 4: After Modeling 11.3 How to Engage Communities in the Process: Case Study of the Mobile Lead Testing Unit Project in Newark, New Jersey 11.3.1 Brief History Newark Lead Crises 11.3.2 Project Initiation: The Newark Water Coalition and the Initiation of the Mobile Lead Testing Unit 11.3.3 Identifying Stakeholders 11.3.4 Data Collection 11.3.5 Building Community Capacity 11.4 Chapter Summary Chapter Glossary References End of Chapter Problems