Software Design X-Rays : Fix Technical Debt with Behavioral Code Analysis

Are you working on a codebase where cost overruns, death marches, and heroic fights with legacy code monsters are the norm? Battle these adversaries with novel ways to identify and prioritize technical debt, based on behavioral data from how developers work with code. And that's just for starters. Because good code involves social design, as well as technical design, you can find surprising dependencies between people and code to resolve coordination bottlenecks among teams. Best of all, the techniques build on behavioral data that you already have: your version-control system. Join the fight for better code! Use statistics and data science to uncover both problematic code and the behavioral patterns of the developers who build your software. This combination gives you insights you can't get from the code alone. Use these insights to prioritize refactoring needs, measure their effect, find implicit dependencies between different modules, and automatically create knowledge maps of your system based on actual code contributions. In a radical, much-needed change from common practice, guide organizational decisions with objective data by measuring how well your development teams align with the software architecture. Discover a comprehensive set of practical analysis techniques based on version-control data, where each point is illustrated with a case study from a real-world codebase. Because the techniques are language neutral, you can apply them to your own code no matter what programming language you use. Guide organizational decisions with objective data by measuring how well your development teams align with the software architecture. Apply research findings from social psychology to software development, ensuring you get the tools you need to coach your organization towards better code. If you're an experienced programmer, software architect, or technical manager, you'll get a new perspective that will change how you work with code. What You Need: You don't have to install anything to follow along in the book. TThe case studies in the book use well-known open source projects hosted on GitHub. You'll use CodeScene, a free software analysis tool for open source projects, for the case studies. We also discuss alternative tooling options where they exist. Cover 1 Table of Contents 7 Acknowledgments 10 The World of Behavioral Code Analysis 11 Why You Should Read This Book 11 Who Is This Book For? 12 How Should You Read This Book? 13 How Do I Get Behavioral Data for My Code? 14 Online Resources 16 Part I—Prioritize and React to Technical Debt 17 1. Why Technical Debt Isn’t Technical 18 Questioning Technical Debt 18 The Perils of Quantifying Technical Debt 21 Mine Your Organization’s Collective Intelligence 25 Prioritize Improvements Guided by Data 28 2. Identify Code with High Interest Rates 30 Measure Interest Rates 30 Prioritize Technical Debt with Hotspots 34 Evaluate Hotspots with Complexity Trends 39 Use X-Rays to Get Deep Insights into Code 42 Escape the Technical-Debt Trap 46 Exercises 48 3. Coupling in Time: A Heuristic for the Concept of Surprise 50 Uncover Expensive Change Patterns 50 Detect Cochanging Files 52 The Dirty Secret of Copy-Paste 59 The Power of Language-Neutral Analyses 61 Exercises 63 4. Pay Off Your Technical Debt 66 Follow the Principle of Proximity 66 Refactor Congested Code with the Splinter Pattern 72 Build Temporary Tests as a Safety Net 79 Turn Hotspot Methods into Brain-Friendly Chunks 82 The Curse of a Successful System 85 5. The Principles of Code Age 87 Stabilize Code by Age 87 The Three Generations of Code 90 Refactor Your Package Structure 96 Scale from Files to Systems 101 Exercises 101 Part II—Work with Large Codebases and Organizations 104 6. Spot Your System’s Tipping Point 105 Is Software Too Hard? 105 Divide and Conquer with Architectural Hotspots 108 Analyze Subsystems 112 Fight the Normalization of Deviance 119 Toward Team-Oriented Measures 124 Exercises 125 7. Beyond Conway’s Law 128 Software Architecture Is About Making Choices 128 Measure Coordination Needs 130 Code Ownership and Diffusion of Responsibility 136 Analyze Operational Team Boundaries 140 Social Groups: The Flip Side to Conway’s Law 145 Combine Social and Technical Information 149 8. Toward Modular Monoliths through the Social View of Code 151 Dodge the Silver Bullet 151 Layered Architectures and the Cost of Consistency 154 Monolithic Alternatives: Use Case and Feature-Centric 157 Discover Bounded Contexts Through Change Patterns 161 The Perils of Feature Teams 165 Clean and Dirty Architectures 171 Exercises 172 9. Systems of Systems: Analyzing Multiple Repositories and Microservices 174 Analyze Code in Multiple Repositories 174 Compare Hotspots Across Repositories 179 Track Change Patterns in Distributed Systems 182 Measure Technical Sprawl 189 Distribution Won’t Cure the Dependency Blues 194 Exercises 196 10. An Extra Team Member: Predictive and Proactive Analyses 198 Detect Deviating Evolutionary Patterns 198 Catch the Absence of Change 206 Guide On- and Offboarding with Social Data 209 Know the Biases and Workarounds for Behavioral Code Analysis 214 Your Code Is Still a Crime Scene 216 Exercises 218 A1. The Hazards of Productivity and Performance Metrics 220 Adaptive Behavior and the Destruction of a Data Source 220 The Situation Is Invisible in Code 221 A2. Code Maat: An Open Source Analysis Engine 224 Run Code Maat 224 Data Mining with Code Maat 224 Visualizations 226 A3. Data Mining with Git, cloc, and CodeScene 229 Behavioral Code Analysis with Git 229 A Brief Introduction to cloc 231 Export Analysis Data from CodeScene 232 A4. Hints and Solutions to the Exercises 234 Solutions: Identify Code with High Interest Rates 234 Solutions: Coupling in Time 235 Solutions: The Principles of Code Age 236 Solutions: Spot Your System's Tipping Point 236 Solutions: Modular Monoliths 237 Solutions: Systems of Systems 239 Solutions: An Extra Team Member 239 Bibliography 242 Index 246 – SYMBOLS – 246 – A – 246 – B – 247 – C – 247 – D – 249 – E – 250 – F – 251 – G – 251 – H – 252 – I – 252 – J – 252 – K – 252 – L – 253 – M – 253 – N – 254 – O – 254 – P – 254 – Q – 255 – R – 255 – S – 256 – T – 257 – U – 258 – V – 258 – W – 258 – X – 258 – Y – 259