Big Data Application in Power Systems

Big Data Application in Power Systems brings together experts from academia, industry and regulatory agencies who share their understanding and discuss the big data analytics applications for power systems diagnostics, operation and control. Recent developments in monitoring systems and sensor networks dramatically increase the variety, volume and velocity of measurement data in electricity transmission and distribution level. The book focuses on rapidly modernizing monitoring systems, measurement data availability, big data handling and machine learning approaches to process high dimensional, heterogeneous and spatiotemporal data. The book chapters discuss challenges, opportunities, success stories and pathways for utilizing big data value in smart grids. Provides expert analysis of the latest developments by global authorities Contains detailed references for further reading and extended research Provides additional cross-disciplinary lessons learned from broad disciplines such as statistics, computer science and bioinformatics Focuses on rapidly modernizing monitoring systems, measurement data availability, big data handling and machine learning approaches to process high dimensional, heterogeneous and spatiotemporal data cover Big Data Application in Power Systems cover inside Front Matter elsevier Copyright contributors Contributors about the editors About the Editors preface Preface: Objective and Overview of the Book Section One: Harness the Big Data From Power Systems Section Two: Harness the Power of Big Data Section Three: Put the Power of Big Data Into Power Systems acknowlledge Acknowledgments c1 A Holistic Approach to Becoming a Data-Driven Utility Introduction Aligning Internal and External Stakeholders Taking a Holistic Approach ``Strong ́ ́ First, Then ``Smart ́ ́ Increasing Visibility With IEDs Network Response Requirements Integration Before Automation Functional Data Paths: Keep it Simple From Sensor to End User: The Process Consumers/Customers: Another Source of Data Extracting Value From Data, and Presenting It The Transformation Three Case Studies Frankfort, Kentucky, and Greenfield SCADA, SA Ketchikan, Alaska, Deals With Unsupported, Legacy RTUs North Carolina Agency Pursues New SCADA, Boosts Revenue Conclusion Looking Ahead References c2 Emerging Security and Data Privacy Challenges for Utilities: Case Studies and Solutions Introduction Case Studies: The State and Scope of the Threat Coordinated Cyberattack Causes Outage in the Ukraine Severe Financial Impacts at Saudi Aramco The Misunderstood Near Miss: Burlington Electric and Grizzly Steppe Impact on Practices in the Utility Industry The Digitized Network Increases Vulnerability Attack Scenarios The Role of Data Analytics The Role of Privacy Conclusion References c3 The Role of Big Data and Analytics in Utility Innovation Introduction of Big Data and Analytics as an Accelerator of Innovation Approaches to Data Driven Innovation Integration of Renewable Energy Grid Operations Cognitive Computing on Big Data Weather, the Biggest Data Topic for Power Systems References c4 Frameworks for Big Data Integration, Warehousing, and Analytics Introduction Frameworks for Big Data Platform Architecture Storage Security Big Data With HPC System Architecture (Fig. 3) Service Type Internet of Things HPC Platform for Smart Grid Big Data With Complex Event Processing Application of Big Data Techniques for Power Systems Conclusion Acknowledgment References c5 Moving Toward Agile Machine Learning for Data Analytics in Power Systems Introduction Classic Supervised and Unsupervised Learning Supervised Learning Overview Examples of Supervised Learning as a Regularization Empirical Risk Minimization Linear/Ridge Regression Logistic Regression Support Vector Machine Nonparametric Regression Decision Tree Bayesian Perspectives Linear/Ridge Regression Revisited Example: HMM With Autoregressive Emissions Unsupervised Learning Overview Principle Component Analysis k-Means Clustering Model Selection (Hyperparameter Selection) Theoretical Intuition Generalization Bound Bias-Variance Trade-Off Practical Model Selection Techniques Cross-Validation Being a Bayesian Regularization Path Algorithms Bayesian Optimization Feature Selection Overview Filter Method Wrapper Method Embedded Method An Practical Example of FS Using Information Theory Criterion Single Stream Features Extraction Interstream Features Other Promising Research Directions Leverage Unlabeled Data With Semisupervised Learning Knowledge Transfer With Multitask Learning Information Fusion With Multiview Learning Other Promising Directions References c6 Unsupervised Learning Methods for Power System Data Analysis Introduction Smart Meter Data Preparation Statistical Analysis Clustering Algorithm Clustering Approach and Visualization Features Extraction Typical Daily Patterns Visualization Tool Conclusions References c7 Deep Learning for Power System Data Analysis Introduction From Neural Network Towards Deep Learning Deep Learning Methods Supervised Energy Prediction Using Deep Learning Conditional Restricted Boltzmann Machine Inference in CRBM Learning for CRBM Using Contrastive Divergence Factored Conditional Restricted Boltzmann Machine Total Energy for FCRBM Inference in FCRBM Learning and Update Rules for FCRBMs Experiments and Results Unsupervised Energy Prediction Using Deep Learning Problem Formulation Reinforcement Learning Markov Decision Process Q-Learning SARSA States Estimation via DBNs Deep Belief Networks Numerical Results Commercial to Residential Transfer Residential to Residential Transfer Conclusions References c8 Compressive Sensing for Power System Data Analysis Introduction Mathematical Modeling of a Compressive Sensing-Sparse Recovery Problem Compressive Sensing Sparse Recovery Problem Applications of CS-SR Techniques in Smart Grids Sparse Recovery-Based DSSE in Smart Grid Sparse Recovery-Based Fault Location in Smart Distribution and Transmission Networks Compressive Sensing-Based PD Pattern Recognition Conclusions and Future of the CS-SR in Smart Grids c9 Time-Series Classification Methods: Review and Applications to Power Systems Data Introduction Contribution Notation The Classification Problem Classification Methods Taxonomy Computational Issues Data Sources Dimensionality Reduction DR Techniques Review Numerosity Reduction NR Techniques Review Classification Methods Feature-Based Methods Metrics-Based Approaches Occurrence Counting Approaches Dynamics-Based Approaches Distance-Based Methods Purely Distance-Based Methods Reduction Distance-Based Methods Dictionary Learning Parametric Distance-Based Methods Methods Comparison Applications Concluding Remarks References c10 Future Trends for Big Data Application in Power Systems Introduction Transmission System Dynamic Behavior Analysis Steady-State Analysis TSO-DSO Cooperation Distribution System Monitoring and Situational Awareness Predictive Control and Management Asset Management Electricity Markets Demand-Side Flexibility Conclusions and Future Challenges Acknowledgments References c11 On Data-Driven Approaches for Demand Response Introduction Sources of Big Data in DR Big Data Applications in DR Assessment of Energy Consumption Behavior Electric Load Classification Demand and Renewable Energy Generation Forecasting Dynamic Pricing Real-World Applications and Research on Big Data-Driven Demand Response Summary and Future Prospects Acknowledgment References c12 Topology Learning in Radial Distribution Grids Introduction Prior Work Technical Contribution Distribution Grid: Structure and Power Flows Radial Structure PF Model Properties of Voltage Magnitudes in Radial Grids Topology Learning With Full Observation Algorithm Complexity Extension to Multiple Trees Topology Learning With Missing Data Missing Nodes Separated by Three or More Hops Complexity Note All Nonleaf Nodes Are Missing Computational Complexity Experiments Conclusions References c13 Grid Topology Identification via Distributed Statistical Hypothesis Testing Introduction Related Works Power Distribution Grid Model Voltage Correlation Analysis A Distributed Topology Test Numerical Experiments Conclusions References c14 Supervised Learning-Based Fault Location in Power Grids Fundamentals of SVM Power System Applications of SVM Fault Classification and Location for Three-Terminal Transmission Lines SVM-Based Fault Classification Single-Ended Traveling Wave-Based Fault Location Results and Discussion Fault Location for Hybrid HVAC Transmission Lines Single-Ended Traveling Wave-Based Fault Location Results and Discussion Summary c15 Data-Driven Voltage Unbalance Analysis in Power Distribution Networks Chapter Points Introduction Problem Statement Unbalance in Low-Voltage Distribution Networks Utilized Voltage Band Data Acquisition and Storage Smart Meter Data Acquisition System Distributed Database Storage Distributed Data Processing Statistical Method Distributed Queries and Functions Data Discovery Distribution of Voltages Relation Between Meters With High Unbalance Events Timely Distribution of High Unbalance Events Relation Between Different Network States Maximum and Minimum Voltage: Voltage Spread Performance Evaluations Comparison Requirements Code Data Database System Evaluation Setup MapReduce Function CalcEventsLongFormat MapReduce Function MeterMinMax Conclusion References c16 Predictive Analytics for Comprehensive Energy Systems State Estimation Introduction Resource Forecasting Renewable Forecasting Wind Forecasting Wind Forecasting Overview Big Data-Driven Wind Forecasting Hours- to Day-Ahead NWP-Based Wind Forecasting Minutes- to 2-Hour-Ahead Machine-Learning-Based Wind Forecasting Wind Forecasting Datasets Solar Forecasting Solar Forecasting Overview Big Data-Driven Solar Forecasting Hours- to Day-Ahead NWP-Based Solar Forecasting Minutes- to 2-Hour-Ahead Sky Imaging-Based Solar Forecasting Renewable Forecasting Performance Evaluation Metrics User Energy System State Estimation Overview Load Forecasting Conventional Methods Artificial Intelligence-Based Methods Gaussian Process-Based Method User Energy System State Estimation: Demand Response Power System State Estimation Overview Conventional Nonlinear State Estimation PMU Data-Based Linear State Estimation and Dynamic State Estimation Predictive State Estimation Forecasting-Aided State Estimation Predictive State Estimation Distribution System State Estimation Conclusions References c17 Data Analytics for Energy Disaggregation: Methods and Applications Introduction Appliance Categories Device Classifications Based on the Operational States On/Off Finite State Machines Continuous Variable Device Permanent Consumer Device Classifications Based on the Corresponding Load Characteristics NILM Methodology Device Signatures Macroscopic Signatures Microscopic Signatures Nontraditional Signatures Disaggregation Algorithms Supervised Algorithms Pattern Recognition (Event-Based) Methods Optimization (Eventless) Methods Unsupervised Algorithms Accuracy Metrics Available Open Datasets Available Energy Disaggregation Open-Source Tools Main Use Cases of Energy Disaggregation Conclusion References c18 Energy Disaggregation and the Utility-Privacy Tradeoff Introduction Background Energy Disaggregation Background Utility-Privacy Tradeoff Background Fundamental Limits of NILM Problem Statement Model of Energy Disaggregation Algorithms Aggregate Device Model NILM Algorithms Fundamental Limits of Energy Disaggregation Distinguishing Two Scenarios Distinguishing a Finite Number of Scenarios Distinguishing Two Collections of Scenarios Gaussian Case Two Scenarios K Scenarios Linear Systems Utility-Privacy Tradeoff Framework The Utility of Data The Privacy of Data User and Data Mechanism Model Adversary Model Inferential Privacy Metric Example: Direct Load Control DLC Model Thermostatically Controlled Load Model Direct Load Control Objective Direct Load Control Capabilities Direct Load Controller DLC Model Simulations DLC Privacy Analysis Closing Remarks on the Utility-Privacy Tradeoff Conclusions References index Index A B C D E F G H I K L M N O P Q R S T U V W Y Annotation 'Big Data Application in Power Systems' brings together experts from academia, industry and regulatory agencies who share their understanding and discuss the big data analytics applications for power systems diagnostics, operation and control. Recent developments in monitoring systems and sensor networks dramatically increase the variety, volume and velocity of measurement data in electricity transmission and distribution level. The book focuses on rapidly modernising monitoring systems, measurement data availability, big data handling and machine learning approaches to process high dimensional, heterogeneous and spatiotemporal data. The book chapters discuss challenges, opportunities, success stories and pathways for utilising big data value in smart grids