Multiagent Systems : Introduction and Coordination Control

Multiagent systems (MAS) are one of the most exciting and the fastest growing domains in the intelligent resource management and agent-oriented technology, which deals with modeling of autonomous decisions making entities. Recent developments have produced very encouraging results in the novel approach of handling multiplayer interactive systems. In particular, the multiagent system approach is adapted to model, control, manage or test the operations and management of several system applications including multi-vehicles, microgrids, multi-robots, where agents represent individual entities in the network. Each participant is modeled as an autonomous participant with independent strategies and responses to outcomes. They are able to operate autonomously and interact pro-actively with their environment. In recent works, the problem of information consensus is addressed, where a team of vehicles communicate with each other to agree on key pieces of information that enable them to work together in a coordinated fashion. The problem is challenging because communication channels have limited range and there are possibilities of fading and dropout. The book comprises chapters on synchronization and consensus in multiagent systems. It shows that the joint presentation of synchronization and consensus enables readers to learn about similarities and differences of both concepts. It reviews the cooperative control of multi-agent dynamical systems interconnected by a communication network topology. Using the terminology of cooperative control, each system is endowed with its own state variable and dynamics. A fundamental problem in multi-agent dynamical systems on networks is the design of distributed protocols that guarantee consensus or synchronization in the sense that the states of all the systems reach the same value. It is evident from the results that research in multiagent systems offer opportunities for further developments in theoretical, simulation and implementations. This book attempts to fill this gap and aims at presenting a comprehensive volume that documents theoretical aspects and practical applications. Multiagent systems are one of the most exciting and the fastest growing domains in the intelligent resource management and agent-oriented technology, which deals with modelling of autonomous decisions making entities. Recent developments have produced ve Cover 1 Title Page 2 Copyright Page 3 Dedication 4 Preface 5 Acknowledgement 7 Table of Contents 8 Author Biography 14 1: Introduction 16 1.1 Overview 16 1.2 Elements of Graph Theory 17 1.2.1 Basic Results 18 1.2.2 Laplacian Spectrum of Graphs 18 1.2.3 Properties of Adjacency Matrix 19 1.2.4 Nonlinear Stochastic Dynamical Systems 22 1.2.5 Complex Dynamical Systems 27 1.2.6 Delay Effects 30 1.2.7 Sampled-Data Framework 31 1.3 Multiagent System Approach 32 1.3.1 Practical Examples 32 1.3.2 Some Relevant Definitions 33 1.4 Mathematical Models for Agent Dynamics 35 1.4.1 Single Integrator Model 36 1.4.2 Double Integrator Model 36 1.4.3 Uncertain Fully Actuated Model 37 1.4.4 Non-Holonomic Unicycle Model 38 1.5 Coordination and Control Problems 38 1.5.1 Aggregation and Social Foraging 39 1.5.2 Flocking and Rendezvous 40 1.5.3 Synchronization of Coupled Nonlinear Oscillators 40 1.6 Scope and Book Layout 42 2: Theoretical Background 44 2.1 Preliminaries of Distributed Systems 44 2.1.1 Problem Description 45 2.1.2 Control Design Scheme 47 2.1.3 Without Communication Delays 48 2.1.4 With Communication Delays 52 2.2 Networked Multiagent Systems 53 2.2.1 Consensus in Networks 55 2.2.2 The f-Consensus Problem 56 2.2.3 Iterative Consensus and Markov Chains 57 2.3 Applications 57 2.3.1 Synchronization of Coupled Oscillators 57 2.3.2 Flocking Theory 58 2.3.3 Fast Consensus in Small-Worlds 58 2.3.4 Rendezvous in Space 59 2.3.5 Distributed Sensor Fusion in Sensor Networks 59 2.3.6 Distributed Formation Control 59 2.4 Information Consensus 60 2.4.1 Algebraic Connectivity and Spectral Properties 62 2.4.2 Convergence Analysis for Directed Networks 62 2.4.3 Consensus in Discrete-Time 65 2.4.4 Performance of Consensus Algorithms 67 2.4.5 Alternative Forms of Consensus Algorithms 69 2.4.6 Weighted-Average Consensus 72 2.4.7 Consensus Under Communication Time-Delays 73 2.5 Consensus in Switching Networks 74 2.6 Cooperation in Networked Control Systems 76 2.6.1 Collective Dynamics of Multivehicle Formation 77 2.6.2 Stability of Relative Dynamics 78 2.7 Simulation Studies 79 2.7.1 Consensus in Complex Networks 79 2.7.2 Multivehicle Formation Control 81 2.8 Notes 82 3: Distributed Intelligence in Power Systems 84 3.1 Introduction to MAS Technology 84 3.1.1 Autonomous Microgrid System 86 3.1.2 A State-Space Model 86 3.1.3 Heuristic Dynamic Programming 88 3.1.4 Discrete-Time Bellman Equation 89 3.1.5 Value Iteration Algorithm 90 3.1.6 Adaptive Critics Implementation 91 3.1.7 Actor-Critic Implementation 92 3.1.8 Simulations Results 93 3.1.9 Actor-Critic Tuning Results 93 3.1.10 Robustness of the Proposed Controller 95 3.2 Operation in Islanded Mode 96 3.2.1 Autonomous Microgrid 100 3.2.2 Primary Control 101 3.2.3 Fixed Gain Distributed Secondary Control 104 3.2.4 Neural Network Distributed Secondary Control 106 3.2.5 Stage 1: Selection of Training Data 106 3.2.6 Stage 2: Selection of Artificial Neural Network 109 3.2.7 Stage 3: Neural Network Training 110 3.2.8 Simulation Results I 111 3.3 Multiagent Coordination for Distributed Energy Resources 118 3.3.1 Introduction 118 3.3.2 Advantages of MAS Approach 119 3.3.3 Agent Platform 120 3.3.4 Software System Analysis 121 3.3.5 Distributed Control System 122 3.3.6 Simulation Studies I 123 3.3.7 Coordination Between Agents 125 3.3.8 Checking Reliability 127 3.3.9 Simulation Results II 128 3.4 Notes 128 4: Consensus for Heterogeneous Systems with Delays 132 4.1 Introduction 132 4.2 Multiagent Leader-Follower Consensus Problem 133 4.3 Distributed Adaptive Control Design 135 4.4 Illustrative Example 144 4.5 Tracking and Coordination Using Sensor Networks 145 4.6 Target Tracking in Sensor Networks 150 4.7 Control System Architecture 151 4.7.1 Sensor Network and Models 153 4.7.2 Multitarget Tracking 155 4.7.3 Agent Dynamics and Coordination Objective 158 4.8 Control System Implementation 160 4.8.1 Multisensor Fusion Module 160 4.8.2 Multitarget Tracking and Multitrack Fusion Modules 165 4.8.3 Multiagent Coordination Module 170 4.9 Experimental Results 176 4.9.1 Platform 178 4.9.2 Live Demonstration 180 4.10 Notes 183 5: Secure Control of Distributed Multiagent Systems 184 5.1 Introduction 184 5.2 Problem Formulation 186 5.3 Main Results 187 5.4 Illustrative Examples 192 5.5 Notes 195 6: Advanced Consensus Algorithms 198 6.1 Event-Triggered Control for Multiagent Systems 198 6.1.1 Introduction 198 6.1.2 System Model and Problem Statement 200 6.1.3 Design Tracking Results 203 6.1.4 Numerical Example 210 6.2 Pinning Coordination Control of Networked Systems 213 6.2.1 Networked Multi-Vehicle Systems 216 6.2.2 Fixed Communication Topology 221 6.2.3 Case of General Graphs 222 6.2.4 Example 6.1 224 6.2.5 Strongly Connected and Balanced Graphs 226 6.2.6 Selection of the Pinned Nodes 229 6.2.7 Pinning Control with Variable Topology 233 6.2.8 Simulation Examples 234 6.3 Distributed Consensus Control 236 6.3.1 Consensus with Observer-Type Protocol 239 6.3.2 Dynamic Consensus 240 6.3.3 Consensus Region 242 6.3.4 Consensus with Neutrally Stable Matrix 243 6.3.5 Consensus with Prescribed Convergence Speed 245 6.3.6 Illustrative Example 6.2 247 6.3.7 Consensus with Static Protocols 248 6.3.8 Formation Control 250 6.3.9 Illustrative Example 6.3 251 6.4 Consensus Control for Time-Delay Systems 253 6.4.1 Problem Formulation 253 6.4.2 Fixed Interconnection Topology 256 6.4.3 Switched Interconnection Topology 260 6.4.4 Illustrative Example 6.4 263 6.4.5 Illustrative Example 6.5 264 6.5 Robust Consensus of Multiagent Systems 265 6.5.1 Problem Description 266 6.5.2 Analytic Results 268 6.5.3 Illustrative Example 6.6 276 6.6 Notes 285 7: Cooperative Control of Networked Power Systems 288 7.1 Coordinated Model Predictive Power Flows 288 7.1.1 Introduction 289 7.1.2 System Architecture 290 7.1.3 Wind Power Generation 291 7.1.4 Photovoltaic Module Generators 292 7.1.5 Energy Storage System Dynamics 292 7.1.6 Loads 293 7.1.7 Energy Management Unit 293 7.1.8 Power Price Mechanism 293 7.2 Power Scheduling in Networked MG 294 7.2.1 Networked Topology 294 7.2.2 GCC of Networked MG 294 7.2.3 MPC-Based Power Scheduling 294 7.2.4 Optimization Problem Formulation 295 7.2.5 State Equations and Constraints 296 7.2.6 Case Studies 297 7.2.7 Simulation Setup 298 7.2.8 Case Study 1 298 7.2.9 Case Study 2 305 7.2.10 Case Study 3 307 7.3 Distributed Robust Control in Smart Microgrids 308 7.3.1 A Microgrid Model 308 7.3.2 Microgrid Group Model 308 7.3.3 Problem Formulation 310 7.3.4 Robust Group Control 311 7.3.5 Distributed Information Models 313 7.3.6 Simulation Study 314 7.3.7 Solution Procedure A 315 7.3.8 Solution Procedure B 316 7.3.9 Solution Procedure C 316 7.3.10 Solution Procedure D 316 7.4 Notes 317 8: Dynamic Graphical Games 320 8.1 Constrained Graphical Games 320 8.1.1 Reinforcement Learning 321 8.1.2 Synchronization Control Problem 322 8.1.3 Performance Evaluation of the Game 323 8.1.4 Optimality Conditions 324 8.1.5 Bellman Equations 325 8.1.6 the Hamiltonian Function 326 8.1.7 Coupled IRL-Hamilton-Jacobi Theory 328 8.1.8 Coupled IRL-HJB Equations 330 8.1.9 Nash Equilibrium Solution 332 8.1.10 Stability Analysis 333 8.2 Value Iteration Solution and Implementation 335 8.2.1 Value Iteration Algorithm 336 8.2.2 Graph Solution Implementation 336 8.2.3 Online Actor-Critic Neural Networks Tuning 338 8.2.4 Simulation Results I 338 8.2.5 Simulation Case 1 339 8.2.6 Simulation Case 2 339 8.2.7 Simulation Case 3 340 8.3 Multiagent Reinforcement Learning for Microgrids 346 8.3.1 Microgrid Control Requirements 347 8.3.2 Features of MAS Technology 348 8.3.3 A Multiagent Reinforcement Learning Method 350 8.3.4 Critical Operation in Island Mode 353 8.3.5 Simulation Results II 355 8.4 Notes 359 References 360 Index 410 Resource;,Management;,Agent,Oriented;,Technology Resource,Management,Agent Oriented,Technology "Multiagent systems offer tremendous opportunities for development in computing and its applications. The objective of this book is to identify preliminary requirements and fundamental issues related to multiagent systems; provide coherent solutions for adopting multiagent framework for examining problems critically in smart microgrid systems; and present advanced analysis of multiagent systems under cyber-physical attacks and develop resilient control strategies to guarantee safe operation. The book is a comprehensive volume on the subject"-- Provided by publisher