Mastering Python Networking: Your one-stop solution to using Python for network automation, DevOps, and Test-Driven Development, 2nd Edition

Key FeaturesExplore the power of Python libraries to tackle difficult network problems efficiently and effectivelyUse Python for network device automation, DevOps, and software-defined networkingBecome an expert in implementing advanced network-related tasks with PythonBook DescriptionNetworks in your infrastructure set the foundation for how your application can be deployed, maintained, and serviced. Python is the ideal language for network engineers to explore tools that were previously available to systems engineers and application developers. In this second edition of Mastering Python Networking, you'll embark on a Python-based journey to transition from traditional network engineers to network developers ready for the next-generation of networks. This book begins by reviewing the basics of Python and teaches you how Python can interact with both legacy and API-enabled network devices. As you make your way through the chapters, you will then learn to leverage high-level Python packages and frameworks to perform network engineering tasks for automation, monitoring, management, and enhanced security. In the concluding chapters, you will use Jenkins for continuous network integration as well as testing tools to verify your network. By the end of this book, you will be able to perform all networking tasks with ease using Python.What you will learnUse Python libraries to interact with your networkIntegrate Ansible 2.5 using Python to control Cisco, Juniper, and Arista eAPI network devicesLeverage existing frameworks to construct high-level APIsLearn how to build virtual networks in the AWS CloudUnderstand how Jenkins can be used to automatically deploy changes in your networkUse PyTest and Unittest for Test-Driven Network DevelopmentWho this book is forMastering Python Networking is for network engineers and programmers who want to use Python for networking. Basic familiarity with Python programming and networking-related concepts such as Transmission Control Protocol/Internet Protocol (TCP/IP) will be useful. Title Page......Page 2 Copyright and Credits......Page 4 Mastering Python Networking Second Edition......Page 5 Packt Upsell......Page 6 Why subscribe?......Page 7 PacktPub.com......Page 8 Contributors......Page 9 About the author......Page 10 About the reviewer......Page 11 Packt is searching for authors like you......Page 12 Preface......Page 20 Who this book is for......Page 21 What this book covers......Page 22 To get the most out of this book......Page 24 Download the example code files......Page 25 Download the color images......Page 26 Conventions used......Page 27 Get in touch......Page 28 Reviews......Page 29 Review of TCP/IP Protocol Suite and Python......Page 30 An overview of the internet ......Page 32 Servers, hosts, and network components......Page 33 The rise of data centers......Page 34 Enterprise data centers......Page 35 Cloud data centers......Page 36 Edge data centers......Page 37 The OSI model......Page 38 Client-server model......Page 40 Network protocol suites......Page 41 The transmission control protocol......Page 42 Functions and characteristics of TCP......Page 43 TCP messages and data transfer......Page 44 User datagram protocol......Page 45 The internet protocol......Page 46 The IP NAT and security......Page 47 IP routing concepts......Page 48 Python language overview......Page 49 Python versions......Page 50 Operating system......Page 51 Running a Python program......Page 52 Python built-in types......Page 53 The None type......Page 54 Numerics......Page 55 Sequences......Page 56 Mapping......Page 58 Sets......Page 59 Python operators......Page 60 Python control flow tools......Page 61 Python functions......Page 63 Python classes......Page 64 Python modules and packages......Page 65 Summary......Page 67 Low-Level Network Device Interactions......Page 68 The challenges of the CLI......Page 69 Constructing a virtual lab......Page 71 Cisco VIRL......Page 73 VIRL tips......Page 75 Cisco DevNet and dCloud......Page 78 GNS3......Page 80 Python Pexpect library......Page 82 Pexpect installation......Page 83 Pexpect overview......Page 84 Our first Pexpect program......Page 88 More Pexpect features......Page 89 Pexpect and SSH......Page 90 Putting things together for Pexpect......Page 91 The Python Paramiko library......Page 92 Installation of Paramiko......Page 93 Paramiko overview......Page 94 Our first Paramiko program......Page 96 More Paramiko features......Page 97 Paramiko for servers......Page 98 Putting things together for Paramiko......Page 99 Looking ahead......Page 100 Downsides of Pexpect and Paramiko compared to other tools......Page 101 Idempotent network device interaction......Page 102 Bad automation speeds bad things up......Page 103 Summary......Page 104 APIs and Intent-Driven Networking......Page 105 Infrastructure as code......Page 106 Intent-Driven Networking......Page 107 Screen scraping versus API structured output......Page 108 Data modeling for infrastructure as code......Page 111 The Cisco API and ACI......Page 112 Cisco NX-API......Page 113 Lab software installation and device preparation......Page 114 NX-API examples......Page 115 The Cisco and YANG models......Page 118 The Cisco ACI......Page 119 The Python API for Juniper networks......Page 121 Juniper and NETCONF......Page 122 Device preparation......Page 123 Juniper NETCONF examples......Page 124 Juniper PyEZ for developers......Page 126 Installation and preparation......Page 127 PyEZ examples......Page 129 The Arista Python API......Page 131 Arista eAPI management......Page 132 The eAPI preparation......Page 133 eAPI examples......Page 135 The Arista Pyeapi library......Page 137 Pyeapi installation......Page 138 Pyeapi examples......Page 139 Vendor-neutral libraries......Page 141 Summary......Page 142 The Python Automation Framework – Ansible Basics......Page 143 A more declarative framework......Page 144 A quick Ansible example......Page 146 The control node installation......Page 147 Running different versions of Ansible from source......Page 148 Lab setup......Page 149 Your first Ansible playbook......Page 150 The public key authorization......Page 151 The inventory file......Page 152 Our first playbook......Page 153 The advantages of Ansible......Page 155 Agentless......Page 156 Idempotent......Page 157 Simple and extensible......Page 158 Network vendor support......Page 159 The Ansible architecture......Page 160 YAML......Page 161 Inventories......Page 162 Variables......Page 163 Templates with Jinja2......Page 166 Ansible networking modules......Page 167 Local connections and facts......Page 168 Provider arguments......Page 169 The Ansible Cisco example......Page 171 Ansible 2.5 connection example......Page 173 The Ansible Juniper example......Page 175 The Ansible Arista example......Page 177 Summary......Page 178 The Python Automation Framework – Beyond Basics......Page 179 Ansible conditionals......Page 180 The when clause......Page 181 Ansible network facts......Page 183 Network module conditional......Page 185 Ansible loops......Page 186 Standard loops......Page 187 Looping over dictionaries......Page 188 Templates......Page 190 The Jinja2 template......Page 191 Jinja2 loops......Page 192 The Jinja2 conditional......Page 193 Group and host variables......Page 195 Group variables......Page 196 Host variables......Page 197 The Ansible Vault......Page 198 The Ansible include and roles......Page 200 The Ansible include statement......Page 201 Ansible roles......Page 202 Writing your own custom module......Page 205 The first custom module......Page 206 The second custom module......Page 208 Summary......Page 209 Network Security with Python......Page 210 The lab setup......Page 211 Python Scapy......Page 214 Installing Scapy......Page 215 Interactive examples......Page 216 Sniffing......Page 217 The TCP port scan......Page 218 The ping collection......Page 220 Common attacks......Page 221 Scapy resources......Page 222 Access lists......Page 223 Implementing access lists with Ansible......Page 224 MAC access lists......Page 226 The Syslog search......Page 227 Searching with the RE module......Page 228 Other tools......Page 230 Private VLANs......Page 231 UFW with Python......Page 232 Further reading......Page 233 Summary......Page 234 Network Monitoring with Python – Part 1......Page 235 Lab setup......Page 236 SNMP......Page 237 Setup......Page 239 PySNMP......Page 241 Python for data visualization......Page 246 Matplotlib......Page 247 Installation......Page 248 Matplotlib – the first example......Page 249 Matplotlib for SNMP results......Page 251 Additional Matplotlib resources......Page 255 Pygal......Page 256 Installation......Page 257 Pygal – the first example......Page 258 Pygal for SNMP results......Page 260 Additional Pygal resources......Page 263 Python for Cacti......Page 264 Installation......Page 265 Python script as an input source......Page 267 Summary......Page 269 Network Monitoring with Python – Part 2......Page 270 Graphviz......Page 271 Lab setup......Page 272 Installation......Page 274 Graphviz examples......Page 275 Python with Graphviz examples......Page 277 LLDP neighbor graphing......Page 278 Information retrieval......Page 280 Python parser script......Page 281 Final playbook......Page 284 Flow-based monitoring......Page 286 NetFlow parsing with Python......Page 287 Python socket and struct......Page 289 ntop traffic monitoring......Page 292 Python extension for ntop......Page 295 sFlow......Page 299 SFlowtool and sFlow-RT with Python......Page 300 Elasticsearch (ELK stack)......Page 303 Setting up a hosted ELK service......Page 304 The Logstash format......Page 306 Python helper script for Logstash formatting......Page 307 Summary......Page 309 Building Network Web Services with Python......Page 310 Comparing Python web frameworks......Page 312 Flask and lab setup......Page 314 Introduction to Flask......Page 316 The HTTPie client......Page 317 URL routing......Page 319 URL variables......Page 320 URL generation......Page 321 The jsonify return......Page 322 Network resource API......Page 323 Flask-SQLAlchemy......Page 324 Network content API......Page 326 Devices API......Page 328 The device ID API......Page 330 Network dynamic operations......Page 331 Asynchronous operations......Page 333 Security......Page 335 Additional resources......Page 337 Summary......Page 338 AWS Cloud Networking......Page 339 AWS setup......Page 340 AWS CLI and Python SDK......Page 342 AWS network overview......Page 345 Virtual private cloud......Page 348 Route tables and route targets......Page 352 Automation with CloudFormation......Page 354 Security groups and the network ACL......Page 357 Elastic IP......Page 359 NAT Gateway......Page 360 Direct Connect and VPN......Page 362 VPN Gateway......Page 363 Direct Connect......Page 364 Network scaling services......Page 365 Elastic Load Balancing......Page 366 Route53 DNS service......Page 367 CloudFront CDN services......Page 368 Other AWS network services......Page 369 Summary......Page 370 Working with Git......Page 371 Introduction to Git......Page 372 Benefits of Git......Page 373 Git terminology......Page 374 Git and GitHub......Page 375 Setting up Git......Page 376 Gitignore......Page 377 Git usage examples......Page 378 GitHub example......Page 382 Collaborating with pull requests......Page 386 Git with Python......Page 390 GitPython......Page 391 PyGitHub......Page 392 Automating configuration backup......Page 394 Collaborating with Git......Page 397 Summary......Page 398 Continuous Integration with Jenkins......Page 399 Traditional change-management process......Page 400 Introduction to continuous integration......Page 402 Installing Jenkins......Page 403 Jenkins example......Page 405 First job for the Python script......Page 406 Jenkins plugins......Page 412 Network continuous integration example......Page 414 Jenkins with Python......Page 422 Continuous integration for Networking......Page 423 Summary......Page 424 Test-Driven Development for Networks......Page 425 Test-driven development overview......Page 426 Test definitions......Page 427 Topology as code......Page 428 Python's unittest module......Page 431 More on Python testing......Page 434 pytest examples......Page 435 Writing tests for networking......Page 437 Testing for reachability......Page 438 Testing for network latency......Page 439 Testing for security......Page 440 Testing for transactions......Page 441 Testing for network configuration......Page 442 Testing for Ansible......Page 443 Pytest in Jenkins......Page 444 Jenkins integration......Page 445 Summary......Page 450 Other Books You May Enjoy......Page 451 Leave a review - let other readers know what you think......Page 454 This book begins by reviewing the basics of Python and teaches how Python can interact with both legacy and API-enabled network devices, and how to leverage high-level Python packages and frameworks to perform network engineering tasks for automation, monitoring, management, and enhanced security. Also, in the concluding chapters, the book shows how to use Jenkins for continuous network integration as well as testing tools to verify network