Mastering Python Scripting for System Administrators : Write Scripts and Automate Them for Real-world Administration Tasks Using Python

Cover Title Page Copyright and Credits Dedication About Packt Contributors Table of Contents Preface Chapter 1: Python Scripting Overview Technical requirements Why Python? Python syntax compared to other programming languages Python installation Installation on the Linux platform Installation on the Windows platform Installing and using pip to install packages Installation on Mac Installing Jupyter notebook Installing and using the virtual environment Installing Geany and PyCharm Python interpreter The difference between Python and Bash scripting Starting the interactive console Writing scripts with the Python interactive console Multiple lines Importing modules through the Python interpreter Exiting the Python console The keyboard shortcut Using the quit() or exit() functions Indentation and tabs Variables Creating and assigning values to variables Numbers Number type conversion Strings Concatenation (+) and repetition (*) String slicing Accessing values in strings Updating strings Escape characters Special string operators % string formatting operator Triple quotes in Python Strings are immutable Understanding lists Accessing values in lists Updating lists Deleting list elements Basic list operations List operations Indexing, slicing, and matrices Tuples Accessing values in tuples Updating tuples Deleting tuple elements Basic tuple operations Indexing, slicing, and matrices max() and min() Sets Dictionaries Parsing command-line arguments Command-line arguments in Python Sys.argv Decision making Python if statement syntax Python if...else statement syntax Python if...elif...else statement Loops for loop The range() function while loop Iterators Generators How to create a generator in Python? Functions The return statement Lambda functions Modules Importing modules Summary Questions Further reading Chapter 2: Debugging and Profiling Python Scripts What is debugging? Python debugging techniques Error handling (exception handling) Debuggers tools The pdb debugger Within an interpreter From a command line Within a Python script Debugging basic program crashes Profiling and timing programs The cProfile module timeit Making programs run faster Summary Questions Further reading Chapter 3: Unit Testing - Introduction to the Unit Testing Framework What is unittest? Creating unit tests Methods used in unit testing Summary Questions Further reading Chapter 4: Automating Regular Administrative Activities Accepting input by redirection, pipe, and input files Input by redirection Input by pipe Input by input file Handling passwords at runtime in scripts Executing external commands and getting their output Capturing output using the subprocess module Prompting for passwords during runtime and validation Reading configuration files Adding logging and warning code to scripts Generating warnings Putting limits on CPU and memory usage Launching webbrowser Using the os module for handling directory and files Creating and deleting the directory Examining the content of a filesystem Making backups (with rsync) Summary Questions Further reading Chapter 5: Handling Files, Directories, and Data Using the os module to work with directories Get the working directory Changing the directory Listing files and directories Renaming a directory Copying, moving, renaming, and deleting data Copying the data Moving the data Renaming data Deleting data Working with paths Comparing data Merging data Pattern matching files and directories Metadata: data about data Compressing and restoring Using the tarfile module to create TAR archives Using a tarfile module to examine the contents of TAR files Summary Questions Further reading Chapter 6: File Archiving, Encrypting, and Decrypting Creating and unpacking archives Creating archives Unpacking archives Tar archives ZIP creation File encryption and decryption Summary Questions Further reading Chapter 7: Text Processing and Regular Expressions Text wrapping The wrap() function The fill() function The dedent() function The indent() function The shorten() function Regular expressions The match() function The search() function The findall() function The sub() function Unicode strings Unicode code point Encoding Decoding Avoiding UnicodeDecodeError Summary Questions Further reading Chapter 8: Documentation and Reporting Standard input and output Information formatting Sending email Summary Questions Further reading Chapter 9: Working with Various Files Working with PDF files Reading a PDF document and getting the number of pages Extracting text Rotating PDF pages Working with Excel files Using the xlrd module Reading an Excel file Extracting the names of columns Using pandas Reading an Excel file Reading specific columns in an Excel file Using openpyxl Creating a new Excel file Appending values Reading multiple cells Working with CSV files Reading a CSV file Writing into a CSV file Working with txt files The open() function File opening The close() function Writing a text file Reading a text file Summary Questions Further reading Chapter 10: Basic Networking - Socket Programming Sockets The http package The http.client module The http.server module The ftplib module Downloading files Getting a welcome message using getwelcome(): Sending commands to the server using the sendcmd() function The urllib package Python urllib response headers Summary Questions Further reading Chapter 11: Handling Emails Using Python Scripting Email message format Adding HTML and multimedia content POP3 and IMAP servers Receiving email using the poplib library Receiving email using the imaplib library Summary Questions Chapter 12: Remote Monitoring of Hosts Over Telnet and SSH The telnetlib() module SSH The subprocess.Popen() module SSH using fabric module SSH using the Paramiko library SSH using the Netmiko library Summary Questions Further reading Chapter 13: Building Graphical User Interfaces Introduction to GUI Using a library to create a GUI-based application Installing and using the Apache Log Viewer app Summary Questions Further reading Chapter 14: Working with Apache and Other Log Files Parsing complex log files The need for exceptions Analyzing exceptions Tricks for parsing different files Error log Access log Common log format Parsing other log files Summary Questions Further reading Chapter 15: SOAP and REST API Communication What is SOAP? Using libraries for SOAP What is a RESTful API? Using standard libraries for RESTful APIs Working with JSON data Summary Questions Further reading Chapter 16: Web Scraping - Extracting Useful Data from Websites What is web scraping? Data extraction The requests library The beautifulsoup library Extracting information from Wikipedia Summary Questions Further reading Chapter 17: Statistics Gathering and Reporting NumPY module Using arrays and scalars Array indexing Indexing a 2D array Universal array functions Pandas module Series DataFrames Data visualization Matplotlib Histograms Scatter plots Bar charts Plotly Scatter plots Line scatter plots Box plots Contour plots Summary Questions Further reading Chapter 18: MySQL and SQLite Database Administrations MySQL database administration Getting a database version Creating a table and inserting data Retrieving the data Updating the data Deleting the data SQLite database administration Connecting to the database Creating a table Inserting the data Retrieving the data Updating the data Deleting the data Summary Questions Further reading Assessments Other Books You May Enjoy Index Leverage the features and libraries of Python to administrate your environment efficiently. Key Features Learn how to solve problems of system administrators and automate routine activities Learn to handle regular expressions, network administration Building GUI, web-scraping and database administration including data analytics Book Description Python has evolved over time and extended its features in relation to every possible IT operation. Python is simple to learn, yet has powerful libraries that can be used to build powerful Python scripts for solving real-world problems and automating administrators' routine activities. The objective of this book is to walk through a series of projects that will teach readers Python scripting with each project. This book will initially cover Python installation and quickly revise basic to advanced programming fundamentals. The book will then focus on the development process as a whole, from setup to planning to building different tools. It will include IT administrators' routine activities (text processing, regular expressions, file archiving, and encryption), network administration (socket programming, email handling, the remote controlling of devices using telnet/ssh, and protocols such as SNMP/DHCP), building graphical user interface, working with websites (Apache log file processing, SOAP and REST APIs communication, and web scraping), and database administration (MySQL and similar database data administration, data analytics, and reporting). By the end of this book, you will be able to use the latest features of Python and be able to build powerful tools that will solve challenging, real-world tasks What you will learn Understand how to install Python and debug Python scripts Understand and write scripts for automating testing and routine administrative activities Understand how to write scripts for text processing, encryption, decryption, and archiving Handle files, such as pdf, excel, csv, and txt files, and generate reports Write scripts for remote network administration, including handling emails Build interactive tools using a graphical user interface Handle Apache log files, SOAP and REST APIs communication Automate database administration and perform statistical analysis Who this book is for This book would be ideal for users with some basic understanding of Python programming and who are interested in scaling their programming skills to command line scripting and system administration. Prior knowledge of Python would be necessary. Table of Contents Python Scripting Overview Debugging and Profiling Python Scripts Unit Testing - Introduction to the Unit Testing Framework Automating Regular Administrative Activities Handling Files, Directories, and Data File Archiving, Encrypting, and Decrypting Text Processing and Regular Expressions Documentation and Reporting Working with Various Files Basic Networking - Socket Programming Handling Emails Using Python Scripting Remote Monitoring of Hosts Over Telnet and SSH Building Graphical User Interfaces Working with Apache and Other Log Files SOAP and REST API Communication Web Scraping - Extracting Useful Data from Websites Statistics Gathering and Reporting MySQL and SQLite Database Administrations Assessments