Network security through data analysis : from data to action

Traditional intrusion detection and logfile analysis are no longer enough to protect today's complex networks. In the updated second edition of this practical guide, security researcher Michael Collins shows InfoSec personnel the latest techniques and tools for collecting and analyzing network traffic datasets. You'll understand how your network is used, and what actions are necessary to harden and defend the systems within it. In three sections, this book examines the process of collecting and organizing data, various tools for analysis, and several different analytic scenarios and techniques. New chapters focus on active monitoring and traffic manipulation, insider threat detection, data mining, regression and machine learning, and other topics. You'll learn how to: Use sensors to collect network, service, host, and active domain data Work with the SiLK toolset, Python, and other tools and techniques for manipulating data you collect Detect unusual phenomena through exploratory data analysis (EDA), using visualization and mathematical techniques Analyze text data, traffic behavior, and communications mistakes Identify significant structures in your network with graph analysis Examine insider threat data and acquire threat intelligence Map your network and identify significant hosts within it Work with operations to develop defenses and analysis techniques Traditional intrusion detection and logfile analysis are no longer enough to protect today’s complex networks. In this practical guide, security researcher Michael Collins shows you several techniques and tools for collecting and analyzing network traffic datasets. You’ll understand how your network is used, and what actions are necessary to protect and improve it.Divided into three sections, this book examines the process of collecting and organizing data, various tools for analysis, and several different analytic scenarios and techniques. It’s ideal for network administrators and operational security analysts familiar with scripting.Explore network, host, and service sensors for capturing security dataStore data traffic with relational databases, graph databases, Redis, and HadoopUse SiLK, the R language, and other tools for analysis and visualizationDetect unusual phenomena through Exploratory Data Analysis (EDA)Identify significant structures in networks with graph analysisDetermine the traffic that’s crossing service ports in a networkExamine traffic volume and behavior to spot DDoS and database raidsGet a step-by-step process for network mapping and inventory System and network administrators have traditionally monitored their systems through general tools such as intrusion detection and logfile analysis. But modern, complex networks, suffering from more and more sophisticated attacks, deserve more analytical tools. Michael Collins, a leading researcher in security, introduces the techniques needed in this book and highlights some of the computing tools that will help catch problems. The book is divided into three large sections: data collection, analysis, and taking action. These can be iterative, as each discovery alerts the administrator to data that should be collected. Several forms of analysis and visualization are covered. Topics include: What data to capture on your systems Data fusion Structures and storage systems for data Using R, SiLK, and Python for analysis Visualization and exploratory data analysis Graph analysis Network mapping Address forensics: determining where traffic originates Handling malware Traditional intrusion detection and logfile analysis are no longer enough to protect today’s complex networks. In the updated second edition of this practical guide, security researcher Michael Collins shows InfoSec personnel the latest techniques and tools for collecting and analyzing network traffic datasets. You’ll understand how your network is used, and what actions are necessary to harden and defend the systems within it. In three sections, this book examines the process of collecting and organizing data, various tools for analysis, and several different analytic scenarios and techniques. New chapters focus on active monitoring and traffic manipulation, insider threat detection, data mining, regression and machine learning, and other topics. -- Provided by publisher Discusses techniques for collecting, organizing, and analyzing network traffic data, covering such topics as data fusion; using SiLK, R, and Python; network mapping and inventory; and handling malware