Windows Performance Analysis Field Guide

Microsoft Windows 8.1 and Windows Server 2012 R2 are designed to be the best performing operating systems to date, but even the best systems can be overwhelmed with load and/or plagued with poorly performing code. **__Windows Performance Analysis Field Guide__** gives you a practical field guide approach to performance monitoring and analysis from experts who do this work every day. Think of this book as your own guide to "What would Microsoft support do?" when you have a Windows performance issue. Author Clint Huffman, a Microsoft veteran of over fifteen years, shows you how to identify and alleviate problems with the computer resources of disk, memory, processor, and network. You will learn to use performance counters as the initial indicators, then use various tools to "dig in" to the problem, as well as how to capture and analyze boot performance problems. This field guide gives you the tools and answers you need to improve Microsoft Windows performance, including: * Save money on optimizing Windows performance with deep technical troubleshooting that tells you "What would Microsoft do to solve this?" * Includes performance counter templates so you can collect the right data the first time. * Learn how to solve performance problems using free tools from Microsoft such as the Windows Sysinternals tools and more. * In a rush? Chapter 1 Start Here gets you on the quick path to solving the problem. * Also covers earlier versions such as Windows 7 and Windows Server 2008 R2. Front Cover Windows Performance Analysis Field Guide Copyright Dedication Acknowledgments Biography Contents Foreword Introduction Who should read this book? Chapter 1: Start here Introduction Is it a problem with boot or shutdown performance? Is it a hardware or software system hang? Troubleshooting hardware system hangs Review the System Event Logs Potential cause: Busy or malfunctioning hardware Hardware Diagnostics Potential cause: Virtual machine host Potential cause: Power and/or excessive temperatures Complete or partial system hangs Potential Cause: High Processor or Disk Usage Potential Cause: Lack of Kernel Pool Memory Potential Cause: High Processor Interrupts or DPCs Process terminated unexpectedly General slow system performance If on the console/desktop of a single system If monitoring one or more systems Common environmental and hardware-induced performance problems Power usage analysis Ensure the computer is plugged in with the proper power adapter Ensure the hard drives and optical drives are running at full performance Review the system's power plans Is the system overheated? Conclusion Windows Performance Analysis Industry Experts and Their Contact Information Chapter 2: Performance monitor Introduction Introduction to Performance Monitor Why do you need to know Performance Monitor? What are performance counters and can we trust them? Basic usage Starting Performance Monitor Adding and removing counters Hiding and showing counters Highlighting a counter The Last, Minimum, Average, and Maximum fields in the Graph Graph types View as a line chart View as a histogram (bar chart) View as a Report The overhead of viewing live performance counters Viewing remote performance counters Showing and hiding the console tree Counter instances are not automatically added or removed Gaps in Performance Monitor data Opening a counter log Using the time range control Performance counters Counter descriptions Counter paths Counters are always positive Unique instance names Special counter instances Scaling counters Automatic scaling Changing the scale of the chart Default scaling in log files Counter correlation Data collectors Introduction to data collectors Creating and starting a performance counter data collector using the wizard Deleting a data collector set Starting and stopping performance counter data collectors Creating a performance counter data collector template Using the PAL tool to create a data collector template Creating and starting a data collector set using a template Creating a circular data collector Selecting a sample interval Selecting counter paths for a performance counter data collector Impact of collecting counter data Managing the size of counter logs Creating a ``black box ́ ́ data collector Automatically starting a data collector set after a reboot Best practices for managing performance counter data collectors Tools Logman Creating and managing a performance counter data collector using Logman Querying a performance counter data collector using Logman Commonly used parameters of Logman Remote Server (-s ) Counter Path (-c ) File Type (-f ) Sample Interval (-si ) Begin Time (-b ) End Time (-e ) Output File Path (-o ) Counter File (-cf ) XML Template (-xml ) Maximum File Size (-max ) LODCTR and UNLODCTR Detecting and repairing corrupted performance counters Installing and uninstalling performance counters Relog Converting a counter log from binary to CSV using Relog Reducing the size of a counter log using Relog Counter Filtering (-c ) Counter Filtering with a File (-cf ) Data Filtering (-t ) Time filtering (-b and -e ) Fixing corrupted counter logs using Relog Merging counter logs using Relog Conclusion Chapter 3: Storage Introduction Initial indicators of poor disk performance Storage hardware and industry terminology Hardware and terminology I/O Request Packet I/Os per Second Hard Disk Drive (spindle) Solid-State Drive Direct-Attached Storage Just a Bunch of Disks and RAID Types Network-Attached Storage SAN and Fibre Channel Internet Small Computer Systems Interface I/O operations per second Dedicated versus shared spindles Physical disks and logical disks Disk capacity Identifying low storage capacity File system allocation size (block size) considerations Troubleshooting free space Disk Cleanup Wizard Disk Usage WinDirStat (Windows Directory Statistics) Windows Management Instrumentation (WMI) Understanding and measuring disk performance PhysicalDisk and LogicalDisk counter objects Understanding the disk queue Avg. Disk Queue Length Current Disk Queue Length % Idle Time Disk transfers, IOPS, and read/write ratios The effects of I/O sizes Split I/O I/O response times Process I/O operations Disk performance analysis tools Disk analysis using performance monitor Disk analysis using the PAL tool Disk analysis using task manager and resource monitor Disk analysis using process monitor Disk analysis using Windows Performance Analyzer Common causes, recommendations for poorly performing disks, and best practices Too much sharing Too much I/O demand High disk fragmentation Hardware failure ChkDsk FC troubleshooting Load testing disk performance and how to speak SAN-ish Capacity I/O Sizes IOPS Disk performance when load testing File system filter drivers Storport tracing Conclusion Disk performance analysis experts Chapter 4: Process memory Process virtual address space What you need to know about an application's virtual address space Identifying applications that run out of virtual address space How to determine the maximum virtual address space for an application Can the maximum virtual address space of an application be retrieved remotely using Windows Management Instrumentation? Identifying application virtual address space problems using Performance Monitor and the application event log Identifying application virtual address space problems using the PAL tool Investigating application virtual address space problems using VMMap About DebugDiag Preparing for a call with Microsoft Support Capture a performance counter log of when the application(s) ran out of memory Offer symbols files Dealing with 32-bit applications that run out of virtual address space Adding physical memory or increasing paging files has no effect Analyze memory usage and fix the code Recompile the 32-bit application to 64-bit Run the 32-bit application on a 64-bit version of Windows or Windows Server Consider IncreaseUserVA with caution Distribute the components of the 32-bit application across more processes Identifying and adding large address aware The concept and advantages of virtual memory 32-bit (x86) virtual address space 64-bit (x64) virtual address space Many processes, one kernel How can each application have a private 8TB on a system with 4GB of physical memory? Virtual memory and paging files Reserved, committed, and free memory Free memory Reserved memory Process committed memory Touched memory Identifying application out of virtual address space conditions System.OutOfMemoryException errors Read this if you are considering /3GB or IncreaseUserVa Identifying processes leaking system committed memory Some tools for measuring process private committed memory When is it considered a leak? Troubleshooting processes leaking system committed memory using Sysinternals VMMap Troubleshooting processes leaking system committed memory using debug dumps Treating the symptoms of process committed memory leaks Conclusion Chapter 5: Kernel memory Introduction What you need to know about kernel (system) memory Pool Paged Pool Nonpaged System page table entries Initial indicators of pool paged and pool nonpaged kernel memory What to look for What to do 64-bit (x64) versions of Windows and Windows Server Troubleshooting a lack of PTEs What are PTEs? What to look for What to do Monitoring kernel memory using process explorer Download Process Explorer Download and install the Debugging Tools for Windows Configure the symbol path Run as administrator View System Information Analyzing kernel memory using WPA Download and install the Windows Performance Toolkit Capture data Analyze in WPA Analyzing kernel memory using poolmon.exe Installing a kernel debugger The Debugging Tools for Windows on production systems Analyzing kernel memory with a kernel debugger The page frame number database, physical memory, and virtual address space Hot-add memory Read this if considering the /3GB switch or IncreaseUserVa Conclusion Twitter handles of kernel memory analysis experts Chapter 6: System committed memory Introduction The system commit limit How much is enough and how much is too much? The true system commit limit System-managed by default Monitoring system committed memory with Task Manager Monitoring system committed memory with Performance Monitor \Memory\% Committed Bytes In Use \Memory\Commit Limit \Memory\Committed Bytes Monitoring system committed memory with Sysinternals Process Explorer Current Limit Peak Peak/Limit Current/Limit Monitoring system committed memory with Windows Management Instrumentation FreeVirtualMemory TotalVirtualMemory Where did all of the system committed memory go? The usual suspects Process private memory AWE memory usage Kernel pool memory Driver locked memory System committed backed shared memory sections System cache resident memory Treating the symptoms of high system committed memory A case study of system committed memory Conclusion Chapter 7: Page files Introduction Page file sizing Systems with a low amount of physical memory Add more physical memory Reduce memory usage Add or move paging files to fast disks Systems with a large amount of physical memory Optionally reduce physical memory Optionally reduce the page file size System crash dumps Automatic memory dump System committed memory and paging files System-managed paging files Dedicated dump files What is written to a page file? Other crash dump-related registry keys IgnorePageFileSize DumpFile Other page file-related performance counters \Memory\Pages/sec and other hard page fault counters \Paging File(*)\% Usage Multiple page files and disk considerations Running without a page file Should the page file be moved from C: drive? Page file fragmentation Tracking page file reads and writes Using Resource Monitor Using Windows Performance Recorder/Analyzer Using Sysinternals Process Monitor High security? Consider cleaning the page file Conclusion Chapter 8: Physical memory Introduction Free memory is different than available memory Identifying a low-available-physical memory condition using Performance Monitor Low available memory with no page file Low available memory with a page file Identify the logical disks hosting an available page file Are the disks overwhelmed? Working set trims and page file usage An example Identifying a low available physical memory condition using task manager Identifying a low-available physical memory condition using Resource Monitor Monitoring for low-memory conditions using scripting Where did all of the physical memory go? Physical memory is consumed by, but not limited to, the following Adding it up with performance counters Process working sets Minimum working sets Driver-locked memory Address Windowing Extensions (AWE) Considerations of AWE memory usage Locking memory with Microsoft SQL Server Out of physical memory, but not out of committed memory How physical memory is managed Detecting bad physical memory Page faults Applications reading image files Memory-mapped files Page file reads and writes Hard page faults and disk performance Sizing physical memory ReadyBoost Prefetch Superfetch System cache Too much physical memory and power considerations Conclusion Chapter 9: Network Introduction Initial indicators Measuring the slowest node and black hole routers Monitoring network utilization using Performance Monitor Monitoring network utilization using Task Manager Monitoring network utilization using Resource Monitor Detecting NIC duplex settings Chattiness and latency Conclusion Chapter 10: Processor Introduction Identifying high processor usage using Task Manager Searching the file system for a process's executable file Identifying high processor usage using Performance Monitor Identifying high processor usage using Resource Monitor Identifying high processor usage using Process Explorer Introducing the Microsoft Windows Performance Analyzer Introducing Microsoft Xperf.exe Capturing and analyzing processor interrupts and DPC events using the Windows Performance Toolkit Capturing and analyzing user mode processor events using the Windows Performance Toolkit Capturing processor events using Microsoft WPR VM considerations Conclusion Chapter 11: Boot performance Introduction Common causes of poor boot performance Startup impact in Task Manager Using Autoruns to validate startup drivers, services, and applications Download and running Autoruns.exe Filter Autoruns Validate drivers Validating everything else Recording a boot trace using Windows Performance Recorder Analyzing a boot trace using WPA Boot phases in WPA The following are public resources for learning more about boot performance analysis using Event Tracing for Windows An example of a bad boot trace using the WPA Conclusion Chapter 12: Performance Analysis of Logs (PAL) Tool Introduction Installation and prerequisites Creating a counter log using a PAL template Using the PAL Wizard Start the PAL Wizard Welcome Counter log Threshold File Questions Output options File output Queue Execute Finish Interpreting the report Disclaimer Running the PAL tool without the PAL Wizard Examining the PAL log How to create a threshold file for the PAL tool Excluding counter instances Question variables (optional) Adding a threshold (optional) Adding visual thresholds to the chart (optional) Generated counters (optional) PAL version is incompatible Converting a Perfmon template to a PAL threshold file How to use the script Considerations Conclusion Appendix A: Tools Debug Diagnostic Tool (DebugDiag) v2.0 Microsoft Network Monitor 3.4 (Netmon) PathPing Performance Monitor (Perfmon) Poolmon Process Explorer Process Monitor RAMMap Resource Monitor (Resmon) Microsoft Server Performance Advisor Task Manager TCPView VMMap Windows Debugger (WinDBG) Windows Performance Analyzer Appendix B: Collecting Process Memory Dumps Using Task Manager Using DebugDiag Using ADPlus Using ProcDump Using Windows Error Reporting Using Process Explorer Using WinDBG Verifying the process memory dump file Appendix C: Debug symbols Introduction Using symbol paths Creating symbols Symbols and security concerns Managing symbol files using symbol servers Index

Microsoft Windows 8.1 and Windows Server 2012 R2 are designed to be the best performing operating systems to date, but even the best systems can be overwhelmed with load and/or plagued with poorly performing code. Windows Performance Analysis Field Guide gives you a practical field guide approach to performance monitoring and analysis from experts who do this work every day. Think of this book as your own guide to "What would Microsoft support do?" when you have a Windows performance issue.

Author Clint Huffman, a Microsoft veteran of over fifteen years, shows you how to identify and alleviate problems with the computer resources of disk, memory, processor, and network. You will learn to use performance counters as the initial indicators, then use various tools to "dig in" to the problem, as well as how to capture and analyze boot performance problems.

This field guide gives you the tools and answers you need to improve Microsoft Windows performance, including:

• Save money on optimizing Windows performance with deep technical troubleshooting that tells you "What would Microsoft do to solve this?"
• Includes performance counter templates so you can collect the right data the first time.
• Learn how to solve performance problems using free tools from Microsoft such as the Windows Sysinternals tools and more.
• In a rush? Chapter 1 Start Here gets you on the quick path to solving the problem.
• Also covers earlier versions such as Windows 7 and Windows Server 2008 R2.