OpenWest 2014/Linux Performance Tools

by Ed Felt - DBA/Developer at LDS Church

Keep it simple ... at least at first

No Mockumentation - Document well, even over document

Recommended course - Linux Foundation - Linux Performance Tools (LF426) - ~$3,000
 * https://training.linuxfoundation.org/linux-courses/system-administration-training/linux-performance-tuning

Track changes as you test

Human memory sucks - keep personal notes (journaling)

Keep focused on one variable at a time (Exception: fix it now emergency fire!) - most of us think linearly

Examples:
 * ngrep to troubleshoot Segmentation Fault that doesn't make it to the logs
 * actime0=0 # NFS  (?kill it/keep it?)

Stress testing is NOT base-lining/profiling or even troubleshooting

/proc - TLDP describes it as "window" in to kernel and processes

PIDS:
 * /proc/PID/cmdline
 * /proc/PID/cwd
 * /proc/PID/environ
 * /proc/PID/exe
 * ps

CPU:
 * /proc/cpuinfo
 * top

Memory:
 * /proc/meminfo
 * free

Netstat:
 * /proc/net/netstat
 * netstat

Logs:
 * /var/log/messages
 * /var/log/[application]

Write your own:
 * BASH
 * Perl
 * Python

Tools:
 * hdparm -tT /dev/....
 * top
 * sar (sysstat)
 * collectl
 * mlmon

Other tools:
 * watch
 * tail
 * strace
 * iostat
 * vmstat
 * nmon
 * iptraf
 * lsof
 * htop