+91 – 7838219999

contact@nitinfotech.com

HomeTech SolutionsLinuxHow to Diagnose and Resolve High Server Load on Linux: A Comprehensive...

How to Diagnose and Resolve High Server Load on Linux: A Comprehensive Guide

Wednesday, July 17, 2024

Diagnosing the cause of high server load on a Linux server involves examining various aspects of system performance, including CPU, memory, disk I/O, and network usage. Here’s a structured approach using different tools and commands to identify the underlying cause of the high load:

1. Check High Server Load

uptime and top

  • uptime: Quickly shows load averages.
uptime

Output:

15:24:19 up 10 days, 2:35, 1 user, load average: 0.72, 1.01, 1.29
  • top: Provides a real-time view of system performance.
top

2. Analyze CPU Usage

top and htop

  • top: Check CPU usage.
top -o %CPU

Press Shift + P to sort by CPU usage.

  • htop: An enhanced version of top.
sudo apt-get install htop (On Ubuntu Machine)
sudo yum install htop htop (On Centos Machine)

Allow for easier navigation and better visualization.

mpstat

  • mpstat: Part of the sysstat package, useful for checking CPU usage by individual cores.
mpstat -P ALL 1

3. Check Memory Usage

free and vmstat

  • free: Shows memory usage.
free -h
  • vmstat: Provides detailed memory, CPU, and I/O statistics.
vmstat 1

4. Inspect Disk I/O

iostat , iotop , and dstat

  • iostat: Part of sysstat , useful for checking I/O statistics.
iostat -dx 1
  • iotop: Monitors I/O usage by processes.
sudo iotop
  • dstat: A versatile tool for various system statistics, including disk I/O.
dstat -d --disk-util --disk-tps

5. Examine Network Usage

ifstat , iftop , and netstat

  • ifstat: Displays network interface statistics.
sudo apt-get install ifstat  (On Ubuntu Machine)
sudo yum install ifstat (On Centos Machine)
ifstat 1
  • iftop: Monitors bandwidth usage by host.
sudo apt-get install iftop (On Ubuntu Machine)
sudo yum install iftop (On Centos Machine)
sudo iftop
  • netstat: Provides detailed network statistics.
netstat -at

6. Check for Heavy Processes

ps and pgrep

  • ps: Lists running processes. Sort by memory or CPU usage.
ps aux --sort=-%mem | head -n 10 # Sort by memory 
ps aux --sort=-%cpu | head -n 10 # Sort by CPU

  • pgrep: Finds processes based on name and other attributes.
pgrep -l high_load_process_name

7. Look for System Logs

dmesg and /var/log

  • dmesg: Shows kernel and driver messages.
dmesg | tail
  • /var/log/syslog or /var/log/messages: Check for system errors or warnings.
tail -f /var/log/syslog

8. Analyze Running Services

systemctl and service

  • systemctl: Check the status of running services.
systemctl list-units --type=service --state=running
  • service: Legacy command for managing services.
service --status-all

9. Check for Zombie Processes

top and ps

  • top: Look for processes with status Z (zombie)
  • ps: List zombie processes.
ps aux | grep 'Z'

10. Monitor System with sar

sar can collect, report, and save system activity information.

  • Install sysstat:
sudo apt-get install sysstat # For Debian/Ubuntu 
sudo yum install sysstat  # For CentOS/RHEL
  • Use sar to review CPU, memory, and I/O stats.
sar -u 1 5 # CPU usage 
sar -r 1 5 # Memory usage 
sar -d 1 5 # Disk I/O

Summary of Commands

Here’s a quick reference for the commands mentioned:

  • System Load:
uptime top
  • CPU Usage:
top -o %CPU htop mpstat -P ALL 1
  • Memory Usage:
free -h vmstat 1
  • Disk I/O:
iostat -dx 1 sudo iotop dstat -d --disk-util --disk-tps
  • Network Usage:
ifstat 1 sudo iftop netstat -at
  • Processes:
ps aux --sort=-%mem | head -n 10 ps aux --sort=-%cpu | head -n 10
  • Logs:
dmesg | tail tail -f /var/log/syslog
  • Services:
systemctl list-units --type=service --state=running service --status-all
  • Zombie Processes:
ps aux | grep 'Z'
  • sar Monitoring:
sar -u 1 5 sar -r 1 5 sar -d 1 5

Using these tools and commands, you should be able to pinpoint the cause of high load on your Linux server.