From the MRTG data I can see that suddenly on a currently not very
busy machine the load averege went over 15 or more. This happened
around 10 in the morning. Not many entries in httpd access log, smtp
server was not too much loaded (at that time it generally produces a
load average of about 1).
But this time:
2008-02-27 10:03:36 1JUICO-0002Bs-23 no immediate delivery: load average 15.88
Where can I look for more information about what happened. The server
was rebooted early in the morning so its swap consumption was almost
non-existent. Around 10 it suddenly consumed probably around 200MB of
swap, now it stands at 135MB which is pretty normal with this machine
(it only has 512 RAM). SWAP usage is normal with this machine but it
usually is a matter of 2-3 days before it gradually takes over 10-13%
of swap space.
I tried by looking at:
http access log
Nothing in there indicating an outburst of sudden activities.
heh - I now think I may be wrong. The load average did not necessarily
produce so much swap consumption so fast. At 10:13 I run a cron job
optimising all mysql tables. So maybe in fact the swap was used by
mysql operations, although I am still interested to now what casued
load average to go above 15 at 10:03.
10:00AM up 3:14, 0 users, load averages: 0.40, 0.13, 0.05
10:01AM up 3:15, 0 users, load averages: 0.23, 0.14, 0.05
10:02AM up 3:16, 0 users, load averages: 0.08, 0.11, 0.05
10:03AM up 3:17, 0 users, load averages: 11.54, 3.09, 1.15
10:04AM up 3:18, 0 users, load averages: 13.26, 5.69, 2.28
10:05AM up 3:19, 0 users, load averages: 4.98, 4.69, 2.14
10:06AM up 3:20, 0 users, load averages: 1.79, 3.80, 1.99
Thanks for any pointers.
No way to know. You'll have to set up more detailed logging, e.g. a
script that runs ps if load is over a certain limit.