I/O Evaluation
Once you have reviewed both CPU usage and memory usage you will want to look at I/O which can easily be a problem. The disk I/O is evaluated on the basis of throughput and latency. Disk drives make faster transfers when you have large sequential transfers than small random ones because of the movement of the head in reads and writes as well as disk rotation speeds. One thing to note is that I/O is closely connected to hardware so any questions here should be an evaluation of the hardware that is used.
One tool to evaluate I/O is iostat. The iostat command evaluates how long the disk is active in relation to the average transfer rates. The tps column shows the number of I/O requests that are given to the disk. The Blk_read/s and Blk_wrtn/s is the data read and written to the disk in number of blocks per second. Blk_read and Blk_wrtn are the total amounts that are read and written to the disk.
Evaluate Server Performance
Xeon
Linux 2.6.18-92.1.6.el5 (jm) 02/08/2009
avg-cpu: %user %nice %system %iowait %steal %idle
5.71 0.06 0.24 0.30 0.00 93.69
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 4.00 1.02 102.56 3409534 342968043
sdb 4.00 0.92 102.56 3089066 342968272
md2 0.63 0.84 4.84 2808714 16187024
md0 11.17 1.10 88.88 3682738 297230448
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/md2 4956224 4066752 633644 87% /
/dev/sda1 295561 25179 255122 9% /boot
/dev/md0 19840804 1499288 17317380 8% /var
Here is the output of vmstat.
procs ———–memory———- —swap– —–io—- –system– —–cpu——
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 108 331332 318992 1029688 0 0 0 26 0 2 6 0 94 0 0
Note that the blocks read per second (bo) are when there is no swap activity (si, so), so that would suggest that there is no performance issues related to swap. However, looking at the md0 which is RAID 1 that appears to be at least one problem. This is where there are the most blocks read and written and yet with RAID one the server is having to write twice for the provided redundancy.
Here is the output of iostat -x
Linux 2.6.18-92.1.6.el5 (jm) 02/08/2009
avg-cpu: %user %nice %system %iowait %steal %idle
5.70 0.06 0.24 0.30 0.00 93.69
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz
sda 0.01 8.28 0.03 3.97 1.02 102.51 25.87 0.10
sdb 0.01 8.28 0.03 3.97 0.92 102.51 25.84 0.07
md2 0.00 0.00 0.03 0.60 0.84 4.84 8.97 0.00
md0 0.00 0.00 0.06 11.10 1.10 88.84 8.06 0.00
Device: await svctm %util
sda 25.88 3.06 1.22
sdb 17.50 2.64 1.06
md2 0.00 0.00 0.00
md0 0.00 0.00 0.00
In evaluation of the information for the disk it looks like RAID 1 was a mistake in terms of performance. Initially, the server was not that busy but now it looks as if this is at least one bottleneck.
{ 2 trackbacks }