Hi,
I have build a two node cluster with GFS running on a shared SCSI disk array with integrated raid controller. The array contains 2 partitions: one raid0 partition of about 2 TB and one raid5 partition of about 1,5 TB. The stripe size is 512k.
I'm using kernel 2.4.21-27 for x86_64 and GFS version 6.0.2-17.
The problem is that a du -s takes about 6 minutes on either partition every time the command is run. I've mounted the partitions with noatime. Is this a normal time for GFS to do a du run on a 2TB partition?
The SCSI bus is running fine. I get about 190 MB/s bandwith to the array from both nodes. This is according to the specs of the internal raid controller.
When the command ran I noticed a lot of traffic on interface lo. This seems logical as this node is also running the lockmanager. But what bothers me is that the traffic does not acceed about 1,5 MB/s avarage. The loopback interface should be able to handle much more so therefore it looks that there some sort of bottleneck but I don't see it. Does anybody have a clue?
This is a partial capture of vmstat when the du command is running:
procs memory swap io system cpu
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 179524 192808 41552 316656 0 0 1542 16 326 1473 0 4 95 2
0 0 179524 189832 41552 319640 0 0 1492 0 305 1692 0 2 96 1
1 1 179524 187980 41556 321488 0 0 932 6 309 1536 0 4 96 0
0 0 179524 185372 41556 324088 0 0 1292 0 318 1516 0 2 97 1
0 0 179524 182744 41556 326680 0 0 1296 0 324 1896 0 1 97 2
0 0 179524 174532 41556 334872 0 0 4096 0 415 3358 1 4 94 1
0 0 179524 173524 41556 335880 0 0 504 0 295 1622 0 0 99 0
0 1 179524 171420 41560 337980 0 0 1050 6 374 1750 1 4 92 4
3 0 179524 164780 41560 344620 0 0 3320 0 794 3211 1 2 91 6
5 0 179524 161080 41560 348320 0 0 1850 0 417 2305 0 1 94 5
0 0 179524 157660 41564 351736 0 0 1708 6 336 2366 0 4 94 2
0 1 179524 155660 41564 353736 0 0 1008 0 306 1795 0 1 99 0
[root@hera raid0]# pwd
/mnt/raid0
[root@hera raid0]# time du -s .
144530864 .
real 5m27.894s
user 0m0.170s
sys 0m4.110s
kind regards,
Martijn Brizee
Linvision, The Netherlands
-- Linux-cluster@xxxxxxxxxx http://www.redhat.com/mailman/listinfo/linux-cluster
