|
Dear all,
Recenly, I have installed the gLuster 3.2.2 source and do much IO performance on gLuster by tuning the cache-size and cache-timeout. And I found the cache-size doesn't work for multi bricks whick is strange and stuck me.
For example, volume info:
Volume Name: strip-vol
Type: Stripe
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: cloud04:/web-data01/data02 //multi bricks
Brick2: cloud04:/web-data01/data03
Options Reconfigured:
performance.cache-refresh-timeout: 60
performance.cache-size: 1024000000
test scripts:
for((i=0;i<3;i++)); do dd if=f20M of=/dev/zero bs=1M; sleep 2;done
test results:
First test:
Description: distributed volume with multi bricks
[root@cloud04 testfs]# for((i=0;i<3;i++)); do dd if=f20M of=/dev/zero bs=1M; sleep 2;done
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.107977 seconds, 89 MB/s
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.108642 seconds, 85 MB/s
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.10802 seconds, 90 MB/s
From the dd test doing in 1Gb/s Ethenet, I got the performance shows read a same files which should have cached rather than got them through network.
Second test:
Description: distributed volume with only one brick in a single server
Volume Name: test-vol
Type: Stripe
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: cloud04:/web-data01/data02 //single brick
Options Reconfigured:
performance.cache-refresh-timeout: 60
performance.cache-size: 1024000000
[root@cloud04 testfs]# for((i=0;i<3;i++)); do dd if=f20M of=/dev/zero bs=1M; sleep 2;done
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.107977 seconds, 89 MB/s
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.108642 seconds, 899 MB/s
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 0.10802 seconds, 897 MB/s
The result shows it read the cached file which is reasonable.
Could you know why I got the low performance when doing the first test with multi-bricks? Hence, I debug and trace the two volumes and found that a volume with single brick can cache the file accessed, then next time to read the same file again just read from cache not from network. But for a volume with multi-bricks, cache feature does not work. Could you give me some configuration tips to optimize IO performance when volume with multi-bricks.
Thank you for your any help in advance.
Cheers,
Qiulan
2011-10-18 ====================================================================
Computing center,the Institute of High Energy Physics, CAS, China
Qiulan
Huang Tel: (+86) 10 8823 6010-105
P.O. Box 918-7 Fax: (+86) 10 8823 6839
Beijing 100049 P.R. China Email: Qiulan.Huang@xxxxxxxxxx
===================================================================
|
