Hi
I have just moved our 400 TB HPC storage from lustre to gluster. It is part of a research institute and users have very small files to big files ( few KB to 20GB) . Our setup consists of 5 servers, each with 96TB RAID 6 disks. All servers are connected through 10G ethernet but not all clients. Gluster volumes are distributed without any replication. There are approximately 80 million files in file system.I am mounting using glusterfs on clients.
time du -sh /data/aa/bb/cc
3.7G /data/aa/bb/cc
real 7m29.243s
user 0m1.448s
sys 0m7.067s
time du -sh /data/aa/bb/cc
3.7G /data/aa/bb/cc
real 16m43.735s
user 0m1.097s
sys 0m5.802s
16m and 7m is too long for a 3.7 G directory. I must mention that the directory contains huge number of files (208736)
but running du on same directory on old data gives this result
time du -sh /olddata/aa/bb/cc
4.0G /olddata/aa/bb/cc
real 3m1.255s
user 0m0.755s
sys 0m38.099s
time du -sh /olddata/aa/bb/cc
4.0G /olddata/aa/bb/cc
real 3m1.255s
user 0m0.755s
sys 0m38.099s
much better if I run same command again
time du -sh /olddata/aa/bb/cc
4.0G /olddata/aa/bb/cc
real 0m8.309s
user 0m0.313s
sys 0m7.755s
time du -sh /olddata/aa/bb/cc
4.0G /olddata/aa/bb/cc
real 0m8.309s
user 0m0.313s
sys 0m7.755s
Is there anything I can do to improve this performance? I would also like hear from some one who is running same kind of setup.
Thanks
Kashif
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users