Re: drop in write performance

[Ason Hu <tobechihiro@xxxxxxxxx>]

Hi Pranith,

Gluster Version :

glusterfs 3.6.2 built on Jan 22 2015 12:58:10

• 100,000 file test in XFS directly
  

[root@glusterfs-node1 gfs_data]# java FileCreationAndSearchtest 10w_test 100000

Creation finished!! Time: 8961(ms)

File Search Finished!! 33223 searches, time: 124ms, misses:0

NIO Search Finished!! 33223 searches, time: 367ms, misses:0

• 100,000 file test from client (Distributed Replicated)
  

[root@glusterfs-client01 drvol]# java FileCreationAndSearchtest 10w_test01 100000

Start!!

Creation finished!! Time: 265713(ms)

File Search Finished!! 33453 searches, time: 18403(ms), misses:0

NIO Search Finished!! 33453 searches, time: 17827(ms), misses:0

after many small file created (≒200 million), it cost more than 1000000(ms) to finish 100,000 file test

I've flushed all volumes and create new one to reproduce the condition again.

2015-02-05 19:19 GMT+08:00 Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx>:

On 02/02/2015 02:42 PM, Ason Hu wrote:

Hi,

In my small file test, 3 types of volume are created in same GlusterFS Cluster (4 nodes)

• Distritubed
  
• Distributed Replicated
  
• Distributed Striped

10 million small files (8 byte) create and search each test cycle life 

Nodes info : 

• OS : CentOS 6.6 64bit
• RAM : 64GB
  
• HDD : 4TB, 7200RPM, SAS *6 RAID 0
  
• FileSystem : XFS, BS=4K

Problem :

It occur when file reach ≒200 million (sum of 3 volumes)

files created speed drop from  ≒1500 to  ≒100

If you do this test directly on the XFS partition, what is the kind of performance difference you see?

Pranith

Though the speed of creation become slow but search speed and throughput still good 

Seems each types of volume have same problem, I don't know what cause this problem..

Workaround :

I try to create new volume ( Distributed | Distributed Replicated | Distributed Striped ), mount it, 

problem solved!

But the old volume write speed still sad..

I'd  try to increase below volume setting, but don't work at all

• performance.cache-size 
  
• performance.io-thread-count

I didn't enable the volume qouta option.

Does glusterfs volume have some kind of limitation of file creation?

Regards

Nosa Hu,

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users