Tuning for small files?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello Gluster Community,

I was wondering if there are some things I could do to optimize gluster for small files.

 

Our bricks are 4 SSD in Raid0, and we have a replica  set factor of 3

 

We have a use-case where we would like to optimize our gluster mount for small files (about 6kb).

 

Here’s my current benchmarks on my mount

 

# 1mb block size on RAID0 SSD

dd if=/dev/zero of=/ssd/test_ssd_1mb bs=1M count=100

100+0 records in

100+0 records out

104857600 bytes (105 MB) copied, 0.0577655 s, 1.8 GB/s

 

# 1kb block size on RAID0 SSD

dd if=/dev/zero of=/ssd/test_ssd_1kb bs=1K count=100000

100000+0 records in

100000+0 records out

102400000 bytes (102 MB) copied, 0.181732 s, 563 MB/s

 

# 1mb block size on Gluster mount

dd if=/dev/zero of=/gluster/test_gluster_1mb bs=1M count=100

100+0 records in

100+0 records out

104857600 bytes (105 MB) copied, 0.219619 s, 477 MB/s

 

# 1kb block size on Gluster mount

dd if=/dev/zero of=/gluster/test_gluster_1kb bs=1K count=100000

100000+0 records in

100000+0 records out

102400000 bytes (102 MB) copied, 6.73986 s, 15.2 MB/s

 

As you can see, our performances on small block size are very low.

 

We’re running glusterfs 3.4.2, with the following options:

diagnostics.count-fop-hits: on

diagnostics.latency-measurement: off

performance.cache-max-file-size: 64KB

performance.write-behind-window-size: 8MB

performance.cache-size: 2GB

performance.io-thread-count: 64

performance.flush-behind: off

performance.io-cache: on

 

Thank you for your help

 

Jocelyn Hotte

Tools Programmer, Ubisoft

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux