Configuration suggestions (aka poor/slow performance on new hardware)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Ramiro

ideas off the top of my head:

Get rid of performance/quick-read - it has a memory leak bug due to be 
fixed in gluster v3.0.5

If the files are going to be accessed by a program (which doesn't list 
the directories often) rather than a user (who might) then you can get 
rid of performance/stat-prefetch too.

In the long term - will this cluster be mostly used for serving files 
(ie. read-mostly) or will you be creating files as often as reading 
them? If mostly-read then get rid of performance/write-behind. Also the 
calculated option cache-size only uses 20% of your ram for the io-cache. 
Hard code it to a value you know works best. See also my write-up 
http://www.sirgroane.net/2010/03/tuning-glusterfs-for-apache-on-ec2 perhaps.

As you have physical disks then I'm guessing performance/read-ahead 
should be good for you.

Does the genfiles.sh script allow you to simulate multiple processes - 
if not then you're not seeing the full benefit of your 6 back-end stores...

Cheers,

Ian

On 26/03/2010 17:04, Ramiro Magallanes wrote:
> 	Hello there!
>
> Im working on a 6-nodes cluster, with SuperMicro new hardware.
> The cluster have to store a millons of JPG's about (200k-4MB),and little
> text files.
>
> Each node is :
>
> 	-Single Xeon(R) CPU E5405  @ 2.00GHz (4 cores)
> 	-4 GB RAM
> 	-64 bits Distro-based (Debian Lenny)
> 	-3ware 9650 sataII-raid, with 1 logical drive in raid 5 mode,  the unit
> with 3 sata hardisk of 2TB wdc with 64MB of cache each one.
> 	-Xfs filesystem on each logical unit.
>
> When i run the "genfiles.sh" test on each node in local (in the raid-5
> unit) mode i've have the follow results:
>
> 	-3143 files created in 60 seconds.
>
> and if i comment the "sync" line in the script:
>
> 	-8947 files created in 60 seconds.
>
> Now , with Gluster mounted (22TB) i run the test and the results are:
>
> 	-1370 files created in 60 seconds.
>
> Now, I'm running the cluster with standard distributed configuration,
> and i was making significant number of change in the test process , but
> i obtain the same number of wroted files all the time.
> Never more than 1400 files created, and 170mbits of network load (top).
>
> The switching layer is gigabit (obviusly) , and there's no high
> resources being used , all is normal.
>
> I'm using the 3.0.3 version of Gluster.
>
> Here is my configuration file (only the last part of the file):
>
> ##############################################################################
> volume distribute
>          type cluster/distribute
>          subvolumes 172.17.15.1-1 172.17.15.2-1 172.17.15.3-1
> 172.17.15.4-1 172.17.15.5-1 172.17.15.6-1
> end-volume
>
> volume writebehind
>          type performance/write-behind
>         option cache-size 1MB
>          option flush-behind on
>          subvolumes distribute
> end-volume
>
> volume readahead
>          type performance/read-ahead
>          option page-count 4
>          subvolumes writebehind
> end-volume
>
> volume iocache
>          type performance/io-cache
>          option cache-size `grep 'MemTotal' /proc/meminfo  | awk '{print
> $2 * 0.2 / 1024}' | cut -f1 -d.`MB
>
>          option cache-timeout 1
>          subvolumes readahead
> end-volume
>
> volume iothreads
>          type performance/io-threads
>          option thread-count 32 # default is 16
>          subvolumes distribute
> end-volume
>
> volume quickread
>      type performance/quick-read
>      option cache-timeout 1
>      option max-file-size 128kB
>      subvolumes iocache
> end-volume
>
> volume statprefetch
>      type performance/stat-prefetch
>      subvolumes quickread
> end-volume
> ##############################################################################
>
> Any idea or suggestion to make the performance goes up?
> Thanks everyone!
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>    


-- 
www.ContactClean.com
Making changing email address as easy as clicking a mouse.
Helping you keep in touch.




[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux