Why so bad performance?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



OK, They are HP 380 G4 rack server boxes.
P4 Xeon 3.0GHz*2 with Hyper-Thread, 4G RAM.
scsi 72g*5 raid5 for /exports/ns, sata 250G*7*2 raid5 for /exports/disk1 and
/exports/disk2.
All of the 4 boxes are connected with gigabit Ethernet switcher.
10.10.123.21 and 10.10.123.22 are servers, while 10.10.123.25 and
10.10.65.64 are clients.


OS is RHEL-5.2/x86_64, kerner-2.6.18-92el5.
And important rpms are:
fuse.x86_64-2.7.4-1.el5.rf 				from dag.wieers.com
dkms-fuse.noarch-2.7.4-1.nodist.rf		from dag.wieers.com
dkms.noarch-2.0.20.4-1.el5.rf			from dag.wieers.com
glusterfs.x86_64-1.3.10-1				from glusterfs.org

I have already post my COMPLETE spec file in the very beginning post. The 2
servers use the same spec file.
[@123.21 /]# glusterfsd -f /etc/glusterfs/ glusterfs-server.vol
[@123.21 /]# glusterfsd -f /etc/glusterfs/ glusterfs-server.vol
[@123.25 /]# glusterfs -s 10.10.123.21 /mnt
[@65.64/]# glusterfs -s 10.10.123.21 /mnt

I have tried run 'dd' concurrently on the 2 client boxes. The sum of 2
client's speed is only 13MB/s, a bit more than solo.

-----Original Message-----
From: a s p a s i a [mailto:aspasia.sf at gmail.com] 
Sent: Saturday, December 06, 2008 1:43 AM
To: Kirby Zhou
Cc: RedShift; gluster-users at gluster.org
Subject: Re: Why so bad performance?

interesting ...

did you mention what is your HW for the glusterFS server side?  can
you post complete specs of your config?

- HW - proc speed/type, RAM, etc.  - where you have installed both
GlusterFS server and client?

- a.


On Fri, Dec 5, 2008 at 8:30 AM, Kirby Zhou <kirbyzhou at sohu-rd.com> wrote:
> I have tested using scp:
>
> [@123.25 /]# scp /opt/xxx 10.10.123.22:/opt/
> xxx                100%  256MB  51.2MB/s   00:05
>
> [@123.25 /]# dd if=/opt/xxx of=/mnt/xxx bs=2M
> 128+0 records in
> 128+0 records out
> 268435456 bytes (268 MB) copied, 23.0106 seconds, 11.7 MB/s
>
> So, you can see how slow the speed my gluster.
> I wanna what can I do to improve the performance.
>
> -----Original Message-----
> From: gluster-users-bounces at gluster.org
> [mailto:gluster-users-bounces at gluster.org] On Behalf Of RedShift
> Sent: Friday, December 05, 2008 11:45 PM
> To: gluster-users at gluster.org
> Subject: Re: Why so bad performance?
>
> Hello Kirby,
>
>
> Please check if every involved device is running at gigabit speed and test
> with at least 100 mb of data.
>
>
> Glenn
>
> Kirby Zhou wrote:
>> I conustructed a 2-server/ 1 client gluster with Gigabit-ethernet, but
got
>> so bad a benchmark.
>> Is there any thing can I tune?
>>
>> [@65.64 ~]# for ((i=0;i<17;++i)) ; do dd if=/dev/zero of=/mnt/yyy$i bs=4M
>> count=2 ; done
>> 2+0 records in
>> 2+0 records out
>> 8388608 bytes (8.4 MB) copied, 0.770213 seconds, 10.9 MB/s
>> 2+0 records in
>> 2+0 records out
>> 8388608 bytes (8.4 MB) copied, 0.771131 seconds, 10.9 MB/s
>> ...
>>
>> [@123.21 glusterfs]# cat glusterfs-server.vol
>> volume brick1
>>  type storage/posix
>>  option directory /exports/disk1
>> end-volume
>>
>> volume brick2
>>  type storage/posix
>>  option directory /exports/disk2
>> end-volume
>>
>> volume brick-ns
>>  type storage/posix
>>  option directory /exports/ns
>> end-volume
>>
>> ### Add network serving capability to above brick.
>> volume server
>>  type protocol/server
>>  option transport-type tcp/server     # For TCP/IP transport
>>  subvolumes brick1 brick2 brick-ns
>>  option auth.ip.brick1.allow 10.10.* # Allow access to "brick" volume
>>  option auth.ip.brick2.allow 10.10.* # Allow access to "brick" volume
>>  option auth.ip.brick-ns.allow 10.10.* # Allow access to "brick-ns"
volume
>> end-volume
>>
>> [@123.21 glusterfs]# cat glusterfs-client.vol
>> volume remote-brick1_1
>>   type protocol/client
>>   option transport-type tcp/client
>>   option remote-host 10.10.123.21
>>   option remote-subvolume brick1
>> end-volume
>>
>> volume remote-brick1_2
>>   type protocol/client
>>   option transport-type tcp/client
>>   option remote-host 10.10.123.21
>>   option remote-subvolume brick2
>> end-volume
>>
>> volume remote-brick2_1
>>   type protocol/client
>>   option transport-type tcp/client
>>   option remote-host 10.10.123.22
>>   option remote-subvolume brick1
>> end-volume
>>
>> volume remote-brick2_2
>>   type protocol/client
>>   option transport-type tcp/client
>>   option remote-host 10.10.123.22
>>   option remote-subvolume brick2
>> end-volume
>>
>> volume brick-afr1_2
>>   type cluster/afr
>>   subvolumes remote-brick1_1 remote-brick2_2
>> end-volume
>>
>> volume brick-afr2_1
>>   type cluster/afr
>>   subvolumes remote-brick1_2 remote-brick2_1
>> end-volume
>>
>> volume remote-ns1
>>   type protocol/client
>>   option transport-type tcp/client
>>   option remote-host 10.10.123.21
>>   option remote-subvolume brick-ns
>> end-volume
>>
>> volume remote-ns2
>>   type protocol/client
>>   option transport-type tcp/client
>>   option remote-host 10.10.123.22
>>   option remote-subvolume brick-ns
>> end-volume
>>
>> volume ns-afr0
>>   type cluster/afr
>>   subvolumes remote-ns1 remote-ns2
>> end-volume
>>
>> volume unify0
>>   type cluster/unify
>>   option scheduler alu
>>   option alu.limits.min-free-disk 10%
>>   option alu.order disk-usage
>>   option namespace ns-afr0
>>   subvolumes brick-afr1_2 brick-afr2_1
>> end-volume
>>
>>
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>>
>>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>



-- 
A S P A S I A
. . . . . . . . . . ..





[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux