Re: Fwd:Re: client is terrible with large amount of small files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 05/05/2015 03:43 PM, Kamal wrote:
Hi Vijay,

       We Tried the same. But its doesn't improve speed.

For testing glusterfs, we are running storage in virtual machines. Is
that will make any difference ?

Performance testing on physical machines is a better bet as it takes some variables away from the equation.

Since you happen to use a replicated volume, can you try by enabling this option?

gluster volume set <volname> cluster.ensure-durability off

Additionally you might want to try "strace -Tcf git clone .." and "gluster volume profile ..." to figure out where the latency is stemming from.


But, copying the same folder between two storage machines is really
fast. fyi.


Does this copying involve gluster or not?

Regards,
Vijay

Regards,
Kamal



---- On Tue, 05 May 2015 15:08:27 +0530 *Vijay
Bellur<vbellur@xxxxxxxxxx>* wrote ----

    On 05/05/2015 12:59 PM, Kamal wrote:
     > Hi Amukher,
     >
     > Even after upgrade to 3.7 small files transfer rate is slow.
     >
     > Below is the volume info.
     >
     > Volume Name: integvol1
     > Type: Replicate
     > Volume ID: 31793ba4-eeca-462a-a0cd-9adfb281225b
     > Status: Started
     > Number of Bricks: 1 x 2 = 2
     > Transport-type: tcp
     > Bricks:
     > Brick1: integ-gluster1:/srv/sdb2/brick4
     > Brick2: integ-gluster2:/srv/sdb2/brick4
     > Options Reconfigured:
     > server.event-threads: 30
     > client.event-threads: 30
     > ----
     >
     > I understand that for replication it would take some more time, but
     > here its taking more time.
     >
     >
     > Time taken for git clone in non gluster directory = 25 sec
     >
     > Time taken for git clone in gluster directory = 14 minutes
     >
     > Its a huge difference. Plz let me know any other tuning
    parameters need
     > to be done.
     >
     >

    I have seen this before and it primarily seems to be related to the
    readdir calls done by git clone.

    Turning on these options might help to some extent:

    gluster volume set <volname> performance.readdir-ahead on

    gluster volume set <volname> cluster.readdir-optimize on

    Please do let us know what you observe with these options enabled.

    Regards,
    Vijay




_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users




[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux