Re: Gluster - Performance issue while copying bulk files/folders

[Игорь Бирюлин <biryulini@xxxxxxxxx>]

Hello,
I doesn't use version 3.7 in production.

I think need try new options (I don't know how it improve work with big files but it should improve work with small files):
gluster v set prodcmsroot client.event-threads 4
gluster v set prodcmsroot server.event-threads 4
gluster v set prodcmsroot cluster.lookup-optimize on

Моre info about this:
https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/Small_File_Performance_Enhancements.html

And may be increase number of threads in IO threads translator:
gluster v set prodcmsroot performance.io-thread-count 48

I recommend try this on test environment.

2015-12-10 17:29 GMT+03:00 Srikanth Mampilakal <shrikanth1980@xxxxxxxxx>:

Hi members,

Really appreciate if you can share your thoughts or any feedback for resolving the slow copy issue

Regards
Srikanth

On 10-Dec-2015 2:12 AM, "Srikanth Mampilakal" <srikanth.mampilakal@xxxxxxxxx> wrote:

Hi,

I have production gluster file service used as a shared storage where the content management system uses it as document root. I have run in to a performance issue with the gluster/fuse client.

Looking for your thoughts and experience in resolving Gluster performance issues:

Gluster Infrastructure

Gluster version :GlusterFS 3.7.6

2 gluster nodes of the same config below

Redhat EL7.0-64

Memory : 4GB

Processor : 2 x 2.0 Ghz

Network : 100 Mbps

File Storage Volume : NETAPP Storage LUN with 2.0 IOPS/GB

Gluster Volume information:

[root@GlusterFileServe1 ~]# gluster volume info

Volume Name: prodcmsroot

Type: Replicate

Volume ID: f1284bf0-1939-46f9-a672-a7716e362947

Status: Started

Number of Bricks: 1 x 2 = 2

Transport-type: tcp

Bricks:

Brick1: Server1:/glusterfs/brick1/prodcmsroot

Brick2: Server2:/glusterfs/brick1/prodcmsroot

Options Reconfigured:

performance.io-thread-count: 64

performance.cache-size: 1073741824

performance.readdir-ahead: on

performance.write-behind-window-size: 524288

[root@GlusterFileServe1  ~]#

The replication between Gluster node are quick and consistent. 

The apache webservers are accessing the Gluster volume using native gluster fuse client and located in the same VLAN as the Gluster Server.

GlusterFileServe1:/prodcmsroot  /mnt/glusterfs glusterfs direct-io-mode=disable,defaults,_netdev 0 0

The server utilization (memory,cpu,network and disk 1/0) is relatively low

I am experiencing very slow performance while copying multiple file/folders (approx 75 MB) and it takes atleast approx 35 min. Even copy a folder (with multiple files/subfolders) within the Gluster volume take the same time.

However, if I do dd to check the copy speed, I get the below result.

[root@ClientServer ~]#  time sh -c "dd if=/dev/zero of=/mnt/testmount/test.tmp bs=4k count=20000 && sync"

20000+0 records in

20000+0 records out

81920000 bytes (82 MB) copied, 17.1357 s, 4.8 MB/s

real    0m17.337s

user    0m0.031s

sys     0m0.317s

Anyone experience the same kind of performance issue, please let me know your thoughts.

Cheers

Srikanth

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users