Re: slow write perf for disperse volume

[Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx>]

+Ashish

Ashish,

       Could you help Ingard? Do let me know what you find.

On Mon, Apr 24, 2017 at 4:50 PM, Ingard Mevåg <ingard@xxxxxxxx> wrote:

Hi. I can't see a fuse thread at all. Please see attached screenshot of top process with threads. Keep in mind this is from inside the container.

2017-04-24 12:17 GMT+02:00 Pranith Kumar Karampuri <pkarampu@xxxxxxxxxx>:

We were able to saturate hardware with EC as well. Could you check 'top' in threaded mode to see if fuse thread is saturated when you run dd?

On Mon, Apr 24, 2017 at 3:27 PM, Ingard Mevåg <ingard@xxxxxxxx> wrote:

Hi

I've been playing with disperse volumes the past week, and so far i can not get more than 12MB/s when i do a write test. I've tried a distributed volume on the same bricks and gotten close to gigabit speeds. iperf confirms gigabit speeds to all three servers in the storage pool.

The three storage servers have 10gig nics (connected to the same switch). The client is for a now a docker container in a 2nd DC (latency roughly 8-9 ms).

dpkg -l|grep -i gluster

ii  glusterfs-client               3.10.1-ubuntu1~xenial1          amd64        clustered file-system (client package)

ii  glusterfs-common               3.10.1-ubuntu1~xenial1          amd64        GlusterFS common libraries and translator modules

ii  glusterfs-server               3.10.1-ubuntu1~xenial1          amd64        clustered file-system (server package)

$ gluster volume info

Volume Name: DFS-ARCHIVE-001

Type: Disperse

Volume ID: 1497bc85-cb47-4123-8f91-a07f55c11dcc

Status: Started

Snapshot Count: 0

Number of Bricks: 1 x (4 + 2) = 6

Transport-type: tcp

Bricks:

Brick1: dna-001:/mnt/data01/brick

Brick2: dna-001:/mnt/data02/brick

Brick3: dna-002:/mnt/data01/brick

Brick4: dna-002:/mnt/data02/brick

Brick5: dna-003:/mnt/data01/brick

Brick6: dna-003:/mnt/data02/brick

Options Reconfigured:

transport.address-family: inet

nfs.disable: on

Anyone know the reason for the slow speeds on disperse vs distribute?

kind regards

ingard

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users

--

Pranith

--

Ingard Mevåg

Driftssjef

Jottacloud    
 
Mobil: +47 450 22 834
E-post: ingard@xxxxxxxxxxxxxx
Webside: www.jottacloud.com

--

Pranith

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users