Re: [External] Re: Samba+Gluster: Performance measurements for small files

[Davide Obbi <davide.obbi@xxxxxxxxxxx>]

Hi David,

i haven't tested samba but glusterfs fuse, i have posted the results few months ago, tests conducted using gluster 4.1.5:

Options Reconfigured:
client.event-threads 3
performance.cache-size 8GB
performance.io-thread-count 24
network.inode-lru-limit 1048576
performance.parallel-readdir on
performance.cache-invalidation on
performance.md-cache-timeout 600
features.cache-invalidation on
features.cache-invalidation-timeout 600
performance.client-io-threads on

nr of clients	6
Network	10Gb
Clients Mem	128GB
Clients Cores	22
Centos	7.5.1804
Kernel	3.10.0-862.14.4.el7.x86_64
nr of servers/bricks per volume	3
Network	100Gb	*node to node is 100Gb, cleints 10Gb
Server Mem	377GB
Server Cores	56	*Intel 5120 CPU
Storage	4x8TB NVME
Centos	7.5.1804
Kernel	3.10.0-862.14.4.el7.x86_64

This for example are FOPS with 128K IO size (cnsidered sweet spot for glusterfs according to documentation). In Blue 8threads per client and red 4threads for client

Below 4K

and 1MB

On Tue, Jan 22, 2019 at 9:09 AM David Spisla <spisla80@xxxxxxxxx> wrote:

Hello Amar,

thank you for the advice. We already use nl-cache option and a bunch of other settings. At the moment we try the samba-vfs-glusterfs plugin to access a gluster volume via samba. The performance increase now.

Additionally we are looking for some performance measurements to compare with. Maybe someone in the community also does performance tests. Does Redhat has some official reference measurement?

Regards

David Spisla

Am Di., 22. Jan. 2019 um 07:14 Uhr schrieb Amar Tumballi Suryanarayan <atumball@xxxxxxxxxx>:

For Samba usecase, please make sure you have nl-cache (ie, 'negative-lookup cache') enabled. We have seen some improvements from this value.

-Amar

On Tue, Dec 18, 2018 at 8:23 PM David Spisla <spisla80@xxxxxxxxx> wrote:

Dear Gluster Community,

it is a known fact that Samba+Gluster has a bad smallfile performance. We now have some test measurements created by this setup: 2-Node-Cluster on real hardware with Replica-2 Volume (just one subvolume), Gluster v.4.1.6, Samba v4.7. Samba writes to Gluster via FUSE. Files created by fio. We used a Windows System as Client which is in the same network like the servers.

The measurements are as follows. In each test case 400 files were written:

                       64KiB_x_400 files            1MiB_x_400 files           10MiB_x_400 files

1 Thread          0,77 MiB/s                       8,05 MiB/s                    72,67 MiB/s

4 Threads        0,86 MiB/s                       8,92 MiB/s                     90,38 MiB/s        

8 Threads        0,87 MiB/s                       8,92 MiB/s                     94,75 MiB/s

Does anyone have measurements that are in a similar range or are significantly different?
We do not know which values can still be considered "normal" and which are not. 
We also know that there are options to improve performance. But first of all we are interested 
in whether there are reference values.

Regards
David Spisla

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

--

Amar Tumballi (amarts)

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

--

Davide Obbi

Senior System Administrator

Booking.com B.V.
Vijzelstraat 66-80 Amsterdam 1017HL Netherlands

Direct +31207031558

Empowering people to experience the world since 1996

43 languages, 214+ offices worldwide, 141,000+ global destinations, 29 million reported listings 
Subsidiary of Booking Holdings Inc. (NASDAQ: BKNG)

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users