On Tue, Mar 20, 2018 at 1:55 AM, TomK <tomkcpr@xxxxxxxxxxx> wrote:
On 3/19/2018 10:52 AM, Rik Theys wrote:
Hi,Correct. Higher block sizes gave me better numbers earlier. Curious about improving the small file size performance though, preferrably via gluster tunables, if possible.
On 03/19/2018 03:42 PM, TomK wrote:
On 3/19/2018 5:42 AM, Ondrej Valousek wrote:
Removing NFS or NFS Ganesha from the equation, not very impressed on my
own setup either. For the writes it's doing, that's alot of CPU usage
in top. Seems bottle-necked via a single execution core somewhere trying
to facilitate read / writes to the other bricks.
Writes to the gluster FS from within one of the gluster participating
bricks:
[root@nfs01 n]# dd if=/dev/zero of=./some-file.bin
393505+0 records in
393505+0 records out
201474560 bytes (201 MB) copied, 50.034 s, 4.0 MB/s
That's not really a fare comparison as you don't specify a blocksize.
What does
dd if=/dev/zero of=./some-file.bin bs=1M count=1000 oflag=direct
give?
Rik
Though it could be said I guess that compressing a set of large files and transferring them over that way is one solution. However needed the small block size on dd to perhaps quickly simulate alot of small requests in a somewhat ok-ish way.
Aggregating large number of small writes by write-behind into large writes has been merged on master:
Would like to know whether it helps for this usecase. Note that its not part of any release yet. So you've to build and install from repo.
Another suggestion is to run tests with turning off option performance.write-behind-trickling-writes.
# gluster volume set <volname> performance.write-behind-trickling-writes off
A word of caution though is if your files are too small, these suggestions may not have much impact.
Here's the numbers from the VM:
[ Via Gluster ]
[root@nfs01 n]# dd if=/dev/zero of=./some-file.bin bs=1M count=10000 oflag=direct
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 96.3228 s, 109 MB/s
[root@nfs01 n]# rm some-file.bin
rm: remove regular file âsome-file.binâ? y
[ Via XFS ]
[root@nfs01 n]# cd /bricks/0/gv01/
[root@nfs01 gv01]# dd if=/dev/zero of=./some-file.bin bs=1M count=10000 oflag=direct
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 44.79 s, 234 MB/s
[root@nfs01 gv01]#
top - 12:49:48 up 1 day, 9:39, 2 users, load average: 0.66, 1.15, 1.82
Tasks: 165 total, 1 running, 164 sleeping, 0 stopped, 0 zombie
%Cpu0 : 10.3 us, 9.6 sy, 0.0 ni, 28.0 id, 50.4 wa, 0.0 hi, 1.8 si, 0.0 st
%Cpu1 : 13.8 us, 13.8 sy, 0.0 ni, 38.6 id, 30.0 wa, 0.0 hi, 3.8 si, 0.0 st
%Cpu2 : 8.7 us, 6.9 sy, 0.0 ni, 48.7 id, 34.9 wa, 0.0 hi, 0.7 si, 0.0 st
%Cpu3 : 10.6 us, 7.8 sy, 0.0 ni, 57.1 id, 24.1 wa, 0.0 hi, 0.4 si, 0.0 st
KiB Mem : 3881708 total, 3543280 free, 224008 used, 114420 buff/cache
KiB Swap: 4063228 total, 3836612 free, 226616 used. 3457708 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14115 root 20 0 2504832 27640 2612 S 43.5 0.7 432:10.35 glusterfsd
1319 root 20 0 1269620 23780 2636 S 38.9 0.6 752:44.78 glusterfs
1334 root 20 0 2694264 56988 1672 S 16.3 1.5 311:20.90 ganesha.nfsd
27458 root 20 0 108984 1404 540 D 3.0 0.0 0:00.24 dd
14127 root 20 0 1164720 4860 1960 S 0.7 0.1 1:47.59 glusterfs
750 root 20 0 389864 5528 3988 S 0.3 0.1 0:08.77 sssd_be
--
Cheers,
Tom K.
------------------------------------------------------------ -------------------------
Living on earth is expensive, but it includes a free trip around the sun.
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-users
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users
