Without knowing the details, I'm putting my money on cache. Choosing how to mount Gluster is workload dependent. If you're doing a lot of small files with single threaded writes, I suggest NFS. Your client's nfscache will dramatically improve performance from the end-user's point of view. If you're doing heavy multi-threaded reads and writes, and you have very good bandwidth from your client (e.g.: 10GbE) FUSE+GlusterFS is better, as it allows your client to talk to all Gluster nodes. If you are using FUSE+GlusterFS, on the gluster nodes themselves, experiment with the "performance.write-behind-window-size" and "performance.cache-size" options. Note that these will affect the cache used by the clients, so don't set them so high as to exhaust the RAM of any client connecting (or, for low-memory clients, use NFS instead). Gluster ships with conservative defaults for cache, which is a good thing. It's up to the user to tweak for their optimal needs. There's no right or wrong answer here. Experiment with NFS and various cache allocations with FUSE+GlusterFS, and see how you go. And again, consider your workloads, and whether or not they're taking full advantage of the FUSE client's ability to deal with highly parallel workloads. -Dan ---------------- Dan Mons - VFX Sysadmin Cutting Edge http://cuttingedge.com.au On 18 February 2016 at 08:56, Stefan Jakobs <stefan@xxxxxxxxxxxxx> wrote: > Van Renterghem Stijn: >> Interval2 >> Block Size: 1b+ 16b+ 32b+ >> No. of Reads: 0 0 >> 0 No. of Writes: 342 25 >> 575 >> >> Block Size: 64b+ 128b+ >> 256b+ No. of Reads: 0 0 >> 0 No. of Writes: 143 898 >> 118 >> >> Block Size: 512b+ 1024b+ >> 2048b+ No. of Reads: 1 4 >> 11 No. of Writes: 82 0 >> 0 >> >> Block Size: 4096b+ 8192b+ >> 16384b+ No. of Reads: 11 31 >> 39 No. of Writes: 0 0 >> 0 >> >> Block Size: 32768b+ 65536b+ >> 131072b+ No. of Reads: 59 148 >> 555 No. of Writes: 0 0 >> 0 >> >> %-latency Avg-latency Min-Latency Max-Latency No. of calls >> Fop --------- ----------- ----------- ----------- ------------ >> ---- 0.00 0.00 us 0.00 us 0.00 us 1 >> FORGET 0.00 0.00 us 0.00 us 0.00 us 201 >> RELEASE 0.00 0.00 us 0.00 us 0.00 us 54549 >> RELEASEDIR 0.00 47.00 us 47.00 us 47.00 us 1 >> REMOVEXATTR 0.00 94.00 us 74.00 us 114.00 us 2 >> XATTROP 0.00 191.00 us 191.00 us 191.00 us 1 >> TRUNCATE 0.00 53.50 us 35.00 us 74.00 us 4 >> STATFS 0.00 79.67 us 70.00 us 91.00 us 3 >> RENAME 0.00 37.33 us 27.00 us 68.00 us 15 >> INODELK 0.00 190.67 us 116.00 us 252.00 us 3 >> UNLINK 0.00 28.83 us 8.00 us 99.00 us 30 >> ENTRYLK 0.00 146.33 us 117.00 us 188.00 us 6 >> CREATE 0.00 37.63 us 12.00 us 73.00 us 84 >> READDIR 0.00 23.75 us 8.00 us 75.00 us 198 >> FLUSH 0.00 65.33 us 42.00 us 141.00 us 204 >> OPEN 0.01 45.78 us 11.00 us 191.00 us 944 >> FINODELK 0.01 80.34 us 31.00 us 211.00 us 859 >> READ 0.02 96.74 us 50.00 us 188.00 us 944 >> FXATTROP 0.02 55.84 us 24.00 us 140.00 us 1707 >> FSTAT 0.02 52.89 us 21.00 us 175.00 us 2183 >> WRITE 0.02 59.69 us 11.00 us 235.00 us 2312 >> GETXATTR 0.03 51.18 us 8.00 us 142.00 us 3091 >> STAT 0.46 48.66 us 1.00 us 179.00 us 54549 >> OPENDIR 1.13 135.93 us 18.00 us 16362.00 us 48124 >> READDIRP 98.29 70.46 us 16.00 us 2903.00 us 8104385 >> LOOKUP >> >> Duration: 7560 seconds >> Data Read: 91208567 bytes = 91MB >> Data Written: 292007 bytes = 0,292MB > > How did you collect these statistics? > > Thanks > Stefan > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
