Dear Gluster community, I have problems with tuning Gluster for small files performance on SSD. My usage scenario is, as I've learned, worst possible scenario, but it's not up to me to change it: - small 1KB files - at least 20M of those - approx. 10 files/directory - mostly writes - average speed 1000 files/sec with peaks up to 10K files/sec. I'm doing something wrong, because I cannot get past performance metrics 4K files/sec for distributed volume (2 bricks) 2K files/sec for replicated volume (2 bricks). I've been experimenting with various XFS formatting and mounting options and with Gluster tuning (md-cache, lookup optimize, thread, writeback, tiering), but no luck. I've learned that it's not disk IO that is the bottleneck (direct tests on mounted XFS partition show waaay better results, like 100K files/sec). As I've learned from http://blog.gluster.org/2014/03/experiments-using-ssds-with-gluster/ it's possible to get 24K files/sec performance (and that was three years ago). Test I'm using, run on one server (2 x Xeons, 256 GB RAM, 10GbE network): smallfile_cli.py --operation create --threads 32 --file-size 1 --files 15625 --top /mnt/testdir/test Setup: 2 servers with 2 Xeons each, 256GB RAM, 8 x 800GB SSD drives in RAID6, 10GbE network Ubuntu 14.04 Gluster 3.7.3 Do you have any hints where I should start investigating for the bottleneck? Best regards, Szymon _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users
