Tested it in two different environments lately with exactly same results. Was trying to get better read performance from local mounts with hundreds of thousands maildir email files by using SSD, hoping that .gluster file stat read will improve which does migrate to hot tire. After seeing what you described for 24 hours and confirming all move around on the tires is done - killed it. Here are my volume settings - maybe will be useful to spot conflicting ones. cluster.shd-max-threads: 12 performance.rda-cache-limit: 128MB cluster.readdir-optimize: on cluster.read-hash-mode: 0 performance.strict-o-direct: on cluster.lookup-unhashed: auto performance.nl-cache: on performance.nl-cache-timeout: 600 cluster.lookup-optimize: on client.event-threads: 8 performance.client-io-threads: on performance.md-cache-timeout: 600 server.event-threads: 8 features.cache-invalidation: on features.cache-invalidation-timeout: 600 performance.stat-prefetch: on performance.cache-invalidation: on network.inode-lru-limit: 90000 performance.cache-refresh-timeout: 10 performance.enable-least-priority: off performance.cache-size: 2GB cluster.nufa: on cluster.choose-local: on server.outstanding-rpc-limit: 128 fuse mounting defaults,_netdev,negative-timeout=10,attribute-timeout=30,fopen-keep-cache,direct-io-mode=enable,fetch-attempts=5 On Tue, Jan 30, 2018 at 6:29 PM, Jeff Byers <jbyers.sfly@xxxxxxxxx> wrote: > I am fighting this issue: > > Bug 1540376 – Tiered volume performance degrades badly after a > volume stop/start or system restart. > https://bugzilla.redhat.com/show_bug.cgi?id=1540376 > > Does anyone have any ideas on what might be causing this, and > what a fix or work-around might be? > > Thanks! > > ~ Jeff Byers ~ > > Tiered volume performance degrades badly after a volume > stop/start or system restart. > > The degradation is very significant, making the performance of > an SSD hot tiered volume a fraction of what it was with the > HDD before tiering. > > Stopping and starting the tiered volume causes the problem to > exhibit. Stopping and starting the Gluster services also does. > > Nothing in the tier is being promoted or demoted, the volume > starts empty, a file is written, then read, then deleted. The > file(s) only ever exist on the hot tier. > > This affects GlusterFS FUSE mounts, and also NFSv3 NFS mounts. > The problem has been reproduced in two test lab environments. > The issue was first seen using GlusterFS 3.7.18, and retested > with the same result using GlusterFS 3.12.3. > > I'm using the default tiering settings, no adjustments. > > Nothing of any significance appears to be being reported in > the GlusterFS logs. > > Summary: > > Before SSD tiering, HDD performance on a FUSE mount was 130.87 > MB/sec writes, 128.53 MB/sec reads. > > After SSD tiering, performance on a FUSE mount was 199.99 > MB/sec writes, 257.28 MB/sec reads. > > After GlusterFS volume stop/start, SSD tiering performance on > FUSE mount was 35.81 MB/sec writes, 37.33 MB/sec reads. A very > significant reduction in performance. > > Detaching and reattaching the SSD tier restores the good > tiered performance. > > ~ Jeff Byers ~ > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://lists.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users
