> > Some time ago I had a similar performance problem (with 3.4 if I remember correctly): a just created volume started to work fine, but after some time using it performance was worse. Removing all files from the volume didn't improve the performance again. I guess my problem is a little better depending on how you look at it. If I date the data from the volume, the performance goes back to that of an empty volume. I don't have to delete the .glusterfs entries to regain my performance. I only have to delete the data from the mount point. David > > The only way I had to recover a performance similar to the initial one without recreating the volume was to remove all volume contents and also delete all 256 .glusterfs/xx/ directories from all bricks. > > The backend filesystem was XFS. > > Could you try if this is the same case ? > > Xavi > >> On 02/11/2015 12:22 PM, David F. Robinson wrote: >> Don't think it is the underlying file system. /data/brickxx is the underlying xfs. Performance to this is fine. When I created a volume it just puts the data in /data/brick/test2. The underlying filesystem shouldn't know/care that it is in a new directory. >> >> Also, if I create a /data/brick/test2 volume and put data on it, it gets slow in gluster. But, writing to /data/brick is still fine. And, after test2 gets slow, I can create a /data/test3 volume that is empty and its speed is fine. >> >> My knowledge is admittedly very limited here, but I don't see how it could be the underlying filesystem if the slowdown only occurs on the gluster mount and not on the underlying xfs filesystem. >> >> David (Sent from mobile) >> >> =============================== >> David F. Robinson, Ph.D. >> President - Corvid Technologies >> 704.799.6944 x101 [office] >> 704.252.1310 [cell] >> 704.799.7974 [fax] >> David.Robinson@xxxxxxxxxxxxx >> http://www.corvidtechnologies.com >> >>>> On Feb 11, 2015, at 12:18 AM, Justin Clift <justin@xxxxxxxxxxx> wrote: >>>> >>>> On 11 Feb 2015, at 03:06, Shyam <srangana@xxxxxxxxxx> wrote: >>>> <snip> >>>> 2) We ran an strace of tar and also collected io-stats outputs from these volumes, both show that create and mkdir is slower on slow as compared to the fast volume. This seems to be the overall reason for slowness >>> >>> Any idea's on "why" the create and mkdir is slower? >>> >>> Wondering if it's a case of underlying filesystem parameters (for the bricks) >>> + maybe physical storage structure having become badly optimised over time. >>> eg if its on spinning rust, not ssd, and sector placement is now bad >>> >>> Any idea if there are tools that can analyse this kind of thing? eg meta >>> data placement / fragmentation / on a drive for XFS/ext4 >>> >>> + Justin >>> >>> -- >>> GlusterFS - http://www.gluster.org >>> >>> An open source, distributed file system scaling to several >>> petabytes, and handling thousands of clients. >>> >>> My personal twitter: twitter.com/realjustinclift >>> >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel@xxxxxxxxxxx >> http://www.gluster.org/mailman/listinfo/gluster-devel >> _______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
