Hi again (sorry that I reply to my own email, rather than Brian's, but I've only just subscribed to the list), I'll try to address Brian's email in here with fake quotation. Sorry for breaking the threading :( Brian Foster wrote: > Based on the size and consumption of the fs, first thing that comes > to mind is perhaps fragmented use of inode records. E.g., inode > records are spread all over the storage with handfulls of inodes > free here and there, which means individual inode allocation can > take a hit searching for the record with free inodes. I don't think > that explains rm performance though. > It might be interesting to grab a couple perf traces of the touch > and rm commands and see what cpu usage looks like. E.g., 'perf > record -g touch <file>,' 'perf report -g,' etc. I've attached both perf outputs and after reviewing them briefly I think slowness is caused by different means, i.e. only the touch one is in xfs' territory. > 30265117 xfs: Fix rounding in xfs_alloc_fix_len() > > That originally went into 3.16 and I don't see it in the 3.14 stable > branch. Did xfs_repair actually report anything wrong? Nope, only displayed all the stages, but nothing was fixed. > It seems like you have sufficiently large and available free > space. That said, it's fairly common for filesytems to naturally > drop in performance as free space becomes more limited. E.g., I > think it's common practice to avoid regular usage while over 80% > used if performance is a major concern. Also, I doubt xfs_fsr will > do much to affect inode allocation performance, but I could be > wrong. Yes, we should have monitored that mount point rather than /tmp which we did when bad things happened(TM). Given that we have a high fragmentation of directories, would xfs_fsr help here at all? Regarding v5, currently we are copying data off that disk and will create it anew with -m crc=1,finobt=1 on a recent 3.18 kernel. Apart from that I don't know much we can further do to safe-guard us against this happening again (well kepp it below 80% all the time as well). Thanks a lot for the remarks! cheers Carsten -- Dr. Carsten Aulbert - Max Planck Institute for Gravitational Physics Callinstrasse 38, 30167 Hannover, Germany phone/fax: +49 511 762-17185 / -17193 https://wiki.atlas.aei.uni-hannover.de/foswiki/bin/view/ATLAS/WebHome
Attachment:
rm.perf.data.gz
Description: application/gzip
Attachment:
touch.perf.data.gz
Description: application/gzip
_______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
