Hi all, This patchset dramatically reduces the runtime of the FITRIM calls made during phase 8 of xfs_scrub. It turns out that phase 8 can really get bogged down if the free space contains a large number of very small extents. In these cases, the runtime can increase by an order of magnitude to free less than 1% of the free space. This is not worth the time, since we're spending a lot of time to do very little work. The FITRIM ioctl allows us to specify a minimum extent length, so we can use statistical methods to compute a minlen parameter. It turns out xfs_db/spaceman already have the code needed to create histograms of free space extent lengths. We add the ability to compute a CDF of the extent lengths, which make it easy to pick a minimum length corresponding to 99% of the free space. In most cases, this results in dramatic reductions in phase 8 runtime. Hence, move the histogram code to libfrog, and wire up xfs_scrub, since phase 7 already walks the fsmap. We also add a new -o suboption to xfs_scrub so that people who /do/ want to examine every free extent can do so. If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below. This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome. --D xfsprogs git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=scrub-fstrim-minlen-freesp-histogram fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=scrub-fstrim-minlen-freesp-histogram --- Commits in this patchset: * libfrog: hoist free space histogram code * libfrog: print wider columns for free space histogram * libfrog: print cdf of free space buckets * xfs_scrub: don't close stdout when closing the progress bar * xfs_scrub: remove pointless spacemap.c arguments * xfs_scrub: collect free space histograms during phase 7 * xfs_scrub: tune fstrim minlen parameter based on free space histograms --- db/freesp.c | 89 ++++------------ libfrog/Makefile | 2 libfrog/histogram.c | 270 ++++++++++++++++++++++++++++++++++++++++++++++++++ libfrog/histogram.h | 78 ++++++++++++++ man/man8/xfs_scrub.8 | 16 +++ scrub/phase7.c | 47 ++++++++- scrub/phase8.c | 91 ++++++++++++++++- scrub/spacemap.c | 11 +- scrub/vfs.c | 4 + scrub/vfs.h | 2 scrub/xfs_scrub.c | 45 ++++++++ scrub/xfs_scrub.h | 16 +++ spaceman/freesp.c | 99 ++++++------------ 13 files changed, 620 insertions(+), 150 deletions(-) create mode 100644 libfrog/histogram.c create mode 100644 libfrog/histogram.h
