On Wed, Jun 02, 2021 at 04:11:06PM +1000, Dave Chinner wrote: > Hi Darrick, > > Can you please pull the perag conversion changes inot the next > for-next compose from the tag below? I pulled most of the original > patchset description into the tag itself, so I won't repeat any of > it here..... Pulled, thanks. --D > > Cheers, > > Dave. > > The following changes since commit d07f6ca923ea0927a1024dfccafc5b53b61cfecc: > > Linux 5.13-rc2 (2021-05-16 15:27:44 -0700) > > are available in the Git repository at: > > git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git tags/xfs-perag-conv-tag > > for you to fetch changes up to 509201163fca3d4d906bd50a5320115d42818748: > > xfs: remove xfs_perag_t (2021-06-02 10:48:51 +1000) > > ---------------------------------------------------------------- > xfs: initial agnumber -> perag conversions for shrink > > If we want to use active references to the perag to be able to gate > shrink removing AGs and hence perags safely, we've got a fair bit of > work to do actually use perags in all the places we need to. > > There's a lot of code that iterates ag numbers and then > looks up perags from that, often multiple times for the same perag > in the one operation. If we want to use reference counted perags for > access control, then we need to convert all these uses to perag > iterators, not agno iterators. > > [Patches 1-4] > > The first step of this is consolidating all the perag management - > init, free, get, put, etc into a common location. THis is spread all > over the place right now, so move it all into libxfs/xfs_ag.[ch]. > This does expose kernel only bits of the perag to libxfs and hence > userspace, so the structures and code is rearranged to minimise the > number of ifdefs that need to be added to the userspace codebase. > The perag iterator in xfs_icache.c is promoted to a first class API > and expanded to the needs of the code as required. > > [Patches 5-10] > > These are the first basic perag iterator conversions and changes to > pass the perag down the stack from those iterators where > appropriate. A lot of this is obvious, simple changes, though in > some places we stop passing the perag down the stack because the > code enters into an as yet unconverted subsystem that still uses raw > AGs. > > [Patches 11-16] > > These replace the agno passed in the btree cursor for per-ag btree > operations with a perag that is passed to the cursor init function. > The cursor takes it's own reference to the perag, and the reference > is dropped when the cursor is deleted. Hence we get reference > coverage for the entire time the cursor is active, even if the code > that initialised the cursor drops it's reference before the cursor > or any of it's children (duplicates) have been deleted. > > The first patch adds the perag infrastructure for the cursor, the > next four patches convert a btree cursor at a time, and the last > removes the agno from the cursor once it is unused. > > [Patches 17-21] > > These patches are a demonstration of the simplifications and > cleanups that come from plumbing the perag through interfaces that > select and then operate on a specific AG. In this case the inode > allocation algorithm does up to three walks across all AGs before it > either allocates an inode or fails. Two of these walks are purely > just to select the AG, and even then it doesn't guarantee inode > allocation success so there's a third walk if the selected AG > allocation fails. > > These patches collapse the selection and allocation into a single > loop, simplifies the error handling because xfs_dir_ialloc() always > returns ENOSPC if no AG was selected for inode allocation or we fail > to allocate an inode in any AG, gets rid of xfs_dir_ialloc() > wrapper, converts inode allocation to run entirely from a single > perag instance, and then factors xfs_dialloc() into a much, much > simpler loop which is easy to understand. > > Hence we end up with the same inode allocation logic, but it only > needs two complete iterations at worst, makes AG selection and > allocation atomic w.r.t. shrink and chops out out over 100 lines of > code from this hot code path. > > [Patch 22] > > Converts the unlink path to pass perags through it. > > There's more conversion work to be done, but this patchset gets > through a large chunk of it in one hit. Most of the iterators are > converted, so once this is solidified we can move on to converting > these to active references for being able to free perags while the > fs is still active. > > ---------------------------------------------------------------- > Dave Chinner (23): > xfs: move xfs_perag_get/put to xfs_ag.[ch] > xfs: prepare for moving perag definitions and support to libxfs > xfs: move perag structure and setup to libxfs/xfs_ag.[ch] > xfs: make for_each_perag... a first class citizen > xfs: convert raw ag walks to use for_each_perag > xfs: convert xfs_iwalk to use perag references > xfs: convert secondary superblock walk to use perags > xfs: pass perags through to the busy extent code > xfs: push perags through the ag reservation callouts > xfs: pass perags around in fsmap data dev functions > xfs: add a perag to the btree cursor > xfs: convert rmap btree cursor to using a perag > xfs: convert refcount btree cursor to use perags > xfs: convert allocbt cursors to use perags > xfs: use perag for ialloc btree cursors > xfs: remove agno from btree cursor > xfs: simplify xfs_dialloc_select_ag() return values > xfs: collapse AG selection for inode allocation > xfs: get rid of xfs_dir_ialloc() > xfs: inode allocation can use a single perag instance > xfs: clean up and simplify xfs_dialloc() > xfs: use perag through unlink processing > xfs: remove xfs_perag_t > > fs/xfs/libxfs/xfs_ag.c | 272 +++++++++++++++++++++++++++++++++++++++++++++++- > fs/xfs/libxfs/xfs_ag.h | 136 ++++++++++++++++++++++++ > fs/xfs/libxfs/xfs_ag_resv.c | 11 +- > fs/xfs/libxfs/xfs_ag_resv.h | 15 +++ > fs/xfs/libxfs/xfs_alloc.c | 110 ++++++++++---------- > fs/xfs/libxfs/xfs_alloc.h | 2 +- > fs/xfs/libxfs/xfs_alloc_btree.c | 31 +++--- > fs/xfs/libxfs/xfs_alloc_btree.h | 9 +- > fs/xfs/libxfs/xfs_attr_leaf.c | 1 + > fs/xfs/libxfs/xfs_bmap.c | 1 + > fs/xfs/libxfs/xfs_btree.c | 15 +-- > fs/xfs/libxfs/xfs_btree.h | 10 +- > fs/xfs/libxfs/xfs_ialloc.c | 608 ++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------------- > fs/xfs/libxfs/xfs_ialloc.h | 40 ++------ > fs/xfs/libxfs/xfs_ialloc_btree.c | 46 +++++---- > fs/xfs/libxfs/xfs_ialloc_btree.h | 13 +-- > fs/xfs/libxfs/xfs_refcount.c | 122 +++++++++++----------- > fs/xfs/libxfs/xfs_refcount.h | 9 +- > fs/xfs/libxfs/xfs_refcount_btree.c | 39 +++---- > fs/xfs/libxfs/xfs_refcount_btree.h | 7 +- > fs/xfs/libxfs/xfs_rmap.c | 147 +++++++++++++------------- > fs/xfs/libxfs/xfs_rmap.h | 6 +- > fs/xfs/libxfs/xfs_rmap_btree.c | 46 ++++----- > fs/xfs/libxfs/xfs_rmap_btree.h | 6 +- > fs/xfs/libxfs/xfs_sb.c | 146 ++------------------------ > fs/xfs/libxfs/xfs_sb.h | 9 -- > fs/xfs/libxfs/xfs_types.c | 4 +- > fs/xfs/scrub/agheader.c | 1 + > fs/xfs/scrub/agheader_repair.c | 33 +++--- > fs/xfs/scrub/alloc.c | 3 +- > fs/xfs/scrub/bmap.c | 21 ++-- > fs/xfs/scrub/common.c | 15 ++- > fs/xfs/scrub/fscounters.c | 42 +++----- > fs/xfs/scrub/health.c | 2 +- > fs/xfs/scrub/ialloc.c | 9 +- > fs/xfs/scrub/refcount.c | 3 +- > fs/xfs/scrub/repair.c | 14 +-- > fs/xfs/scrub/rmap.c | 3 +- > fs/xfs/scrub/trace.c | 3 +- > fs/xfs/xfs_buf.c | 2 +- > fs/xfs/xfs_discard.c | 6 +- > fs/xfs/xfs_extent_busy.c | 33 ++---- > fs/xfs/xfs_extent_busy.h | 7 +- > fs/xfs/xfs_filestream.c | 2 +- > fs/xfs/xfs_fsmap.c | 80 +++++++++------ > fs/xfs/xfs_fsops.c | 8 +- > fs/xfs/xfs_health.c | 6 +- > fs/xfs/xfs_icache.c | 17 +-- > fs/xfs/xfs_inode.c | 202 ++++++++++++++++-------------------- > fs/xfs/xfs_inode.h | 9 +- > fs/xfs/xfs_iwalk.c | 84 +++++++++------ > fs/xfs/xfs_log_recover.c | 56 +++++----- > fs/xfs/xfs_mount.c | 126 +---------------------- > fs/xfs/xfs_mount.h | 110 +------------------- > fs/xfs/xfs_qm.c | 10 +- > fs/xfs/xfs_reflink.c | 13 ++- > fs/xfs/xfs_super.c | 1 + > fs/xfs/xfs_symlink.c | 9 +- > fs/xfs/xfs_trace.c | 2 + > fs/xfs/xfs_trace.h | 4 +- > 60 files changed, 1408 insertions(+), 1389 deletions(-) > > -- > Dave Chinner > david@xxxxxxxxxxxxx
