[PIEDAY DELUGE 1/2] xfs: all pending online scrub improvements

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Dave et. al.,

As promised, here's the rebase against 6.3-rc2 of the online fsck design
documentation and all pending scrub fixes.  I've included all of
Allison's feedback about the design doc, along with the various bits
that Dave commented on in #xfs.

(This isn't a true deluge, since I'm only posting this notice, not the
entire patchset.)

--D

The following changes since commit eeac8ede17557680855031c6f305ece2378af326:

Linux 6.3-rc2 (2023-03-12 16:36:44 -0700)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git tags/scrub-strengthen-rmap-checking_2023-03-14

for you to read changes up to 5023db0fd78213c4d9a5e6c027c6f9b55dc1fdc9:

xfs: cross-reference rmap records with refcount btrees (2023-03-14 17:44:07 -0700)

----------------------------------------------------------------
xfs: strengthen rmapbt scrubbing

This series strengthens space allocation record cross referencing by
using AG block bitmaps to compute the difference between space used
according to the rmap records and the primary metadata, and reports
cross-referencing errors for any discrepancies.

Signed-off-by: Darrick J. Wong <djwong@xxxxxxxxxx>

----------------------------------------------------------------
Darrick J. Wong (97):
xfs: try to idiot-proof the allocators
xfs: recheck appropriateness of map_shared lock
xfs: document the motivation for online fsck design
xfs: document the general theory underlying online fsck design
xfs: document the testing plan for online fsck
xfs: document the user interface for online fsck
xfs: document the filesystem metadata checking strategy
xfs: document how online fsck deals with eventual consistency
xfs: document pageable kernel memory
xfs: document btree bulk loading
xfs: document online file metadata repair code
xfs: document full filesystem scans for online fsck
xfs: document metadata file repair
xfs: document directory tree repairs
xfs: document the userspace fsck driver program
xfs: document future directions of online fsck
xfs: give xfs_bmap_intent its own perag reference
xfs: pass per-ag references to xfs_free_extent
xfs: give xfs_extfree_intent its own perag reference
xfs: give xfs_rmap_intent its own perag reference
xfs: give xfs_refcount_intent its own perag reference
xfs: create traced helper to get extra perag references
xfs: fix author and spdx headers on scrub/ files
xfs: update copyright years for scrub/ files
xfs: add a tracepoint to report incorrect extent refcounts
xfs: allow queued AG intents to drain before scrubbing
xfs: clean up scrub context if scrub setup returns -EDEADLOCK
xfs: minimize overhead of drain wakeups by using jump labels
xfs: scrub should use ECHRNG to signal that the drain is needed
xfs: standardize ondisk to incore conversion for free space btrees
xfs: standardize ondisk to incore conversion for inode btrees
xfs: standardize ondisk to incore conversion for refcount btrees
xfs: return a failure address from xfs_rmap_irec_offset_unpack
xfs: standardize ondisk to incore conversion for rmap btrees
xfs: standardize ondisk to incore conversion for bmap btrees
xfs: complain about bad records in query_range helpers
xfs: complain about bad file mapping records in the ondisk bmbt
xfs: hoist rmap record flag checks from scrub
xfs: hoist rmap record flag checks from scrub
xfs: hoist inode record alignment checks from scrub
xfs: fix rm_offset flag handling in rmap keys
xfs: detect unwritten bit set in rmapbt node block keys
xfs: check btree keys reflect the child block
xfs: always scrub record/key order of interior records
xfs: refactor converting btree irec to btree key
xfs: refactor ->diff_two_keys callsites
xfs: replace xfs_btree_has_record with a general keyspace scanner
xfs: implement masked btree key comparisons for _has_records scans
xfs: check the reference counts of gaps in the refcount btree
xfs: ensure that all metadata and data blocks are not cow staging extents
xfs: remove pointless shadow variable from xfs_difree_inobt
xfs: clean up broken eearly-exit code in the inode btree scrubber
xfs: directly cross-reference the inode btrees with each other
xfs: convert xfs_ialloc_has_inodes_at_extent to return keyfill scan results
xfs: teach scrub to check for sole ownership of metadata objects
xfs: ensure that single-owner file blocks are not owned by others
xfs: streamline the directory iteration code for scrub
xfs: xfs_iget in the directory scrubber needs to use UNTRUSTED
xfs: always check the existence of a dirent's child inode
xfs: remove xchk_parent_count_parent_dentries
xfs: simplify xchk_parent_validate
xfs: fix parent pointer scrub racing with subdirectory reparenting
xfs: manage inode DONTCACHE status at irele time
xfs: fix an inode lookup race in xchk_get_inode
xfs: rename xchk_get_inode -> xchk_iget_for_scrubbing
xfs: retain the AGI when we can't iget an inode to scrub the core
xfs: don't take the MMAPLOCK when scrubbing file metadata
xfs: change bmap scrubber to store the previous mapping
xfs: accumulate iextent records when checking bmap
xfs: split xchk_bmap_xref_rmap into two functions
xfs: alert the user about data/attr fork mappings that could be merged
xfs: split the xchk_bmap_check_rmaps into a predicate
xfs: don't call xchk_bmap_check_rmaps for btree-format file forks
xfs: flag free space btree records that could be merged
xfs: flag refcount btree records that could be merged
xfs: check overlapping rmap btree records
xfs: check for reverse mapping records that could be merged
xfs: xattr scrub should ensure one namespace bit per name
xfs: don't shadow @leaf in xchk_xattr_block
xfs: remove unnecessary dstmap in xattr scrubber
xfs: split freemap from xchk_xattr_buf.buf
xfs: split usedmap from xchk_xattr_buf.buf
xfs: split valuebuf from xchk_xattr_buf.buf
xfs: remove flags argument from xchk_setup_xattr_buf
xfs: move xattr scrub buffer allocation to top level function
xfs: check used space of shortform xattr structures
xfs: clean up xattr scrub initialization
xfs: only allocate free space bitmap for xattr scrub if needed
xfs: don't load local xattr values during scrub
xfs: remove the for_each_xbitmap_ helpers
xfs: drop the _safe behavior from the xbitmap foreach macro
xfs: convert xbitmap to interval tree
xfs: introduce bitmap type for AG blocks
xfs: cross-reference rmap records with ag btrees
xfs: cross-reference rmap records with free space btrees
xfs: cross-reference rmap records with inode btrees
xfs: cross-reference rmap records with refcount btrees

Documentation/filesystems/index.rst                |    1 +
.../filesystems/xfs-online-fsck-design.rst         | 5315 ++++++++++++++++++++
.../filesystems/xfs-self-describing-metadata.rst   |    1 +
fs/xfs/Kconfig                                     |    5 +
fs/xfs/Makefile                                    |    5 +-
fs/xfs/libxfs/xfs_ag.c                             |   23 +-
fs/xfs/libxfs/xfs_ag.h                             |    9 +
fs/xfs/libxfs/xfs_alloc.c                          |  128 +-
fs/xfs/libxfs/xfs_alloc.h                          |   22 +-
fs/xfs/libxfs/xfs_alloc_btree.c                    |   32 +-
fs/xfs/libxfs/xfs_bmap.c                           |   32 +-
fs/xfs/libxfs/xfs_bmap.h                           |    8 +-
fs/xfs/libxfs/xfs_bmap_btree.c                     |   19 +-
fs/xfs/libxfs/xfs_btree.c                          |  208 +-
fs/xfs/libxfs/xfs_btree.h                          |  141 +-
fs/xfs/libxfs/xfs_defer.c                          |    6 +-
fs/xfs/libxfs/xfs_ialloc.c                         |  165 +-
fs/xfs/libxfs/xfs_ialloc.h                         |    7 +-
fs/xfs/libxfs/xfs_ialloc_btree.c                   |   35 +-
fs/xfs/libxfs/xfs_ialloc_btree.h                   |    2 +-
fs/xfs/libxfs/xfs_inode_fork.c                     |    3 +-
fs/xfs/libxfs/xfs_refcount.c                       |  117 +-
fs/xfs/libxfs/xfs_refcount.h                       |   10 +-
fs/xfs/libxfs/xfs_refcount_btree.c                 |   31 +-
fs/xfs/libxfs/xfs_rmap.c                           |  364 +-
fs/xfs/libxfs/xfs_rmap.h                           |   38 +-
fs/xfs/libxfs/xfs_rmap_btree.c                     |  102 +-
fs/xfs/libxfs/xfs_types.h                          |   12 +
fs/xfs/scrub/agheader.c                            |   30 +-
fs/xfs/scrub/agheader_repair.c                     |  105 +-
fs/xfs/scrub/alloc.c                               |   69 +-
fs/xfs/scrub/attr.c                                |  312 +-
fs/xfs/scrub/attr.h                                |   64 +-
fs/xfs/scrub/bitmap.c                              |  428 +-
fs/xfs/scrub/bitmap.h                              |  109 +-
fs/xfs/scrub/bmap.c                                |  426 +-
fs/xfs/scrub/btree.c                               |  102 +-
fs/xfs/scrub/btree.h                               |   16 +-
fs/xfs/scrub/common.c                              |  454 +-
fs/xfs/scrub/common.h                              |   32 +-
fs/xfs/scrub/dabtree.c                             |    7 +-
fs/xfs/scrub/dabtree.h                             |    6 +-
fs/xfs/scrub/dir.c                                 |  239 +-
fs/xfs/scrub/fscounters.c                          |   11 +-
fs/xfs/scrub/health.c                              |    8 +-
fs/xfs/scrub/health.h                              |    6 +-
fs/xfs/scrub/ialloc.c                              |  304 +-
fs/xfs/scrub/inode.c                               |  197 +-
fs/xfs/scrub/parent.c                              |  314 +-
fs/xfs/scrub/quota.c                               |    9 +-
fs/xfs/scrub/readdir.c                             |  375 ++
fs/xfs/scrub/readdir.h                             |   19 +
fs/xfs/scrub/refcount.c                            |  193 +-
fs/xfs/scrub/repair.c                              |  112 +-
fs/xfs/scrub/repair.h                              |    7 +-
fs/xfs/scrub/rmap.c                                |  586 ++-
fs/xfs/scrub/rtbitmap.c                            |    6 +-
fs/xfs/scrub/scrub.c                               |   74 +-
fs/xfs/scrub/scrub.h                               |   32 +-
fs/xfs/scrub/symlink.c                             |    6 +-
fs/xfs/scrub/trace.c                               |    6 +-
fs/xfs/scrub/trace.h                               |   75 +-
fs/xfs/scrub/xfs_scrub.h                           |    6 +-
fs/xfs/xfs_bmap_item.c                             |   39 +-
fs/xfs/xfs_drain.c                                 |  135 +
fs/xfs/xfs_drain.h                                 |   80 +
fs/xfs/xfs_extfree_item.c                          |   56 +-
fs/xfs/xfs_icache.c                                |    3 +-
fs/xfs/xfs_icache.h                                |   11 +-
fs/xfs/xfs_inode.c                                 |   29 +
fs/xfs/xfs_iunlink_item.c                          |    4 +-
fs/xfs/xfs_iwalk.c                                 |    5 +-
fs/xfs/xfs_linux.h                                 |    1 +
fs/xfs/xfs_refcount_item.c                         |   38 +-
fs/xfs/xfs_rmap_item.c                             |   34 +-
fs/xfs/xfs_trace.h                                 |   72 +
76 files changed, 10300 insertions(+), 1793 deletions(-)
create mode 100644 Documentation/filesystems/xfs-online-fsck-design.rst
create mode 100644 fs/xfs/scrub/readdir.c
create mode 100644 fs/xfs/scrub/readdir.h
create mode 100644 fs/xfs/xfs_drain.c
create mode 100644 fs/xfs/xfs_drain.h



[Index of Archives]     [XFS Filesystem Development (older mail)]     [Linux Filesystem Development]     [Linux Audio Users]     [Yosemite Trails]     [Linux Kernel]     [Linux RAID]     [Linux SCSI]


  Powered by Linux