Hi Linus,
Please pull this branch containing new code for 5.14. Most of the work
this cycle has been on refactoring various parts of the codebase. The
biggest non-cleanup changes are (1) reducing the number of cache flushes
sent when writing the log; (2) a substantial number of log recovery
fixes; and (3) I started accepting pull requests from contributors if
the commits in their branches match what's been sent to the list.
For a week or so I /had/ staged a major cleanup of the logging code from
Dave Chinner, but it exposed so many lurking bugs in other parts of the
logging and log recovery code that I decided to defer that patchset
until we can address those latent bugs.
Larger cleanups this time include walking the incore inode cache (me)
and rework of the extended attribute code (Allison) to prepare it for
adding logged xattr updates (and directory tree parent pointers) in
future releases.
The branch merges cleanly against upstream as of a few minutes ago.
Please let me know if anything else strange happens during the merge
process. The merge commits I made seem stable enough, but as it's the
first time I've ever accepted a pull request, we'd all be open to
feedback for improvements for next time.
--D
The following changes since commit 8124c8a6b35386f73523d27eacb71b5364a68c4c:
Linux 5.13-rc4 (2021-05-30 11:58:25 -1000)
are available in the Git repository at:
git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git tags/xfs-5.14-merge-6
for you to fetch changes up to 1effb72a8179a02c2dd8a268454ccf50bf68aa50:
xfs: don't wait on future iclogs when pushing the CIL (2021-06-25 14:02:02 -0700)
----------------------------------------------------------------
New code for 5.14:
- Refactor the buffer cache to use bulk page allocation
- Convert agnumber-based AG iteration to walk per-AG structures
- Clean up some unit conversions and other code warts
- Reduce spinlock contention in the directio fastpath
- Collapse all the inode cache walks into a single function
- Remove indirect function calls from the inode cache walk code
- Dramatically reduce the number of cache flushes sent when writing log
buffers
- Preserve inode sickness reports for longer
- Rename xfs_eofblocks since it controls inode cache walks
- Refactor the extended attribute code to prepare it for the addition
of log intent items to make xattrs fully transactional
- A few fixes to earlier large patchsets
- Log recovery fixes so that we don't accidentally mark the log clean
when log intent recovery fails
- Fix some latent SOB errors
- Clean up shutdown messages that get logged to dmesg
- Fix a regression in the online shrink code
- Fix a UAF in the buffer logging code if the fs goes offline
- Fix uninitialized error variables
- Fix a UAF in the CIL when commited log item callbacks race with a
shutdown
- Fix a bug where the CIL could hang trying to push part of the log ring
buffer that hasn't been filled yet
----------------------------------------------------------------
Allison Henderson (15):
xfs: Reverse apply 72b97ea40d
xfs: Add xfs_attr_node_remove_name
xfs: Refactor xfs_attr_set_shortform
xfs: Separate xfs_attr_node_addname and xfs_attr_node_addname_clear_incomplete
xfs: Add helper xfs_attr_node_addname_find_attr
xfs: Hoist xfs_attr_node_addname
xfs: Hoist xfs_attr_leaf_addname
xfs: Hoist node transaction handling
xfs: Add delay ready attr remove routines
xfs: Add delay ready attr set routines
xfs: Remove xfs_attr_rmtval_set
xfs: Clean up xfs_attr_node_addname_clear_incomplete
xfs: Fix default ASSERT in xfs_attr_set_iter
xfs: Make attr name schemes consistent
xfs: Initialize error in xfs_attr_remove_iter
Brian Foster (2):
xfs: hold buffer across unpin and potential shutdown processing
xfs: remove dead stale buf unpin handling code
Christoph Hellwig (4):
xfs: mark xfs_bmap_set_attrforkoff static
xfs: remove ->b_offset handling for page backed buffers
xfs: simplify the b_page_count calculation
xfs: cleanup error handling in xfs_buf_get_map
Darrick J. Wong (38):
xfs: clean up open-coded fs block unit conversions
xfs: remove unnecessary shifts
xfs: move the quotaoff dqrele inode walk into xfs_icache.c
xfs: detach inode dquots at the end of inactivation
xfs: move the inode walk functions further down
xfs: rename xfs_inode_walk functions to xfs_icwalk
xfs: pass the goal of the incore inode walk to xfs_inode_walk()
xfs: separate the dqrele_all inode grab logic from xfs_inode_walk_ag_grab
xfs: move xfs_inew_wait call into xfs_dqrele_inode
xfs: remove iter_flags parameter from xfs_inode_walk_*
xfs: remove indirect calls from xfs_inode_walk{,_ag}
xfs: clean up inode state flag tests in xfs_blockgc_igrab
xfs: make the icwalk processing functions clean up the grab state
xfs: fix radix tree tag signs
xfs: pass struct xfs_eofblocks to the inode scan callback
xfs: merge xfs_reclaim_inodes_ag into xfs_inode_walk_ag
xfs: refactor per-AG inode tagging functions
Merge tag 'xfs-buf-bulk-alloc-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs into xfs-5.14-merge2
Merge tag 'xfs-perag-conv-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs into xfs-5.14-merge2
Merge tag 'unit-conversion-cleanups-5.14_2021-06-03' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-5.14-merge2
Merge tag 'assorted-fixes-5.14-1_2021-06-03' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-5.14-merge2
Merge tag 'inode-walk-cleanups-5.14_2021-06-03' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-5.14-merge2
xfs: only reset incore inode health state flags when reclaiming an inode
xfs: drop IDONTCACHE on inodes when we mark them sick
xfs: change the prefix of XFS_EOF_FLAGS_* to XFS_ICWALK_FLAG_
xfs: selectively keep sick inodes in memory
xfs: rename struct xfs_eofblocks to xfs_icwalk
Merge tag 'fix-inode-health-reports-5.14_2021-06-08' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-5.14-merge2
Merge tag 'rename-eofblocks-5.14_2021-06-08' of https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux into xfs-5.14-merge2
Merge tag 'xfs-delay-ready-attrs-v20.1' of https://github.com/allisonhenderson/xfs_work into xfs-5.14-merge4
xfs: refactor the inode recycling code
xfs: separate primary inode selection criteria in xfs_iget_cache_hit
xfs: fix type mismatches in the inode reclaim functions
xfs: print name of function causing fs shutdown instead of hex pointer
xfs: shorten the shutdown messages to a single line
xfs: fix log intent recovery ENOSPC shutdowns when inactivating inodes
xfs: force the log offline when log intent item recovery fails
xfs: fix endianness issue in xfs_ag_shrink_space
Dave Chinner (47):
xfs: split up xfs_buf_allocate_memory
xfs: use xfs_buf_alloc_pages for uncached buffers
xfs: use alloc_pages_bulk_array() for buffers
xfs: merge _xfs_buf_get_pages()
xfs: move page freeing into _xfs_buf_free_pages()
xfs: move xfs_perag_get/put to xfs_ag.[ch]
xfs: prepare for moving perag definitions and support to libxfs
xfs: move perag structure and setup to libxfs/xfs_ag.[ch]
xfs: make for_each_perag... a first class citizen
xfs: convert raw ag walks to use for_each_perag
xfs: convert xfs_iwalk to use perag references
xfs: convert secondary superblock walk to use perags
xfs: pass perags through to the busy extent code
xfs: push perags through the ag reservation callouts
xfs: pass perags around in fsmap data dev functions
xfs: add a perag to the btree cursor
xfs: convert rmap btree cursor to using a perag
xfs: convert refcount btree cursor to use perags
xfs: convert allocbt cursors to use perags
xfs: use perag for ialloc btree cursors
xfs: remove agno from btree cursor
xfs: simplify xfs_dialloc_select_ag() return values
xfs: collapse AG selection for inode allocation
xfs: get rid of xfs_dir_ialloc()
xfs: inode allocation can use a single perag instance
xfs: clean up and simplify xfs_dialloc()
xfs: use perag through unlink processing
xfs: remove xfs_perag_t
xfs: don't take a spinlock unconditionally in the DIO fastpath
xfs: get rid of xb_to_gfp()
xfs: merge xfs_buf_allocate_memory
xfs: drop the AGI being passed to xfs_check_agi_freecount
xfs: perag may be null in xfs_imap()
xfs: log stripe roundoff is a property of the log
xfs: separate CIL commit record IO
xfs: remove xfs_blkdev_issue_flush
xfs: async blkdev cache flush
xfs: CIL checkpoint flushes caches unconditionally
xfs: remove need_start_rec parameter from xlog_write()
xfs: journal IO cache flush reductions
xfs: Fix CIL throttle hang when CIL space used going backwards
xfs: xfs_log_force_lsn isn't passed a LSN
xfs: add iclog state trace events
xfs: don't nest icloglock inside ic_callback_lock
xfs: remove callback dequeue loop from xlog_state_do_iclog_callbacks
xfs: Fix a CIL UAF by getting get rid of the iclog callback lock
xfs: don't wait on future iclogs when pushing the CIL
Geert Uytterhoeven (1):
xfs: Fix 64-bit division on 32-bit in xlog_state_switch_iclogs()
Jiapeng Chong (1):
xfs: Remove redundant assignment to busy
Shaokun Zhang (2):
xfs: sort variable alphabetically to avoid repeated declaration
xfs: remove redundant initialization of variable error
fs/xfs/libxfs/xfs_ag.c | 280 ++++++++-
fs/xfs/libxfs/xfs_ag.h | 136 +++++
fs/xfs/libxfs/xfs_ag_resv.c | 11 +-
fs/xfs/libxfs/xfs_ag_resv.h | 15 +
fs/xfs/libxfs/xfs_alloc.c | 111 ++--
fs/xfs/libxfs/xfs_alloc.h | 2 +-
fs/xfs/libxfs/xfs_alloc_btree.c | 31 +-
fs/xfs/libxfs/xfs_alloc_btree.h | 9 +-
fs/xfs/libxfs/xfs_attr.c | 956 +++++++++++++++++------------
fs/xfs/libxfs/xfs_attr.h | 403 +++++++++++++
fs/xfs/libxfs/xfs_attr_leaf.c | 5 +-
fs/xfs/libxfs/xfs_attr_leaf.h | 2 +-
fs/xfs/libxfs/xfs_attr_remote.c | 167 +++---
fs/xfs/libxfs/xfs_attr_remote.h | 8 +-
fs/xfs/libxfs/xfs_bmap.c | 3 +-
fs/xfs/libxfs/xfs_bmap.h | 1 -
fs/xfs/libxfs/xfs_btree.c | 15 +-
fs/xfs/libxfs/xfs_btree.h | 10 +-
fs/xfs/libxfs/xfs_ialloc.c | 641 ++++++++++----------
fs/xfs/libxfs/xfs_ialloc.h | 40 +-
fs/xfs/libxfs/xfs_ialloc_btree.c | 46 +-
fs/xfs/libxfs/xfs_ialloc_btree.h | 13 +-
fs/xfs/libxfs/xfs_inode_buf.c | 2 +-
fs/xfs/libxfs/xfs_log_format.h | 3 -
fs/xfs/libxfs/xfs_refcount.c | 122 ++--
fs/xfs/libxfs/xfs_refcount.h | 9 +-
fs/xfs/libxfs/xfs_refcount_btree.c | 39 +-
fs/xfs/libxfs/xfs_refcount_btree.h | 7 +-
fs/xfs/libxfs/xfs_rmap.c | 147 ++---
fs/xfs/libxfs/xfs_rmap.h | 6 +-
fs/xfs/libxfs/xfs_rmap_btree.c | 46 +-
fs/xfs/libxfs/xfs_rmap_btree.h | 6 +-
fs/xfs/libxfs/xfs_sb.c | 146 +----
fs/xfs/libxfs/xfs_sb.h | 9 -
fs/xfs/libxfs/xfs_shared.h | 20 +-
fs/xfs/libxfs/xfs_types.c | 4 +-
fs/xfs/libxfs/xfs_types.h | 1 +
fs/xfs/scrub/agheader.c | 1 +
fs/xfs/scrub/agheader_repair.c | 33 +-
fs/xfs/scrub/alloc.c | 3 +-
fs/xfs/scrub/bmap.c | 21 +-
fs/xfs/scrub/common.c | 15 +-
fs/xfs/scrub/fscounters.c | 42 +-
fs/xfs/scrub/health.c | 2 +-
fs/xfs/scrub/ialloc.c | 9 +-
fs/xfs/scrub/refcount.c | 3 +-
fs/xfs/scrub/repair.c | 14 +-
fs/xfs/scrub/rmap.c | 3 +-
fs/xfs/scrub/trace.c | 3 +-
fs/xfs/xfs_attr_inactive.c | 2 +-
fs/xfs/xfs_bio_io.c | 35 ++
fs/xfs/xfs_bmap_util.c | 6 +-
fs/xfs/xfs_buf.c | 311 ++++------
fs/xfs/xfs_buf.h | 3 +-
fs/xfs/xfs_buf_item.c | 97 ++-
fs/xfs/xfs_discard.c | 6 +-
fs/xfs/xfs_dquot_item.c | 2 +-
fs/xfs/xfs_extent_busy.c | 35 +-
fs/xfs/xfs_extent_busy.h | 7 +-
fs/xfs/xfs_file.c | 70 ++-
fs/xfs/xfs_filestream.c | 2 +-
fs/xfs/xfs_fsmap.c | 80 ++-
fs/xfs/xfs_fsops.c | 24 +-
fs/xfs/xfs_health.c | 15 +-
fs/xfs/xfs_icache.c | 1162 ++++++++++++++++++++----------------
fs/xfs/xfs_icache.h | 58 +-
fs/xfs/xfs_inode.c | 234 ++++----
fs/xfs/xfs_inode.h | 9 +-
fs/xfs/xfs_inode_item.c | 18 +-
fs/xfs/xfs_inode_item.h | 2 +-
fs/xfs/xfs_ioctl.c | 41 +-
fs/xfs/xfs_iops.c | 4 +-
fs/xfs/xfs_iwalk.c | 84 ++-
fs/xfs/xfs_linux.h | 2 +
fs/xfs/xfs_log.c | 273 ++++-----
fs/xfs/xfs_log.h | 5 +-
fs/xfs/xfs_log_cil.c | 138 +++--
fs/xfs/xfs_log_priv.h | 41 +-
fs/xfs/xfs_log_recover.c | 61 +-
fs/xfs/xfs_mount.c | 136 +----
fs/xfs/xfs_mount.h | 110 +---
fs/xfs/xfs_qm.c | 10 +-
fs/xfs/xfs_qm.h | 1 -
fs/xfs/xfs_qm_syscalls.c | 54 +-
fs/xfs/xfs_reflink.c | 13 +-
fs/xfs/xfs_super.c | 10 +-
fs/xfs/xfs_super.h | 1 -
fs/xfs/xfs_symlink.c | 9 +-
fs/xfs/xfs_trace.c | 2 +
fs/xfs/xfs_trace.h | 115 +++-
fs/xfs/xfs_trans.c | 6 +-
fs/xfs/xfs_trans.h | 4 +-
92 files changed, 3855 insertions(+), 3064 deletions(-)
