Hi Linus, Here's the first round of patches for XFS for 4.18. New features this cycle include the ability to relabel mounted filesystems, support for fallocated swapfiles, and using FUA for pure data O_DSYNC directio writes. With this cycle we begin to integrate online filesystem repair and refactor the growfs code in preparation for eventual subvolume support, though the road ahead for both features is quite long. There are also numerous refactorings of the iomap code to remove unnecessary log overhead, to disentangle some of the quota code, and to prepare for buffer head removal in a future upstream kernel. Metadata validation continues to improve, both in the hot path veifiers and the online filesystem check code. I anticipate sending a second pull request in a few days with more metadata validation improvements. This series has been run through a full xfstests run over the weekend and through a quick xfstests run against this morning's master, with no major failures reported. There's a merge conflict between one of the iomap refactorings and the gfs2 changes in your master branch. The file in question is fs/gfs2/bmap.c. I resolved the conflict and pushed the result as a branch[1] into my git tree, and the resolved file should look like[2]. Let me know if things get hairy enough that you want me to take another look at that. --D [1] https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=xfs-4.18-merge-conflicts [2] https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/diff/fs/gfs2/bmap.c?h=xfs-4.18-merge-conflicts&id=54195b796f10860d1778dc09a9e9b1543a70301e The following changes since commit 75bc37fefc4471e718ba8e651aa74673d4e0a9eb: Linux 4.17-rc4 (2018-05-06 16:57:38 -1000) are available in the Git repository at: git://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git tags/xfs-4.18-merge-3 for you to fetch changes up to afd9d6a1df75807684fa40dab77c52e104e5c74b: fs: use ->is_partially_uptodate in page_cache_seek_hole_data (2018-06-01 18:37:33 -0700) ---------------------------------------------------------------- Changes for 4.18: - Strengthen inode number and structure validation when allocating inodes. - Reduce pointless buffer allocations during cache miss - Use FUA for pure data O_DSYNC directio writes - Various iomap refactorings - Strengthen quota metadata verification to avoid unfixable broken quota - Make AGFL block freeing a deferred operation to avoid blowing out transaction reservations when running complex operations - Get rid of the log item descriptors to reduce log overhead - Fix various reflink bugs where inodes were double-joined to transactions - Don't issue discards when trimming unwritten extents - Refactor incore dquot initialization and retrieval interfaces - Fix some locking problmes in the quota scrub code - Strengthen btree structure checks in scrub code - Rewrite swapfile activation to use iomap and support unwritten extents - Make scrub exit to userspace sooner when corruptions or cross-referencing problems are found - Make scrub invoke the data fork scrubber directly on metadata inodes - Don't do background reclamation of post-eof and cow blocks when the fs is suspended - Fix secondary superblock buffer lifespan hinting - Refactor growfs to use table-dispatched functions instead of long stringy functions - Move growfs code to libxfs - Implement online fs label getting and setting - Introduce online filesystem repair (in a very limited capacity) - Fix unit conversion problems in the realtime freemap iteration functions - Various refactorings and cleanups in preparation to remove buffer heads in a future release - Reimplement the old bmap call with iomap - Remove direct buffer head accesses from seek hole/data - Various bug fixes ---------------------------------------------------------------- Brian Foster (10): xfs: create agfl block free helper function xfs: defer agfl block frees when dfops is available xfs: defer agfl block frees from deferred ops processing context xfs: defer agfl frees from inode inactivation xfs: defer frees from common inode allocation paths xfs: defer agfl frees from directory op transactions xfs: add bmapi nodiscard flag xfs: skip online discard during eofblocks trims xfs: don't discard on free of unwritten extents xfs: factor out nodiscard helpers Christoph Hellwig (14): block: add a lower-level bio_add_page interface mm: give the 'ret' variable a better name __do_page_cache_readahead mm: return an unsigned int from __do_page_cache_readahead mm: split ->readpages calls to avoid non-contiguous pages lists iomap: inline data should be an iomap type, not a flag iomap: fix the comment describing IOMAP_NOWAIT iomap: move IOMAP_F_BOUNDARY to gfs2 iomap: use __bio_add_page in iomap_dio_zero iomap: add a iomap_sector helper iomap: add an iomap-based bmap implementation xfs: use iomap_bmap fs: move page_cache_seek_hole_data to iomap.c fs: remove the buffer_unwritten check in page_seek_hole_data fs: use ->is_partially_uptodate in page_cache_seek_hole_data Darrick J. Wong (54): xfs: bmap debugging should never panic the system xfs: add missing rmap error return xfs: release new dquot buffer on defer_finish error xfs: don't spray logs when dquot flush/purge fail xfs: refactor XFS_QMOPT_DQNEXT out of existence xfs: refactor dquot cache handling xfs: delegate dqget input checks to helper function xfs: remove unnecessary xfs_qm_dqattach parameter xfs: split out dqget for inodes from regular dqget xfs: fetch dquots directly during quotacheck xfs: refactor incore dquot initialization functions xfs: refactor xfs_qm_dqtobp and xfs_qm_dqalloc xfs: remove direct calls to _qm_dqread xfs: replace XFS_QMOPT_DQALLOC with a simple boolean xfs: rename on-disk dquot counter zap functions xfs: refactor dquot iteration xfs: avoid ilock games in the quota scrubber xfs: clean up scrub usage of KM_NOFS xfs: btree scrub should check minrecs xfs: refactor scrub transaction allocation function xfs: add helpers to calculate btree size xfs: expose various functions to repair code xfs: add repair helpers for the reverse mapping btree xfs: add repair helpers for the reference count btree xfs: add BMAPI_NORMAP flag to perform block remapping without updating rmapbt xfs: halt auto-reclamation activities while rebuilding rmap iomap: add a swapfile activation function xfs: skip scrub xref if corruption already noted xfs: superblock scrub should use short-lived buffers xfs: refactor quota limits initialization xfs: don't continue scrub if already corrupt xfs: quota scrub should use bmapbtd scrubber xfs: scrub the data fork of the realtime inodes xfs: avoid ABBA deadlock when scrubbing parent pointers xfs: hoist xfs_scrub_agfl_walk to libxfs as xfs_agfl_walk xfs: make xfs_bmapi_remapi work with attribute forks xfs: teach xfs_bmapi_remap to accept some bmapi flags xfs: create tracepoints for online repair xfs: implement the metadata repair ioctl flag fs: clear writeback errors in inode_init_always xfs: fix inobt magic number check xfs: grab the per-ag structure whenever relevant xfs: add helpers to deal with transaction allocation and rolling xfs: add helpers to allocate and initialize fresh btree roots xfs: add helpers to collect and sift btree block pointers during repair xfs: add helpers to dispose of old btree blocks after a repair xfs: recover AG btree roots from rmap data xfs: add helpers to attach quotas to inodes xfs: repair superblocks fs: allow per-device dax status checking for filesystems xfs: xfs_rtword_t should be unsigned, not signed xfs: xfs_rtbuf_get should check the bmapi_read results xfs: strengthen rtalloc query range checks xfs: fix xfs_rtalloc_rec units Dave Chinner (31): xfs: validate cached inodes are free when allocated xfs: validate allocated inode number xfs: make xfs_buf_incore out of line xfs: don't retry xfs_buf_find on XBF_TRYLOCK failure xfs: move generic_write_sync calls inwards iomap: iomap_dio_rw() handles all sync writes iomap: Use FUA for pure data O_DSYNC DIO writes xfs: simplify xfs_file_iomap_begin() logic xfs: clean up locking in xfs_file_iomap_begin xfs: log item flags are racy xfs: add tracing to high level transaction operations xfs: adder caller IP to xfs_defer* tracepoints xfs: don't assert fail with AIL lock held xfs: fix double ijoin in xfs_inactive_symlink_rmt() xfs: fix double ijoin in xfs_reflink_cancel_cow_range xfs: fix double ijoin in xfs_reflink_clear_inode_flag() xfs: add some more debug checks to buffer log item reuse xfs: get rid of the log item descriptor xfs: add mount delay debug option xfs: clear sb->s_fs_info on mount failure xfs: one-shot cached buffers xfs: factor out AG header initialisation from growfs core xfs: convert growfs AG header init to use buffer lists xfs: factor ag btree root block initialisation xfs: turn ag header initialisation into a table driven operation xfs: make imaxpct changes in growfs separate xfs: separate secondary sb update in growfs xfs: rework secondary superblock updates in growfs xfs: move growfs core to libxfs xfs: factor the ag length extension code into libxfs xfs: fix error handling in xfs_refcount_insert() Dave Jiang (1): dax: change bdev_dax_supported() to support boolean returns Eric Sandeen (8): xfs: trace ATTR flags in xattr tracepoints xfs: remove unused flags arg from xfs_dquot_verify xfs: check type in quota verifier during quotacheck xfs: pass full xfs_dqblk to repair during quotacheck xfs: add full xfs_dqblk verifier xfs: print specific dqblk that failed verifiers fs: copy BTRFS_IOC_[SG]ET_FSLABEL to vfs xfs: implement online get/set fs label Omar Sandoval (2): iomap: provide more useful errors for invalid swap files iomap: don't allow holes in swapfiles Souptick Joarder (1): fs: xfs: Change return type to vm_fault_t Documentation/ioctl/ioctl-number.txt | 3 +- block/bio.c | 96 +-- drivers/dax/super.c | 40 +- fs/buffer.c | 114 ---- fs/ext2/super.c | 3 +- fs/ext4/inline.c | 4 +- fs/ext4/super.c | 3 +- fs/gfs2/bmap.c | 11 +- fs/inode.c | 1 + fs/iomap.c | 408 ++++++++++++- fs/xfs/Kconfig | 18 + fs/xfs/Makefile | 9 + fs/xfs/libxfs/xfs_ag.c | 464 +++++++++++++++ fs/xfs/libxfs/xfs_ag.h | 30 + fs/xfs/libxfs/xfs_alloc.c | 129 +++- fs/xfs/libxfs/xfs_alloc.h | 23 +- fs/xfs/libxfs/xfs_alloc_btree.c | 9 + fs/xfs/libxfs/xfs_alloc_btree.h | 2 + fs/xfs/libxfs/xfs_attr.c | 4 +- fs/xfs/libxfs/xfs_attr_remote.c | 2 +- fs/xfs/libxfs/xfs_bmap.c | 92 +-- fs/xfs/libxfs/xfs_bmap.h | 30 +- fs/xfs/libxfs/xfs_bmap_btree.c | 9 + fs/xfs/libxfs/xfs_bmap_btree.h | 3 + fs/xfs/libxfs/xfs_btree.c | 25 +- fs/xfs/libxfs/xfs_btree.h | 3 +- fs/xfs/libxfs/xfs_defer.c | 24 +- fs/xfs/libxfs/xfs_defer.h | 1 + fs/xfs/libxfs/xfs_dquot_buf.c | 94 +-- fs/xfs/libxfs/xfs_errortag.h | 4 +- fs/xfs/libxfs/xfs_format.h | 7 +- fs/xfs/libxfs/xfs_fs.h | 9 +- fs/xfs/libxfs/xfs_ialloc.c | 2 +- fs/xfs/libxfs/xfs_ialloc.h | 3 + fs/xfs/libxfs/xfs_ialloc_btree.c | 11 +- fs/xfs/libxfs/xfs_ialloc_btree.h | 2 + fs/xfs/libxfs/xfs_quota_defs.h | 9 +- fs/xfs/libxfs/xfs_refcount.c | 24 +- fs/xfs/libxfs/xfs_refcount.h | 7 + fs/xfs/libxfs/xfs_rmap.c | 83 +++ fs/xfs/libxfs/xfs_rmap.h | 4 + fs/xfs/libxfs/xfs_rtbitmap.c | 26 +- fs/xfs/libxfs/xfs_sb.c | 147 +++++ fs/xfs/libxfs/xfs_sb.h | 16 + fs/xfs/libxfs/xfs_shared.h | 16 +- fs/xfs/libxfs/xfs_types.h | 2 +- fs/xfs/scrub/agheader.c | 85 +-- fs/xfs/scrub/agheader_repair.c | 70 +++ fs/xfs/scrub/alloc.c | 4 +- fs/xfs/scrub/attr.c | 3 +- fs/xfs/scrub/bmap.c | 8 +- fs/xfs/scrub/btree.c | 42 +- fs/xfs/scrub/common.c | 131 +++- fs/xfs/scrub/common.h | 32 +- fs/xfs/scrub/dir.c | 35 +- fs/xfs/scrub/ialloc.c | 7 +- fs/xfs/scrub/inode.c | 10 +- fs/xfs/scrub/parent.c | 19 +- fs/xfs/scrub/quota.c | 181 +++--- fs/xfs/scrub/refcount.c | 10 +- fs/xfs/scrub/repair.c | 1089 ++++++++++++++++++++++++++++++++++ fs/xfs/scrub/repair.h | 132 +++++ fs/xfs/scrub/rmap.c | 6 +- fs/xfs/scrub/rtbitmap.c | 60 +- fs/xfs/scrub/scrub.c | 142 ++++- fs/xfs/scrub/scrub.h | 5 + fs/xfs/scrub/trace.h | 258 ++++++++ fs/xfs/xfs_aops.c | 21 +- fs/xfs/xfs_bmap_item.c | 4 +- fs/xfs/xfs_bmap_util.c | 10 +- fs/xfs/xfs_buf.c | 100 +++- fs/xfs/xfs_buf.h | 29 +- fs/xfs/xfs_buf_item.c | 10 +- fs/xfs/xfs_dquot.c | 780 ++++++++++++++---------- fs/xfs/xfs_dquot.h | 22 +- fs/xfs/xfs_dquot_item.c | 7 +- fs/xfs/xfs_error.c | 3 + fs/xfs/xfs_extfree_item.c | 6 +- fs/xfs/xfs_file.c | 55 +- fs/xfs/xfs_fsmap.c | 14 +- fs/xfs/xfs_fsops.c | 587 ++++-------------- fs/xfs/xfs_globals.c | 1 + fs/xfs/xfs_icache.c | 94 ++- fs/xfs/xfs_icache.h | 3 + fs/xfs/xfs_icreate_item.c | 4 +- fs/xfs/xfs_inode.c | 47 +- fs/xfs/xfs_inode.h | 14 +- fs/xfs/xfs_inode_item.c | 8 +- fs/xfs/xfs_ioctl.c | 89 ++- fs/xfs/xfs_iomap.c | 186 +++--- fs/xfs/xfs_iops.c | 32 +- fs/xfs/xfs_log.c | 12 +- fs/xfs/xfs_log_cil.c | 22 +- fs/xfs/xfs_log_recover.c | 4 +- fs/xfs/xfs_mount.c | 4 +- fs/xfs/xfs_qm.c | 254 ++++---- fs/xfs/xfs_qm.h | 6 +- fs/xfs/xfs_qm_bhv.c | 2 +- fs/xfs/xfs_qm_syscalls.c | 97 ++- fs/xfs/xfs_quota.h | 22 +- fs/xfs/xfs_quotaops.c | 8 +- fs/xfs/xfs_refcount_item.c | 4 +- fs/xfs/xfs_reflink.c | 18 +- fs/xfs/xfs_rmap_item.c | 4 +- fs/xfs/xfs_rtalloc.h | 9 +- fs/xfs/xfs_super.c | 59 +- fs/xfs/xfs_symlink.c | 10 +- fs/xfs/xfs_sysctl.h | 1 + fs/xfs/xfs_sysfs.c | 31 + fs/xfs/xfs_trace.h | 72 ++- fs/xfs/xfs_trans.c | 89 ++- fs/xfs/xfs_trans.h | 29 +- fs/xfs/xfs_trans_ail.c | 50 +- fs/xfs/xfs_trans_bmap.c | 4 +- fs/xfs/xfs_trans_buf.c | 24 +- fs/xfs/xfs_trans_dquot.c | 4 +- fs/xfs/xfs_trans_extfree.c | 84 ++- fs/xfs/xfs_trans_inode.c | 3 +- fs/xfs/xfs_trans_priv.h | 11 +- fs/xfs/xfs_trans_refcount.c | 4 +- fs/xfs/xfs_trans_rmap.c | 4 +- include/linux/bio.h | 9 + include/linux/buffer_head.h | 2 - include/linux/dax.h | 11 +- include/linux/iomap.h | 27 +- include/uapi/linux/fs.h | 8 +- mm/internal.h | 2 +- mm/readahead.c | 39 +- 128 files changed, 5544 insertions(+), 1980 deletions(-) create mode 100644 fs/xfs/libxfs/xfs_ag.c create mode 100644 fs/xfs/libxfs/xfs_ag.h create mode 100644 fs/xfs/scrub/agheader_repair.c create mode 100644 fs/xfs/scrub/repair.c create mode 100644 fs/xfs/scrub/repair.h -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
