On Tue, Jan 24, 2023 at 07:38:57AM +0000, Allison Henderson wrote: > On Mon, 2023-01-23 at 20:38 -0800, Darrick J. Wong wrote: > > On Mon, Jan 23, 2023 at 06:35:53PM -0700, > > allison.henderson@xxxxxxxxxx wrote: > > > From: Allison Henderson <allison.henderson@xxxxxxxxxx> > > > > > > Hi all, > > > > > > This is the latest parent pointer attributes for xfs. > > > The goal of this patch set is to add a parent pointer attribute to > > > each inode. > > > The attribute name containing the parent inode, generation, and > > > directory > > > offset, while the attribute value contains the file name. This > > > feature will > > > enable future optimizations for online scrub, shrink, nfs handles, > > > verity, or > > > any other feature that could make use of quickly deriving an inodes > > > path from > > > the mount point. > > > > > > This set can be viewed on github here > > > https://urldefense.com/v3/__https://github.com/allisonhenderson/xfs/tree/xfs_new_pptrsv8_r1__;!!ACWV5N9M2RV99hQ!N2nNiERMG5LB2c5JRplSXXpgpp6yVFE8n22VEStqjIgbSm7yY0m92DEvfpPkD7XfwguuE49Sclb-8uSz6Ss5$ > > > > > > > I ran the kernel v8r1 branch through fstests and it came up with > > this: > > https://urldefense.com/v3/__https://djwong.org/fstests/output/.e2ecf3cd98a7b55bfe8b9d7f33d2ef9549ccb6526765421fd929cf6b1fa82265/.238f7848578c98c24e6347e59963548102fe83037127e44802014e48281a8ccc/?C=M;O=A__;!!ACWV5N9M2RV99hQ!N2nNiERMG5LB2c5JRplSXXpgpp6yVFE8n22VEStqjIgbSm7yY0m92DEvfpPkD7XfwguuE49Sclb-8vEH_S-U$ > > Alrighty, I'll take a look, I had run it through a night or so ago, and > hadnt noticed the same failures you list here. Some of these fail out > of the box for me, so I didnt think them associated with pptr changes. <nod> Some of those are just problems resulting from hardcoding xfs_db and mkfs output. I noticed that the shutdowns are a result of reservationless link/rename operations; it's easier just to get rid of that nonfeature. Also, there's a huge memory leak of xfs_parent_defer objects. That's at least what I've found so far. Will send the whole mess through fstests tonight. > > > > > > Looks better than v7, though I haven't tracked down why the fs goes > > down > > in generic/083 yet. I think it's the same "rename doesn't reserve > > enough blocks" problem I was muttering about last time. I think I > > need > > to look through the block reservation calculations again. > > > > That said, I *did* finally write the code that scans the parent > > pointers > > to generate new directories. It works for simple stupid cases where > > fsstress isn't running, but the live hooks part doesn't work because > > I > > haven't though through the locking model yet! :) > > > > > And the corresponding xfsprogs code is here > > > https://urldefense.com/v3/__https://github.com/allisonhenderson/xfsprogs/tree/xfsprogs_new_pptrs_v8_r1__;!!ACWV5N9M2RV99hQ!N2nNiERMG5LB2c5JRplSXXpgpp6yVFE8n22VEStqjIgbSm7yY0m92DEvfpPkD7XfwguuE49Sclb-8nOCLP-H$ > > > > > > > Will rebase xfsprogs against v8r1 tomorrow. > I think you had a scrub patch for this I forgot to add. will do > that... Aside from that, not much change there tho <nod> > > > > > This set has been tested with the below parent pointers tests > > > https://urldefense.com/v3/__https://lore.kernel.org/fstests/20221012013812.82161-1-catherine.hoang@xxxxxxxxxx/T/*t__;Iw!!ACWV5N9M2RV99hQ!N2nNiERMG5LB2c5JRplSXXpgpp6yVFE8n22VEStqjIgbSm7yY0m92DEvfpPkD7XfwguuE49Sclb-8ilcWlei$ > > > > > > > And fix fstests after that. > The testcase i had saved as something Catherine could work on, but > there's no rush on it. The testcase tends to get tossed around any > time there are api changes so it made sense to land at least kernel > side first, though hopefully things should be decently firm at this > point. <nod> I'll not pay too much attention to xfs/018 then. --D > Allison > > > > > --D > > > > > Updates since v7: > > > > > > xfs: Increase XFS_QM_TRANS_MAXDQS to 5 > > > Modified xfs_dqlockn to sort dquotes before locking > > > > > > xfs: Hold inode locks in xfs_trans_alloc_dir > > > Modified xfs_trans_alloc_dir to release locks before retrying > > > trans allocation > > > > > > xfs: Hold inode locks in xfs_rename > > > Modified xfs_rename to release locks before retrying trans > > > allocation > > > > > > xfs: Expose init_xattrs in xfs_create_tmpfile > > > Fixed xfs_generic_create to init attr tree > > > > > > xfs: add parent pointer support to attribute code > > > Updated xchk_xattr_rec with new XFS_ATTR_PARENT flag > > > > > > xfs: Add parent pointer ioctl > > > Include xfs_parent_utils.h in xfs_parent_utils.c to quiet > > > compiler warnings > > > > > > Questions comments and feedback appreciated! > > > > > > Thanks all! > > > Allison > > > > > > Allison Henderson (27): > > > xfs: Add new name to attri/d > > > xfs: Increase XFS_DEFER_OPS_NR_INODES to 5 > > > xfs: Increase XFS_QM_TRANS_MAXDQS to 5 > > > xfs: Hold inode locks in xfs_ialloc > > > xfs: Hold inode locks in xfs_trans_alloc_dir > > > xfs: Hold inode locks in xfs_rename > > > xfs: Expose init_xattrs in xfs_create_tmpfile > > > xfs: get directory offset when adding directory name > > > xfs: get directory offset when removing directory name > > > xfs: get directory offset when replacing a directory name > > > xfs: add parent pointer support to attribute code > > > xfs: define parent pointer xattr format > > > xfs: Add xfs_verify_pptr > > > xfs: extend transaction reservations for parent attributes > > > xfs: parent pointer attribute creation > > > xfs: add parent attributes to link > > > xfs: add parent attributes to symlink > > > xfs: remove parent pointers in unlink > > > xfs: Indent xfs_rename > > > xfs: Add parent pointers to rename > > > xfs: Add parent pointers to xfs_cross_rename > > > xfs: Add the parent pointer support to the superblock version 5. > > > xfs: Add helper function xfs_attr_list_context_init > > > xfs: Filter XFS_ATTR_PARENT for getfattr > > > xfs: Add parent pointer ioctl > > > xfs: fix unit conversion error in xfs_log_calc_max_attrsetm_res > > > xfs: drop compatibility minimum log size computations for reflink > > > > > > fs/xfs/Makefile | 2 + > > > fs/xfs/libxfs/xfs_attr.c | 71 +++++- > > > fs/xfs/libxfs/xfs_attr.h | 13 +- > > > fs/xfs/libxfs/xfs_da_btree.h | 3 + > > > fs/xfs/libxfs/xfs_da_format.h | 38 ++- > > > fs/xfs/libxfs/xfs_defer.c | 28 ++- > > > fs/xfs/libxfs/xfs_defer.h | 8 +- > > > fs/xfs/libxfs/xfs_dir2.c | 21 +- > > > fs/xfs/libxfs/xfs_dir2.h | 7 +- > > > fs/xfs/libxfs/xfs_dir2_block.c | 9 +- > > > fs/xfs/libxfs/xfs_dir2_leaf.c | 8 +- > > > fs/xfs/libxfs/xfs_dir2_node.c | 8 +- > > > fs/xfs/libxfs/xfs_dir2_sf.c | 6 + > > > fs/xfs/libxfs/xfs_format.h | 4 +- > > > fs/xfs/libxfs/xfs_fs.h | 75 ++++++ > > > fs/xfs/libxfs/xfs_log_format.h | 7 +- > > > fs/xfs/libxfs/xfs_log_rlimit.c | 53 ++++ > > > fs/xfs/libxfs/xfs_parent.c | 207 +++++++++++++++ > > > fs/xfs/libxfs/xfs_parent.h | 46 ++++ > > > fs/xfs/libxfs/xfs_sb.c | 4 + > > > fs/xfs/libxfs/xfs_trans_resv.c | 324 ++++++++++++++++++++---- > > > fs/xfs/libxfs/xfs_trans_space.h | 8 - > > > fs/xfs/scrub/attr.c | 4 +- > > > fs/xfs/xfs_attr_item.c | 142 +++++++++-- > > > fs/xfs/xfs_attr_item.h | 1 + > > > fs/xfs/xfs_attr_list.c | 17 +- > > > fs/xfs/xfs_dquot.c | 38 +++ > > > fs/xfs/xfs_dquot.h | 1 + > > > fs/xfs/xfs_file.c | 1 + > > > fs/xfs/xfs_inode.c | 428 +++++++++++++++++++++++++--- > > > ---- > > > fs/xfs/xfs_inode.h | 3 +- > > > fs/xfs/xfs_ioctl.c | 148 +++++++++-- > > > fs/xfs/xfs_ioctl.h | 2 + > > > fs/xfs/xfs_iops.c | 3 +- > > > fs/xfs/xfs_ondisk.h | 4 + > > > fs/xfs/xfs_parent_utils.c | 126 ++++++++++ > > > fs/xfs/xfs_parent_utils.h | 11 + > > > fs/xfs/xfs_qm.c | 4 +- > > > fs/xfs/xfs_qm.h | 2 +- > > > fs/xfs/xfs_super.c | 4 + > > > fs/xfs/xfs_symlink.c | 58 ++++- > > > fs/xfs/xfs_trans.c | 9 +- > > > fs/xfs/xfs_trans_dquot.c | 15 +- > > > fs/xfs/xfs_xattr.c | 5 +- > > > fs/xfs/xfs_xattr.h | 1 + > > > 45 files changed, 1731 insertions(+), 246 deletions(-) > > > create mode 100644 fs/xfs/libxfs/xfs_parent.c > > > create mode 100644 fs/xfs/libxfs/xfs_parent.h > > > create mode 100644 fs/xfs/xfs_parent_utils.c > > > create mode 100644 fs/xfs/xfs_parent_utils.h > > > > > > -- > > > 2.25.1 > > > >
