[PATCH v14 00/15] xfs: Delayed Attributes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi all,

This set is a subset of a larger series for parent pointers. Delayed attributes
allow attribute operations (set and remove) to be logged and committed in the same
way that other delayed operations do. This allows more complex operations (like
parent pointers) to be broken up into multiple smaller transactions. To do
this, the existing attr operations must be modified to operate as a delayed
operation.  This means that they cannot roll, commit, or finish transactions.  
Instead, they return -EAGAIN to allow the calling function to handle the transaction.
In this series, we focus on only the delayed attribute portion. We will introduce
parent pointers in a later set.

At the moment, I would like people to focus their review efforts on just this
"delayed attribute" sub series, as I think that is a more conservative use of peoples
review time.  I also think the set is a bit much to manage all at once, and we
need to get the infrastructure ironed out before we focus too much anything
that depends on it. But I do have the extended series for folks that want to
see the bigger picture of where this is going.

To help organize the set, I've arranged the patches to make sort of mini sets.
I thought it would help reviewers break down the reviewing some. For reviewing
purposes, the set could be broken up into 2 phases:

Delay Ready Attributes: (patches 1-8)
These are the remaining patches belonging to the "Delay Ready" series that
we've been working with.  In these patches, transaction handling is removed
from the attr routines, and replaced with a state machine that allows a high
level function to roll the transaction and repeatedly recall the attr routines
until they are finished.  The behavior of the attr set/remove routines
are now also compatible as a .finish_item callback
  xfs: Add helper xfs_attr_node_remove_step
  xfs: Hoist transaction handling in xfs_attr_node_remove_step
  xfs: Add xfs_attr_node_remove_cleanup
  xfs: Add delay ready attr remove routines
  xfs: Add delay ready attr set routines
  xfs: Add statemachine tracepoints
  xfs: Rename __xfs_attr_rmtval_remove
  xfs: Handle krealloc errors in xlog_recover_add_to_cont_trans

Delayed Attributes: (patches 9 - 15)
These patches go on to fully implement delayed attributes.  New attr intent and
done items are introduced for use in the existing logging infrastructure.  A
mount option is added to toggle the feature on and off, and an error tag is added
to test the log replay
  xfs: Set up infastructure for deferred attribute operations
  xfs: Skip flip flags for delayed attrs
  xfs: Add xfs_attr_set_deferred and xfs_attr_remove_deferred
  xfs: Remove unused xfs_attr_*_args
  xfs: Add delayed attributes error tag
  xfs: Add delattr mount option
  xfs: Merge xfs_delattr_context into xfs_attr_item

Updates since v13: Mostly integrating review feed back, which involved quite
a few changes.  Most significant being the removal of the incompat flag which
has been replaced with a mount option.  Though we do need to set an incompat
flag, we cannot use it to enable/disable the feature.  It needs to be set when
ever a new log item is written to disk.  ATM, writing the flag to the super
block on the fly currently not supported, but overlaps with another work effort
that needs it for other reasons.  For now, writing new log entries to the disk
is simply disabled unless specified at mount time.

xfs: Add helper xfs_attr_node_remove_step
   Initialized state to null in xfs_attr_node_removename
   Fixed typo "shirnk"

xfs: Add xfs_attr_node_remove_cleanup
   NEW

xfs: Hoist transaction handling in xfs_attr_node_remove_step
   NEW

xfs: Add delay ready attr remove routines
   Removed uneeded error = 0; initialization in xfs_attr_remove_args
   Simplified if/else logic in xfs_attr_remove_iter.  Added some commentary
   Collapsed extra state arg in xfs_attr_node_removename_setup
   Moved up state init in xfs_attr_node_removename_setup
   Fixed variable alignment in xfs_attr_node_remove_step
   Removed XFS_DAC_NODE_RMVNAME_INIT
   Modified default switch case in xfs_attr_node_removename_iter to goto out instead of return
   Updated /* fallthrough */ comment in xfs_attr_node_removename_iter
   Updated commentary for xfs_attr_node_remove_step in xfs_attr_node_removename_iter
   Updated xfs_attr_trans_roll to skip roll if defer_finish is called
   Expanded state machine diagram in b/fs/xfs/libxfs/xfs_attr.h
   Fixed typo in xfs_attr_rmtval_remove comment 
   Added explanation to stateless -EAGAIN return
   
xfs: Add delay ready attr set routines
   Rebase adjustments
   Removed uneeded bhold/join from xfs_attr_set_args
   Removed xfs_attr_rmtval_remove prototype from xfs_attr_remote.h.
   Added comments to EAGAIN returns that dont needs states
   Added XFS_DAS_RM_LBLK and XFS_DAS_UNINIT to switch in xfs_attr_set_iter
   Assert on states that belong to remove path 
   Expanded comments in xfs_attr_set_iter
   Fixed ENSPC handler for xfs_attr_leaf_try_add
   Expanded state machine diagram in b/fs/xfs/libxfs/xfs_attr.h
   Fix a bug found with generic/449
      Do no return EAGAIN on sucessfull leaf add if theres nothing left to do
   Moved state set to before xfs_attr_rmtval_remove call

xfs: Add statemachine tracepoints
  NEW

xfs: Rename __xfs_attr_rmtval_remove
   Rebase adjustments

xfs: Handle krealloc errors in xlog_recover_add_to_cont_trans
   NEW

xfs: Set up infastructure for deferred attribute operations
  Added extra buffer_size parameter to xfs_attri_log_item
    This allows a trailing buffer to be allocated allong with the intent
    Simplifies having to realloc the intent during the log commit
  Misc white space cleanups
  Removed unused xfs_attrlen_t
  Removed wrapper function xfs_attri_item_sizeof and xfs_attrd_item_sizeof
  Added inline helper ATTR_NVEC_SIZE.  Simplified with roundup function
  Rephrased comment for xfs_trans_attr
  Moved attrip cleanup in xfs_attr_finish_item to xfs_attri_item_committed
  Removed unused xfs_attri_item_committing  and xfs_attrd_item_committing
  Merged xfs_attrd_init into xfs_trans_get_attrd
  Added XFS_ITEM_RELEASE_WHEN_COMMITTED to xfs_attrd_item_ops
  Reworked xfs_attri_item_recover to preseerve log replay order
  Expanded xfs_attri_item_recover validate check
  Added xfs_attri_item_relog
  Added validate check to xfs_attr_leaf_try_add
  Trimmed down xfs_attri_log_item commentary
  Reordered xfs_attri_log_item members to remove holes
  Renamed xfs_sb_version_hasdelattr to xfs_hasdelattr, and moved to xfs_attr.h
    This will be used for a mount option later instead of an incompat flag

xfs: Skip flip flags for delayed attrs
  NEW

xfs: Add xfs_attr_set_deferred and xfs_attr_remove_deferred
  Merged with patch xfs: Enable delayed attributes
  Manged kmem_alloc_large to kmem_zalloc 
  Merged with previous "Enable delayed attributes" patch

xfs: Enable delayed attributes
  REMOVED

xfs: Remove unused xfs_attr_*_args
  Collapsed in leaf_bp parameter in xfs_attr_set_iter

xfs: Add delayed attributes error tag
   No change

xfs: Add feature bit XFS_SB_FEAT_INCOMPAT_LOG_DELATTR
   REMOVED

xfs: Add delattr mount option
   NEW

b54c08d xfsprogs: Merge xfs_delattr_context into xfs_attr_item
   NEW

This series can be viewed on github here:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_v14

As well as the extended delayed attribute and parent pointer series:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_v14_extended

And the test cases:
https://github.com/allisonhenderson/xfs_work/tree/pptr_xfstestsv2

In order to run the test cases, you will need have the corresponding xfsprogs
changes as well.  Which can be found here:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_xfsprogs_v14
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_xfsprogs_v14_extended

To run the xfs attributes tests run:
check -g attr

To run as delayed attributes run:
export MOUNT_OPTIONS="-o delattr"
check -g attr

To run parent pointer tests:
check -g parent

I've also made the corresponding updates to the user space side as well, and ported anything
they need to seat correctly.

Questions, comment and feedback appreciated! 

Thanks all!
Allison 


Allison Collins (3):
  xfs: Add helper xfs_attr_node_remove_step
  xfs: Add xfs_attr_set_deferred and xfs_attr_remove_deferred
  xfs: Add delayed attributes error tag

Allison Henderson (12):
  xfs: Add xfs_attr_node_remove_cleanup
  xfs: Hoist transaction handling in xfs_attr_node_remove_step
  xfs: Add delay ready attr remove routines
  xfs: Add delay ready attr set routines
  xfs: Add state machine tracepoints
  xfs: Rename __xfs_attr_rmtval_remove
  xfs: Handle krealloc errors in xlog_recover_add_to_cont_trans
  xfs: Set up infastructure for deferred attribute operations
  xfs: Skip flip flags for delayed attrs
  xfs: Remove unused xfs_attr_*_args
  xfs: Add delattr mount option
  xfs: Merge xfs_delattr_context into xfs_attr_item

 fs/xfs/Makefile                 |   1 +
 fs/xfs/libxfs/xfs_attr.c        | 633 ++++++++++++++++++++----------
 fs/xfs/libxfs/xfs_attr.h        | 360 ++++++++++++++++-
 fs/xfs/libxfs/xfs_attr_leaf.c   |   5 +-
 fs/xfs/libxfs/xfs_attr_remote.c | 126 +++---
 fs/xfs/libxfs/xfs_attr_remote.h |   7 +-
 fs/xfs/libxfs/xfs_defer.c       |   1 +
 fs/xfs/libxfs/xfs_defer.h       |   3 +
 fs/xfs/libxfs/xfs_errortag.h    |   4 +-
 fs/xfs/libxfs/xfs_log_format.h  |  44 ++-
 fs/xfs/libxfs/xfs_log_recover.h |   2 +
 fs/xfs/scrub/common.c           |   2 +
 fs/xfs/xfs_acl.c                |   2 +
 fs/xfs/xfs_attr_inactive.c      |   2 +-
 fs/xfs/xfs_attr_item.c          | 830 ++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_attr_item.h          |  52 +++
 fs/xfs/xfs_attr_list.c          |   1 +
 fs/xfs/xfs_error.c              |   3 +
 fs/xfs/xfs_ioctl.c              |   2 +
 fs/xfs/xfs_ioctl32.c            |   2 +
 fs/xfs/xfs_iops.c               |   2 +
 fs/xfs/xfs_log.c                |   4 +
 fs/xfs/xfs_log_recover.c        |   7 +-
 fs/xfs/xfs_mount.h              |   1 +
 fs/xfs/xfs_ondisk.h             |   2 +
 fs/xfs/xfs_super.c              |   6 +-
 fs/xfs/xfs_trace.h              |  21 +-
 fs/xfs/xfs_xattr.c              |   3 +
 28 files changed, 1874 insertions(+), 254 deletions(-)
 create mode 100644 fs/xfs/xfs_attr_item.c
 create mode 100644 fs/xfs/xfs_attr_item.h

-- 
2.7.4




[Index of Archives]     [XFS Filesystem Development (older mail)]     [Linux Filesystem Development]     [Linux Audio Users]     [Yosemite Trails]     [Linux Kernel]     [Linux RAID]     [Linux SCSI]


  Powered by Linux