[PATCH v15 00/22] xfs: Delayed Attributes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi all,

This set is a subset of a larger series for parent pointers. Delayed attributes
allow attribute operations (set and remove) to be logged and committed in the same
way that other delayed operations do. This allows more complex operations (like
parent pointers) to be broken up into multiple smaller transactions. To do
this, the existing attr operations must be modified to operate as a delayed
operation.  This means that they cannot roll, commit, or finish transactions.  
Instead, they return -EAGAIN to allow the calling function to handle the transaction.
In this series, we focus on only the delayed attribute portion. We will introduce
parent pointers in a later set.

At the moment, I would like people to focus their review efforts on just this
"delayed attribute" sub series, as I think that is a more conservative use of peoples
review time.  I also think the set is a bit much to manage all at once, and we
need to get the infrastructure ironed out before we focus too much anything
that depends on it. But I do have the extended series for folks that want to
see the bigger picture of where this is going.

To help organize the set, I've arranged the patches to make sort of mini sets.
I thought it would help reviewers break down the reviewing some. For reviewing
purposes, the set could be broken up into 2 phases:

Delay Ready Attributes: (patches 1-15)
Some of these are the remaining patches belonging to the "Delay Ready" series that
we've been working with.  In these patches, transaction handling is removed
from the attr routines, and replaced with a state machine that allows a high
level function to roll the transaction and repeatedly recall the attr routines
until they are finished.  Patches 4-12 correspond to a refactoring RFC effort that
Brian and I had worked on earlier.  The lower level versions of the RFC patch
deviates slightly to correct some minor logic bugs not seen in the RFC.  I further
continued the refactoring to hoist the last state up into the xfs_attr_set_iter
routine.  The final product does create a bit of a monster function, but the state
management code is much more linear than in previous versions.  It should be noted
that while some of these new patches look a little odd, their purpose is neither to
optimize aesthetics, or even the hoist.  The goal is to prepare an arrangement of the
code such that the code changes in patch 12 are minimal and limited to state machine
mechanics.  As in previous revisions, the final product of this sub series is that the
attr routines are now compatible as a .finish_item call back.
   xfs: Add helper xfs_attr_node_remove_step
   xfs: Add xfs_attr_node_remove_cleanup
   xfs: Hoist transaction handling in xfs_attr_node_remove_step
   xfs: Hoist xfs_attr_set_shortform
   xfs: Add helper xfs_attr_set_fmt
   xfs: Separate xfs_attr_node_addname and xfs_attr_node_addname_work
   xfs: Add helper xfs_attr_node_addname_find_attr
   xfs: Hoist xfs_attr_node_addname
   xfs: Hoist xfs_attr_leaf_addname
   xfs: Hoist node transaction handling
   xfs: Add delay ready attr remove routines
   xfs: Add delay ready attr set routines
   xfs: Add state machine tracepoints
   xfs: Rename __xfs_attr_rmtval_remove
   xfs: Handle krealloc errors in xlog_recover_add_to_cont_trans

Delayed Attributes: (patches 15 - 22)
These patches go on to fully implement delayed attributes.  New attr intent and
done items are introduced for use in the existing logging infrastructure.  A
mount option is added to toggle the feature on and off, and an error tag is added
to test the log replay
  xfs: Set up infastructure for deferred attribute operations
  xfs: Skip flip flags for delayed attrs
  xfs: Add xfs_attr_set_deferred and xfs_attr_remove_deferred
  xfs: Remove unused xfs_attr_*_args
  xfs: Add delayed attributes error tag
  xfs: Add delattr mount option
  xfs: Merge xfs_delattr_context into xfs_attr_item

Updates since v14: Mostly re-layering the changes discussed in the rfc along
with other minor nits from the last revision

xfs: Add helper xfs_attr_node_remove_step
   Typo nits in commit message

xfs: Hoist transaction handling in xfs_attr_node_remove_step
   Fixed typo in commit message
   removed unused dp variable from xfs_attr_node_remove_step
   Changed "return error" to "goto out" in xfs_attr_node_removename

xfs: Add delay ready attr remove routines
   Added extra xfs_freestate in xfs_attr_node_removename_setup
   rebase adjustments

xfs: Hoist xfs_attr_set_shortform
   NEW

xfs: Add helper xfs_attr_set_fmt
   NEW

xfs: Separate xfs_attr_node_addname and xfs_attr_node_addname_work
   NEW

xfs: Add helper xfs_attr_node_addname_find_attr
   NEW

xfs: Hoist xfs_attr_node_addname
   NEW

xfs: Hoist xfs_attr_leaf_addname
   NEW

xfs: Hoist node transaction handling
   NEW

xfs: Add delay ready attr set routines
  Fixed typos in comments and commit message
  Rebased onto refactoring additions
  All state management appears in xfs_attr_set_iter
  State machine gotos replaced with switch
  Flow chart updated

xfs: Add state machine tracepoints
  Rebase adjustments
  Added inode to trace data
  Added separate traces types for each function

xfs: Rename __xfs_attr_rmtval_remove
  Rebase adjustments

xfs: Handle krealloc errors in xlog_recover_add_to_cont_trans
  Expanded commit message about alloc warnings

xfs: Skip flip flags for delayed attrs
  Rebase adjustments

xfs: Remove unused xfs_attr_*_args
  Rebase adjustments

xfs: Merge xfs_delattr_context into xfs_attr_item
  Rebase adjustments


xfsprogs: Introduce error injection to allocate only minlen size extents for files
  Ported as a rebase dependancy
  Amended io/inject.c with error tag name to avoid compiler errors

xfsprogs: Introduce error injection to reduce maximum inode fork extent count
  Ported as a rebase dependancy
  Amended io/inject.c with error tag name to avoid compiler errors


This series can be viewed on github here:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_v15

As well as the extended delayed attribute and parent pointer series:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_v15_extended

And the test cases:
https://github.com/allisonhenderson/xfs_work/tree/pptr_xfstestsv2

In order to run the test cases, you will need have the corresponding xfsprogs
changes as well.  Which can be found here:
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_xfsprogs_v15
https://github.com/allisonhenderson/xfs_work/tree/delay_ready_attrs_xfsprogs_v15_extended

To run the xfs attributes tests run:
check -g attr

To run as delayed attributes run:
export MOUNT_OPTIONS="-o delattr"
check -g attr

To run parent pointer tests:
check -g parent

I've also made the corresponding updates to the user space side as well, and ported anything
they need to seat correctly.

Questions, comment and feedback appreciated! 

Thanks all!
Allison 

Allison Collins (1):
  xfs: Add helper xfs_attr_node_remove_step
  xfs: Add xfs_attr_set_deferred and xfs_attr_remove_deferred

Allison Henderson (20):
  xfs: Add xfs_attr_node_remove_cleanup
  xfs: Hoist transaction handling in xfs_attr_node_remove_step
  xfs: Hoist xfs_attr_set_shortform
  xfs: Add helper xfs_attr_set_fmt
  xfs: Separate xfs_attr_node_addname and xfs_attr_node_addname_work
  xfs: Add helper xfs_attr_node_addname_find_attr
  xfs: Hoist xfs_attr_node_addname
  xfs: Hoist xfs_attr_leaf_addname
  xfs: Hoist node transaction handling
  xfs: Add delay ready attr remove routines
  xfs: Add delay ready attr set routines
  xfs: Add state machine tracepoints
  xfs: Rename __xfs_attr_rmtval_remove
  xfs: Handle krealloc errors in xlog_recover_add_to_cont_trans
  xfs: Set up infastructure for deferred attribute operations
  xfs: Skip flip flags for delayed attrs
  xfs: Remove unused xfs_attr_*_args
  xfs: Add delayed attributes error tag
  xfs: Add delattr mount option
  xfs: Merge xfs_delattr_context into xfs_attr_item

 fs/xfs/Makefile                 |   1 +
 fs/xfs/libxfs/xfs_attr.c        | 955 +++++++++++++++++++++++++---------------
 fs/xfs/libxfs/xfs_attr.h        | 366 ++++++++++++++-
 fs/xfs/libxfs/xfs_attr_leaf.c   |   5 +-
 fs/xfs/libxfs/xfs_attr_remote.c | 127 ++++--
 fs/xfs/libxfs/xfs_attr_remote.h |   7 +-
 fs/xfs/libxfs/xfs_defer.c       |   1 +
 fs/xfs/libxfs/xfs_defer.h       |   3 +
 fs/xfs/libxfs/xfs_errortag.h    |   4 +-
 fs/xfs/libxfs/xfs_log_format.h  |  44 +-
 fs/xfs/libxfs/xfs_log_recover.h |   2 +
 fs/xfs/scrub/common.c           |   2 +
 fs/xfs/xfs_acl.c                |   2 +
 fs/xfs/xfs_attr_inactive.c      |   2 +-
 fs/xfs/xfs_attr_item.c          | 830 ++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_attr_item.h          |  52 +++
 fs/xfs/xfs_attr_list.c          |   1 +
 fs/xfs/xfs_error.c              |   3 +
 fs/xfs/xfs_ioctl.c              |   2 +
 fs/xfs/xfs_ioctl32.c            |   2 +
 fs/xfs/xfs_iops.c               |   2 +
 fs/xfs/xfs_log.c                |   4 +
 fs/xfs/xfs_log_recover.c        |   7 +-
 fs/xfs/xfs_mount.h              |   1 +
 fs/xfs/xfs_ondisk.h             |   2 +
 fs/xfs/xfs_super.c              |   6 +-
 fs/xfs/xfs_trace.h              |  26 +-
 fs/xfs/xfs_xattr.c              |   3 +
 28 files changed, 2056 insertions(+), 406 deletions(-)
 create mode 100644 fs/xfs/xfs_attr_item.c
 create mode 100644 fs/xfs/xfs_attr_item.h

-- 
2.7.4




[Index of Archives]     [XFS Filesystem Development (older mail)]     [Linux Filesystem Development]     [Linux Audio Users]     [Yosemite Trails]     [Linux Kernel]     [Linux RAID]     [Linux SCSI]


  Powered by Linux