On Thu, May 27, 2021 at 12:03:18PM -0700, Darrick J. Wong wrote: > On Wed, May 19, 2021 at 10:13:12PM +1000, Dave Chinner wrote: > > From: Dave Chinner <dchinner@xxxxxxxxxx> > > > > So that we can remove the cil_lock which is a global serialisation > > point. We've already got ordering sorted, so all we need to do is > > treat the CIL list like the busy extent list and reconstruct it > > before the push starts. > > > > This is what we're trying to avoid: > > > > - 75.35% 1.83% [kernel] [k] xfs_log_commit_cil > > - 46.35% xfs_log_commit_cil > > - 41.54% _raw_spin_lock > > - 67.30% do_raw_spin_lock > > 66.96% __pv_queued_spin_lock_slowpath > > > > Which happens on a 32p system when running a 32-way 'rm -rf' > > workload. After this patch: > > > > - 20.90% 3.23% [kernel] [k] xfs_log_commit_cil > > - 17.67% xfs_log_commit_cil > > - 6.51% xfs_log_ticket_ungrant > > 1.40% xfs_log_space_wake > > 2.32% memcpy_erms > > - 2.18% xfs_buf_item_committing > > - 2.12% xfs_buf_item_release > > - 1.03% xfs_buf_unlock > > 0.96% up > > 0.72% xfs_buf_rele > > 1.33% xfs_inode_item_format > > 1.19% down_read > > 0.91% up_read > > 0.76% xfs_buf_item_format > > - 0.68% kmem_alloc_large > > - 0.67% kmem_alloc > > 0.64% __kmalloc > > 0.50% xfs_buf_item_size > > > > It kinda looks like the workload is running out of log space all > > the time. But all the spinlock contention is gone and the > > transaction commit rate has gone from 800k/s to 1.3M/s so the amount > > of real work being done has gone up a *lot*. > > > > Signed-off-by: Dave Chinner <dchinner@xxxxxxxxxx> > > --- > > fs/xfs/xfs_log_cil.c | 69 +++++++++++++++++++------------------------ > > fs/xfs/xfs_log_priv.h | 3 +- > > 2 files changed, 31 insertions(+), 41 deletions(-) > > > > diff --git a/fs/xfs/xfs_log_cil.c b/fs/xfs/xfs_log_cil.c > > index ca6e411e388e..287dc7d0d508 100644 > > --- a/fs/xfs/xfs_log_cil.c > > +++ b/fs/xfs/xfs_log_cil.c > > @@ -72,6 +72,7 @@ xlog_cil_ctx_alloc(void) > > ctx = kmem_zalloc(sizeof(*ctx), KM_NOFS); > > INIT_LIST_HEAD(&ctx->committing); > > INIT_LIST_HEAD(&ctx->busy_extents); > > + INIT_LIST_HEAD(&ctx->log_items); > > I see you moved the log item list to the cil ctx for benefit of > _pcp_dead, correct? Largely, yes. It also helps to have the item push list rooted in the structure that holds all of the push specific state (i.e. the CIL ctx) once we detatch that from the CIL itself. > If so, then this isn't especially different from the last version. *nod* > Yay for shortening lock critical sections, > Reviewed-by: Darrick J. Wong <djwong@xxxxxxxxxx> Ta. -Dave. -- Dave Chinner david@xxxxxxxxxxxxx
