Hello,
here is another version of my patches to improve orphan list scaling by
reducing amount of work done under global s_orphan_mutex. Since previous
version I've fixed some bugs (thanks Thavatchai!), retested with updated
xfstests to verify the problem Ted has spotted is fixed, and rerun the
performance tests because the bugs had a non-trivial impact on the
functionality.
To stress orphan list operations I run my artifical test program. The test
program runs given number of processes, each process is truncating a 4k file
by 1 byte until it reaches 1 byte size and then the file is extended to 4k
again.
The average times for 10 runs for the test program to run on my 48-way box
with ext4 on ramdisk are:
Vanilla Patched
Procs Avg Stddev Avg Stddev
1 2.769200 0.056194 2.750000 (-0.6%) 0.054772
2 5.756500 0.313268 5.669000 (-1.5%) 0.587528
4 11.852500 0.130221 8.311000 (-29.9%) 0.257544
10 33.590900 0.394888 20.162000 (-40%) 0.189832
20 71.035400 0.320914 55.854000 (-21.4%) 0.478815
40 236.671100 2.856885 174.543000 (-26.3%) 0.974547
In the lockstat reports, s_orphan_mutex has been #1 in both cases however
the patches significanly reduced the contention. For 10 threads the numbers
look like:
con-bounces contentions waittime-min waittime-max waittime-total
Orig 7089335 7089335 9.07 3504220.69 1473754546.28
Patched 2547868 2547868 9.18 8218.64 547550185.12
waittime-avg acq-bounces acquisitions holdtime-min holdtime-max
Orig 207.88 14487647 16381236 0.16 211.62
Patched 214.91 7994533 8191236 0.16 203.81
holdtime-total holdtime-avg
Orig 79738146.84 4.87
Patched 30660307.81 3.74
We can see the number of acquisitions dropped to a half (we now check
whether inode already is / is not part of the orphan list before acquiring
s_orphan_mutex). The average hold time is somewhat smaller as well and given
that the patched kernel doesn't have those 50% of short lived aquisitions
just for checking whether the inode is part of the orphan list, we can see
that the patched kernel really does significanly less work with s_orphan_lock
held.
Changes since v2:
* Fixed bug in ext4_orphan_del() leading to orphan list corruption - thanks
to Thavatchai for pointing that out.
* Fixed bug in ext4_orphan_del() that could lead to using freed inodes
Changes since v1:
* Fixed up various bugs in error handling pointed out by Thavatchai and
some others as well
* Somewhat reduced critical sections under s_orphan_lock
Honza
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
