The current implementation with the async thread pool ends up spreading
the work over too many threads. The btrfs workqueue is targeted at high
cpu utilization works and has a threshold mechanism to limit thread
spawning. Unfortunately it still ends up increasing cpu utilization
without a comparable improvement in throughput. Here are the numbers
relative to the multicore disabled case:
idle_thresh throughput cycles
4 +0% +102%
64 +4% +63%
128 +1% +45%
This appears to show that something more fundamental needs to happen to
take advantage of percpu raid processing. More profiling is needed, but
the suspects in my mind are conf->device_lock contention and the fact
that all work is serialized through conf->handle_list with no method for
encouraging stripe_head to thread affinity.
The idle_thresh parameter translates to the number of stripes that can
be in a thread's backlog before new work spills over into a new thread.
The current implementation via async is effectively using a threshold of
zero which ends up impacting throughput due to excessive cpu utilization
and thread creation. I do not think we need any more experimentation
before declaring the current implementation broken.
---
Dan Williams (2):
btrq: uplevel the btrfs thread pool for md/raid456 usage
md/raid456: switch to btrq for multicore operation
drivers/md/Kconfig | 1
drivers/md/raid5.c | 79 +++++++++++++----------
drivers/md/raid5.h | 13 ++--
fs/btrfs/Kconfig | 1
fs/btrfs/Makefile | 2 -
fs/btrfs/ctree.h | 22 +++---
fs/btrfs/disk-io.c | 157 +++++++++++++++++++++++-----------------------
fs/btrfs/extent-tree.c | 7 +-
fs/btrfs/inode.c | 18 +++--
fs/btrfs/relocation.c | 22 +++---
fs/btrfs/volumes.c | 12 ++--
fs/btrfs/volumes.h | 4 +
include/linux/btrqueue.h | 36 +++++------
lib/Kconfig | 6 ++
lib/Makefile | 2 +
lib/btrqueue.c | 119 ++++++++++++++++++-----------------
16 files changed, 265 insertions(+), 236 deletions(-)
rename fs/btrfs/async-thread.h => include/linux/btrqueue.h (76%)
rename fs/btrfs/async-thread.c => lib/btrqueue.c (82%)
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
