On Thu, Apr 18, 2024 at 2:21 PM tada keisuke <keisuke1.tada@xxxxxxxxxx> wrote: > > Changes in v2: > - Add message of performance in cover. > - Fix a problem of error code initialization in patch 6. > - Fix a problem of sleeping during rcu_read_lock() in patch 9. > - Change base-commit from md-6.9 to md-6.10 > > Currently, active_aligned_reads and nr_pending used as counters are atomic types. > Therefore, when inc/dec in a multi-core results in conflicts and READ I/O becomes slow. > To improve performance, use "percpu_ref" counters that can avoid conflicts and maintain consistency. > > Switch modes of percpu_ref to achieve both consistency and conflict avoidance. > During normal operations such as inc/dec, it operates as percpu mode. > When consistency is required, it operates as atomic mode. > The operations that require consistency are as follows: > - Zero check for the counter > - All operations in RAID 1/10 > > Patches 1, 3, 6 change active_aligned_reads, and patches 2, 4, 5, 7 to 11 change nr_pending. > nr_pending temporarily switch from percpu mode to atomic mode in patch 7. > This is to reduce the amount of changes from patches 8 to 10. > Finally, nr_pending switch from atomic mode to percpu mode in patch 11. > > We applied the patch to base-commit and used fio to compare IOPS. > CPU: AMD EPYC 7313P (3.0GHz, 16cores) > DISK: ramdisk x 3 (modprobe brd rd_nr=3) > RAID: level 5 > fio config: bs=4k, rw=randread, iodepth=128, numjobs=16 > > without patch: 3.64 MIOPS > with patch : 3.84 MIOPS Hi Tada Thanks for the patch set. Have you done tests with nvme/ssd or hdd? It's better to see the results with real disks. Best Regards Xiao > > Keisuke TADA (11): > add infra for active_aligned_reads changes > add infra for nr_pending changes > workaround for inconsistency of config state in takeover > minimize execution of zero check for nr_pending > match the type of variables to percpu_ref > avoid conflicts in active_aligned_reads operations > change the type of nr_pending from atomic_t to percpu_ref > add atomic mode switching in RAID 1/10 > add atomic mode switching when removing disk > add atomic mode switching when I/O completion > avoid conflicts in nr_pending operations > > drivers/md/md-bitmap.c | 2 +- > drivers/md/md.c | 48 ++++++++++++++++++--- > drivers/md/md.h | 62 +++++++++++++++++++++++---- > drivers/md/raid1.c | 37 +++++++++++------ > drivers/md/raid10.c | 60 ++++++++++++++++----------- > drivers/md/raid5-cache.c | 4 +- > drivers/md/raid5.c | 90 +++++++++++++++++++++++++++------------- > drivers/md/raid5.h | 17 +++++++- > 8 files changed, 238 insertions(+), 82 deletions(-) > > > base-commit: 9d1110f99c253ccef82e480bfe9f38a12eb797a7 > -- > 2.34.1 > > >
