On Thu, Jun 15, 2023 at 10:06 PM Yu Kuai <yukuai1@xxxxxxxxxxxxxxx> wrote: > > Hi, > > 在 2023/06/15 15:54, Ali Gholami Rudi 写道: > > Perf output: > > > > Samples: 1M of event 'cycles', Event count (approx.): 1158425235997 > > Children Self Command Shared Object Symbol > > + 97.98% 0.01% fio fio [.] fio_libaio_commit > > + 97.95% 0.01% fio libaio.so.1.0.1 [.] io_submit > > + 97.85% 0.01% fio [kernel.kallsyms] [k] __x64_sys_io_submit > > - 97.82% 0.01% fio [kernel.kallsyms] [k] io_submit_one > > - 97.81% io_submit_one > > - 54.62% aio_write > > - 54.60% blkdev_write_iter > > - 36.30% blk_finish_plug > > - flush_plug_callbacks > > - 36.29% raid1_unplug > > - flush_bio_list > > - 18.44% submit_bio_noacct > > - 18.40% brd_submit_bio > > - 18.13% raid1_end_write_request > > - 17.94% raid_end_bio_io > > - 17.82% __wake_up_common_lock > > + 17.79% _raw_spin_lock_irqsave > > - 17.79% __wake_up_common_lock > > + 17.76% _raw_spin_lock_irqsave > > + 18.29% __generic_file_write_iter > > - 43.12% aio_read > > - 43.07% blkdev_read_iter > > - generic_file_read_iter > > - 43.04% blkdev_direct_IO > > - 42.95% submit_bio_noacct > > - 42.23% brd_submit_bio > > - 41.91% raid1_end_read_request > > - 41.70% raid_end_bio_io > > - 41.43% __wake_up_common_lock > > + 41.36% _raw_spin_lock_irqsave > > - 0.68% md_submit_bio > > 0.61% md_handle_request > > + 94.90% 0.00% fio [kernel.kallsyms] [k] __wake_up_common_lock > > + 94.86% 0.22% fio [kernel.kallsyms] [k] _raw_spin_lock_irqsave > > + 94.64% 94.64% fio [kernel.kallsyms] [k] native_queued_spin_lock_slowpath > > + 79.63% 0.02% fio [kernel.kallsyms] [k] submit_bio_noacct > > This looks familiar... Perhaps can you try to test with raid10 with > latest mainline kernel? I used to optimize spin_lock for raid10, and I > don't do this for raid1 yet... I can try to do the same thing for raid1 > if it's valuable. Hi Kuai Which patch? I have a try on raid10. The results are: raid10 READ: bw=3711MiB/s (3892MB/s) WRITE: bw=1590MiB/s (1667MB/s) raid0 READ: bw=5610MiB/s (5882MB/s) WRITE: bw=2405MiB/s (2521MB/s) ram0 READ: bw=5468MiB/s (5734MB/s) WRITE: bw=2343MiB/s (2457MB/s) Because raid10 has a function like raid0. So I did a test on raid0 too. There is a performance gap between raid10 and ram disk too. The strange thing is that raid0 doesn't have a big performance improvement. Regards Xiao > > > > > > > FIO configuration file: > > > > [global] > > name=random reads and writes > > ioengine=libaio > > direct=1 > > readwrite=randrw > > rwmixread=70 > > iodepth=64 > > buffered=0 > > #filename=/dev/ram0 > > filename=/dev/dm/test > > size=1G > > runtime=30 > > time_based > > randrepeat=0 > > norandommap > > refill_buffers > > ramp_time=10 > > bs=4k > > numjobs=400 > > 400 is too aggressive, I think spin_lock from fast path is probably > causing the problem, same as I met before for raid10... > > Thanks, > Kuai > > > group_reporting=1 > > [job1] > > > > . > > >
