Hi,
在 2023/06/15 15:54, Ali Gholami Rudi 写道:
Perf output:
Samples: 1M of event 'cycles', Event count (approx.): 1158425235997
Children Self Command Shared Object Symbol
+ 97.98% 0.01% fio fio [.] fio_libaio_commit
+ 97.95% 0.01% fio libaio.so.1.0.1 [.] io_submit
+ 97.85% 0.01% fio [kernel.kallsyms] [k] __x64_sys_io_submit
- 97.82% 0.01% fio [kernel.kallsyms] [k] io_submit_one
- 97.81% io_submit_one
- 54.62% aio_write
- 54.60% blkdev_write_iter
- 36.30% blk_finish_plug
- flush_plug_callbacks
- 36.29% raid1_unplug
- flush_bio_list
- 18.44% submit_bio_noacct
- 18.40% brd_submit_bio
- 18.13% raid1_end_write_request
- 17.94% raid_end_bio_io
- 17.82% __wake_up_common_lock
+ 17.79% _raw_spin_lock_irqsave
- 17.79% __wake_up_common_lock
+ 17.76% _raw_spin_lock_irqsave
+ 18.29% __generic_file_write_iter
- 43.12% aio_read
- 43.07% blkdev_read_iter
- generic_file_read_iter
- 43.04% blkdev_direct_IO
- 42.95% submit_bio_noacct
- 42.23% brd_submit_bio
- 41.91% raid1_end_read_request
- 41.70% raid_end_bio_io
- 41.43% __wake_up_common_lock
+ 41.36% _raw_spin_lock_irqsave
- 0.68% md_submit_bio
0.61% md_handle_request
+ 94.90% 0.00% fio [kernel.kallsyms] [k] __wake_up_common_lock
+ 94.86% 0.22% fio [kernel.kallsyms] [k] _raw_spin_lock_irqsave
+ 94.64% 94.64% fio [kernel.kallsyms] [k] native_queued_spin_lock_slowpath
+ 79.63% 0.02% fio [kernel.kallsyms] [k] submit_bio_noacct
This looks familiar... Perhaps can you try to test with raid10 with
latest mainline kernel? I used to optimize spin_lock for raid10, and I
don't do this for raid1 yet... I can try to do the same thing for raid1
if it's valuable.
FIO configuration file:
[global]
name=random reads and writes
ioengine=libaio
direct=1
readwrite=randrw
rwmixread=70
iodepth=64
buffered=0
#filename=/dev/ram0
filename=/dev/dm/test
size=1G
runtime=30
time_based
randrepeat=0
norandommap
refill_buffers
ramp_time=10
bs=4k
numjobs=400
400 is too aggressive, I think spin_lock from fast path is probably
causing the problem, same as I met before for raid10...
Thanks,
Kuai
group_reporting=1
[job1]
.
