Hi,
在 2023/06/16 19:51, Ali Gholami Rudi 写道:
Thanks for testing!
Perf's output:
+ 93.79% 0.09% fio [kernel.kallsyms] [k] entry_SYSCALL_64_after_hwframe
+ 92.89% 0.05% fio [kernel.kallsyms] [k] do_syscall_64
+ 86.59% 0.07% fio [kernel.kallsyms] [k] __x64_sys_io_submit
- 85.61% 0.10% fio [kernel.kallsyms] [k] io_submit_one
- 85.51% io_submit_one
- 47.98% aio_read
- 46.18% blkdev_read_iter
- 44.90% __blkdev_direct_IO_async
- 41.68% submit_bio_noacct_nocheck
- 41.50% __submit_bio
- 18.76% md_handle_request
- 18.71% raid10_make_request
- 18.54% raid10_read_request
16.54% read_balance
There is not any spin_lock in fast path anymore. Now, looks like
main cost is raid10 io path now(read_balance looks worth
investigation, 16.54% is too much), and for a real device with ms
io latency, I think latency in io path may not matter.
Thanks,
Kuai
1.80% wait_barrier_nolock
- 14.18% raid10_end_read_request
- 8.16% raid_end_bio_io
7.44% allow_barrier
- 8.40% brd_submit_bio
2.49% __radix_tree_lookup
- 1.39% bio_iov_iter_get_pages
- 1.04% iov_iter_get_pages
- __iov_iter_get_pages_alloc
- 0.93% get_user_pages_fast
0.79% internal_get_user_pages_fast
- 1.17% bio_alloc_bioset
0.59% mempool_alloc
0.93% filemap_write_and_wait_range
0.65% security_file_permission
- 35.21% aio_write
- 34.53% blkdev_write_iter
- 18.25% __generic_file_write_iter
- 17.84% generic_file_direct_write
- 17.35% __blkdev_direct_IO_async
- 16.34% submit_bio_noacct_nocheck
- 16.31% __submit_bio
- 16.28% md_handle_request
- 16.26% raid10_make_request
9.26% raid10_write_one_disk
2.11% wait_blocked_dev
0.58% wait_barrier_nolock
- 16.02% blk_finish_plug
- 16.01% flush_plug_callbacks
- 16.00% raid10_unplug
- 15.89% submit_bio_noacct_nocheck
- 15.84% __submit_bio
- 8.66% raid10_end_write_request
- 3.55% raid_end_bio_io
3.54% allow_barrier
0.72% find_bio_disk.isra.0
- 7.04% brd_submit_bio
1.38% __radix_tree_lookup
0.61% kmem_cache_alloc
+ 84.41% 0.99% fio fio [.] thread_main
+ 83.79% 0.00% fio [unknown] [.] 0xffffffffffffffff
+ 83.60% 0.00% fio fio [.] run_threads
+ 83.32% 0.00% fio fio [.] do_io (inlined)
+ 81.12% 0.43% fio libc-2.31.so [.] syscall
+ 76.23% 0.69% fio fio [.] td_io_queue
+ 76.16% 4.66% fio [kernel.kallsyms] [k] submit_bio_noacct_nocheck
+ 75.63% 0.25% fio fio [.] fio_libaio_commit
+ 75.57% 0.17% fio fio [.] td_io_commit
+ 75.54% 0.00% fio fio [.] io_u_submit (inlined)
+ 75.33% 0.17% fio libaio.so.1.0.1 [.] io_submit
+ 73.66% 0.07% fio [kernel.kallsyms] [k] __submit_bio
+ 67.30% 5.07% fio [kernel.kallsyms] [k] __blkdev_direct_IO_async
+ 48.02% 0.03% fio [kernel.kallsyms] [k] aio_read
+ 46.22% 0.05% fio [kernel.kallsyms] [k] blkdev_read_iter
+ 35.71% 3.88% fio [kernel.kallsyms] [k] raid10_make_request
+ 35.23% 0.02% fio [kernel.kallsyms] [k] aio_write
+ 35.08% 0.06% fio [kernel.kallsyms] [k] md_handle_request
+ 34.55% 0.02% fio [kernel.kallsyms] [k] blkdev_write_iter
+ 20.16% 1.65% fio [kernel.kallsyms] [k] raid10_read_request
+ 18.27% 0.01% fio [kernel.kallsyms] [k] __generic_file_write_iter
+ 18.02% 3.63% fio [kernel.kallsyms] [k] brd_submit_bio
+ 17.86% 0.02% fio [kernel.kallsyms] [k] generic_file_direct_write
+ 17.08% 11.16% fio [kernel.kallsyms] [k] read_balance
+ 16.24% 0.33% fio [kernel.kallsyms] [k] raid10_unplug
+ 16.02% 0.01% fio [kernel.kallsyms] [k] blk_finish_plug
+ 16.02% 0.01% fio [kernel.kallsyms] [k] flush_plug_callbacks
+ 14.25% 0.26% fio [kernel.kallsyms] [k] raid10_end_read_request
+ 12.77% 1.98% fio [kernel.kallsyms] [k] allow_barrier
+ 11.74% 0.40% fio [kernel.kallsyms] [k] raid_end_bio_io
+ 10.21% 1.99% fio [kernel.kallsyms] [k] raid10_write_one_disk
+ 8.85% 1.52% fio [kernel.kallsyms] [k] raid10_end_write_request
8.06% 6.43% fio [kernel.kallsyms] [k] wait_barrier_nolock
..
Thanks,
Ali
.
