block: sbitmap related lockdep warning

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

Just found there is sbmitmap related lockdep warning, not take a close
look yet, maybe
it is caused by recent sbitmap change.

[1] test
- modprobe null_blk queue_mode=2 nr_devices=4 shared_tags=1
submit_queues=1 hw_queue_depth=1
- then run fio on the 4 null_blk devices

[2] lockdep warning
[  100.967642] ================start test sanity/001================
[  101.238280] null: module loaded
[  106.093735]
[  106.094012] =====================================================
[  106.094854] WARNING: SOFTIRQ-safe -> SOFTIRQ-unsafe lock order detected
[  106.095759] 4.20.0-rc3_5d2ee7122c73_for-next+ #1 Not tainted
[  106.096551] -----------------------------------------------------
[  106.097386] fio/1043 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire:
[  106.098231] 000000004c43fa71
(&(&sb->map[i].swap_lock)->rlock){+.+.}, at: sbitmap_get+0xd5/0x22c
[  106.099431]
[  106.099431] and this task is already holding:
[  106.100229] 000000007eec8b2f
(&(&hctx->dispatch_wait_lock)->rlock){....}, at:
blk_mq_dispatch_rq_list+0x4c1/0xd7c
[  106.101630] which would create a new lock dependency:
[  106.102326]  (&(&hctx->dispatch_wait_lock)->rlock){....} ->
(&(&sb->map[i].swap_lock)->rlock){+.+.}
[  106.103553]
[  106.103553] but this new dependency connects a SOFTIRQ-irq-safe lock:
[  106.104580]  (&sbq->ws[i].wait){..-.}
[  106.104582]
[  106.104582] ... which became SOFTIRQ-irq-safe at:
[  106.105751]   _raw_spin_lock_irqsave+0x4b/0x82
[  106.106284]   __wake_up_common_lock+0x119/0x1b9
[  106.106825]   sbitmap_queue_wake_up+0x33f/0x383
[  106.107456]   sbitmap_queue_clear+0x4c/0x9a
[  106.108046]   __blk_mq_free_request+0x188/0x1d3
[  106.108581]   blk_mq_free_request+0x23b/0x26b
[  106.109102]   scsi_end_request+0x345/0x5d7
[  106.109587]   scsi_io_completion+0x4b5/0x8f0
[  106.110099]   scsi_finish_command+0x412/0x456
[  106.110615]   scsi_softirq_done+0x23f/0x29b
[  106.111115]   blk_done_softirq+0x2a7/0x2e6
[  106.111608]   __do_softirq+0x360/0x6ad
[  106.112062]   run_ksoftirqd+0x2f/0x5b
[  106.112499]   smpboot_thread_fn+0x3a5/0x3db
[  106.113000]   kthread+0x1d4/0x1e4
[  106.113457]   ret_from_fork+0x3a/0x50
[  106.113969]
[  106.113969] to a SOFTIRQ-irq-unsafe lock:
[  106.114672]  (&(&sb->map[i].swap_lock)->rlock){+.+.}
[  106.114674]
[  106.114674] ... which became SOFTIRQ-irq-unsafe at:
[  106.116000] ...
[  106.116003]   _raw_spin_lock+0x33/0x64
[  106.116676]   sbitmap_get+0xd5/0x22c
[  106.117134]   __sbitmap_queue_get+0xe8/0x177
[  106.117731]   __blk_mq_get_tag+0x1e6/0x22d
[  106.118286]   blk_mq_get_tag+0x1db/0x6e4
[  106.118756]   blk_mq_get_driver_tag+0x161/0x258
[  106.119383]   blk_mq_dispatch_rq_list+0x28e/0xd7c
[  106.120043]   blk_mq_do_dispatch_sched+0x23a/0x287
[  106.120607]   blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.121234]   __blk_mq_run_hw_queue+0x137/0x17e
[  106.121781]   __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.122366]   blk_mq_run_hw_queue+0x151/0x187
[  106.122887]   blk_mq_sched_insert_requests+0x13f/0x175
[  106.123492]   blk_mq_flush_plug_list+0x7d6/0x81b
[  106.124042]   blk_flush_plug_list+0x392/0x3d7
[  106.124557]   blk_finish_plug+0x37/0x4f
[  106.125019]   read_pages+0x3ef/0x430
[  106.125446]   __do_page_cache_readahead+0x18e/0x2fc
[  106.126027]   force_page_cache_readahead+0x121/0x133
[  106.126621]   page_cache_sync_readahead+0x35f/0x3bb
[  106.127229]   generic_file_buffered_read+0x410/0x1860
[  106.127932]   __vfs_read+0x319/0x38f
[  106.128415]   vfs_read+0xd2/0x19a
[  106.128817]   ksys_read+0xb9/0x135
[  106.129225]   do_syscall_64+0x140/0x385
[  106.129684]   entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.130292]
[  106.130292] other info that might help us debug this:
[  106.130292]
[  106.131226] Chain exists of:
[  106.131226]   &sbq->ws[i].wait -->
&(&hctx->dispatch_wait_lock)->rlock -->
&(&sb->map[i].swap_lock)->rlock
[  106.131226]
[  106.132865]  Possible interrupt unsafe locking scenario:
[  106.132865]
[  106.133659]        CPU0                    CPU1
[  106.134194]        ----                    ----
[  106.134733]   lock(&(&sb->map[i].swap_lock)->rlock);
[  106.135318]                                local_irq_disable();
[  106.136014]                                lock(&sbq->ws[i].wait);
[  106.136747]
lock(&(&hctx->dispatch_wait_lock)->rlock);
[  106.137742]   <Interrupt>
[  106.138110]     lock(&sbq->ws[i].wait);
[  106.138625]
[  106.138625]  *** DEADLOCK ***
[  106.138625]
[  106.139430] 3 locks held by fio/1043:
[  106.139947]  #0: 0000000076ff0fd9 (rcu_read_lock){....}, at:
hctx_lock+0x29/0xe8
[  106.140813]  #1: 000000002feb1016 (&sbq->ws[i].wait){..-.}, at:
blk_mq_dispatch_rq_list+0x4ad/0xd7c
[  106.141877]  #2: 000000007eec8b2f
(&(&hctx->dispatch_wait_lock)->rlock){....}, at:
blk_mq_dispatch_rq_list+0x4c1/0xd7c
[  106.143267]
[  106.143267] the dependencies between SOFTIRQ-irq-safe lock and the
holding lock:
[  106.144351]  -> (&sbq->ws[i].wait){..-.} ops: 82 {
[  106.144926]     IN-SOFTIRQ-W at:
[  106.145314]                       _raw_spin_lock_irqsave+0x4b/0x82
[  106.146042]                       __wake_up_common_lock+0x119/0x1b9
[  106.146785]                       sbitmap_queue_wake_up+0x33f/0x383
[  106.147567]                       sbitmap_queue_clear+0x4c/0x9a
[  106.148379]                       __blk_mq_free_request+0x188/0x1d3
[  106.149148]                       blk_mq_free_request+0x23b/0x26b
[  106.149864]                       scsi_end_request+0x345/0x5d7
[  106.150546]                       scsi_io_completion+0x4b5/0x8f0
[  106.151367]                       scsi_finish_command+0x412/0x456
[  106.152157]                       scsi_softirq_done+0x23f/0x29b
[  106.152855]                       blk_done_softirq+0x2a7/0x2e6
[  106.153537]                       __do_softirq+0x360/0x6ad
[  106.154280]                       run_ksoftirqd+0x2f/0x5b
[  106.155020]                       smpboot_thread_fn+0x3a5/0x3db
[  106.155828]                       kthread+0x1d4/0x1e4
[  106.156526]                       ret_from_fork+0x3a/0x50
[  106.157267]     INITIAL USE at:
[  106.157713]                      _raw_spin_lock_irqsave+0x4b/0x82
[  106.158542]                      prepare_to_wait_exclusive+0xa8/0x215
[  106.159421]                      blk_mq_get_tag+0x34f/0x6e4
[  106.160186]                      blk_mq_get_request+0x48e/0xaef
[  106.160997]                      blk_mq_make_request+0x27e/0xbd2
[  106.161828]                      generic_make_request+0x4d1/0x873
[  106.162661]                      submit_bio+0x20c/0x253
[  106.163379]                      mpage_bio_submit+0x44/0x4b
[  106.164142]                      mpage_readpages+0x3c2/0x407
[  106.164919]                      read_pages+0x13a/0x430
[  106.165633]                      __do_page_cache_readahead+0x18e/0x2fc
[  106.166530]                      force_page_cache_readahead+0x121/0x133
[  106.167439]                      page_cache_sync_readahead+0x35f/0x3bb
[  106.168337]                      generic_file_buffered_read+0x410/0x1860
[  106.169255]                      __vfs_read+0x319/0x38f
[  106.169977]                      vfs_read+0xd2/0x19a
[  106.170662]                      ksys_read+0xb9/0x135
[  106.171356]                      do_syscall_64+0x140/0x385
[  106.172120]                      entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.173051]   }
[  106.173308]   ... key      at: [<ffffffff85094600>] __key.26481+0x0/0x40
[  106.174219]   ... acquired at:
[  106.174646]    _raw_spin_lock+0x33/0x64
[  106.175183]    blk_mq_dispatch_rq_list+0x4c1/0xd7c
[  106.175843]    blk_mq_do_dispatch_sched+0x23a/0x287
[  106.176518]    blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.177262]    __blk_mq_run_hw_queue+0x137/0x17e
[  106.177900]    __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.178591]    blk_mq_run_hw_queue+0x151/0x187
[  106.179207]    blk_mq_sched_insert_requests+0x13f/0x175
[  106.179926]    blk_mq_flush_plug_list+0x7d6/0x81b
[  106.180571]    blk_flush_plug_list+0x392/0x3d7
[  106.181187]    blk_finish_plug+0x37/0x4f
[  106.181737]    __se_sys_io_submit+0x171/0x304
[  106.182346]    do_syscall_64+0x140/0x385
[  106.182895]    entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.183607]
[  106.183830] -> (&(&hctx->dispatch_wait_lock)->rlock){....} ops: 1 {
[  106.184691]    INITIAL USE at:
[  106.185119]                    _raw_spin_lock+0x33/0x64
[  106.185838]                    blk_mq_dispatch_rq_list+0x4c1/0xd7c
[  106.186697]                    blk_mq_do_dispatch_sched+0x23a/0x287
[  106.187551]                    blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.188481]                    __blk_mq_run_hw_queue+0x137/0x17e
[  106.189307]                    __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.190189]                    blk_mq_run_hw_queue+0x151/0x187
[  106.190989]                    blk_mq_sched_insert_requests+0x13f/0x175
[  106.191902]                    blk_mq_flush_plug_list+0x7d6/0x81b
[  106.192739]                    blk_flush_plug_list+0x392/0x3d7
[  106.193535]                    blk_finish_plug+0x37/0x4f
[  106.194269]                    __se_sys_io_submit+0x171/0x304
[  106.195059]                    do_syscall_64+0x140/0x385
[  106.195794]                    entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.196705]  }
[  106.196950]  ... key      at: [<ffffffff84880620>] __key.51231+0x0/0x40
[  106.197853]  ... acquired at:
[  106.198270]    lock_acquire+0x280/0x2f3
[  106.198806]    _raw_spin_lock+0x33/0x64
[  106.199337]    sbitmap_get+0xd5/0x22c
[  106.199850]    __sbitmap_queue_get+0xe8/0x177
[  106.200450]    __blk_mq_get_tag+0x1e6/0x22d
[  106.201035]    blk_mq_get_tag+0x1db/0x6e4
[  106.201589]    blk_mq_get_driver_tag+0x161/0x258
[  106.202237]    blk_mq_dispatch_rq_list+0x5b9/0xd7c
[  106.202902]    blk_mq_do_dispatch_sched+0x23a/0x287
[  106.203572]    blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.204316]    __blk_mq_run_hw_queue+0x137/0x17e
[  106.204956]    __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.205649]    blk_mq_run_hw_queue+0x151/0x187
[  106.206269]    blk_mq_sched_insert_requests+0x13f/0x175
[  106.206997]    blk_mq_flush_plug_list+0x7d6/0x81b
[  106.207644]    blk_flush_plug_list+0x392/0x3d7
[  106.208264]    blk_finish_plug+0x37/0x4f
[  106.208814]    __se_sys_io_submit+0x171/0x304
[  106.209415]    do_syscall_64+0x140/0x385
[  106.209965]    entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.210684]
[  106.210904]
[  106.210904] the dependencies between the lock to be acquired
[  106.210905]  and SOFTIRQ-irq-unsafe lock:
[  106.212541] -> (&(&sb->map[i].swap_lock)->rlock){+.+.} ops: 1969 {
[  106.213393]    HARDIRQ-ON-W at:
[  106.213840]                     _raw_spin_lock+0x33/0x64
[  106.214570]                     sbitmap_get+0xd5/0x22c
[  106.215282]                     __sbitmap_queue_get+0xe8/0x177
[  106.216086]                     __blk_mq_get_tag+0x1e6/0x22d
[  106.216876]                     blk_mq_get_tag+0x1db/0x6e4
[  106.217627]                     blk_mq_get_driver_tag+0x161/0x258
[  106.218465]                     blk_mq_dispatch_rq_list+0x28e/0xd7c
[  106.219326]                     blk_mq_do_dispatch_sched+0x23a/0x287
[  106.220198]                     blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.221138]                     __blk_mq_run_hw_queue+0x137/0x17e
[  106.221975]                     __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.222874]                     blk_mq_run_hw_queue+0x151/0x187
[  106.223686]                     blk_mq_sched_insert_requests+0x13f/0x175
[  106.224597]                     blk_mq_flush_plug_list+0x7d6/0x81b
[  106.225444]                     blk_flush_plug_list+0x392/0x3d7
[  106.226255]                     blk_finish_plug+0x37/0x4f
[  106.227006]                     read_pages+0x3ef/0x430
[  106.227717]                     __do_page_cache_readahead+0x18e/0x2fc
[  106.228595]                     force_page_cache_readahead+0x121/0x133
[  106.229491]                     page_cache_sync_readahead+0x35f/0x3bb
[  106.230373]                     generic_file_buffered_read+0x410/0x1860
[  106.231277]                     __vfs_read+0x319/0x38f
[  106.231986]                     vfs_read+0xd2/0x19a
[  106.232666]                     ksys_read+0xb9/0x135
[  106.233350]                     do_syscall_64+0x140/0x385
[  106.234097]                     entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.235012]    SOFTIRQ-ON-W at:
[  106.235460]                     _raw_spin_lock+0x33/0x64
[  106.236195]                     sbitmap_get+0xd5/0x22c
[  106.236913]                     __sbitmap_queue_get+0xe8/0x177
[  106.237715]                     __blk_mq_get_tag+0x1e6/0x22d
[  106.238488]                     blk_mq_get_tag+0x1db/0x6e4
[  106.239244]                     blk_mq_get_driver_tag+0x161/0x258
[  106.240079]                     blk_mq_dispatch_rq_list+0x28e/0xd7c
[  106.240937]                     blk_mq_do_dispatch_sched+0x23a/0x287
[  106.241806]                     blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.242751]                     __blk_mq_run_hw_queue+0x137/0x17e
[  106.243579]                     __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.244469]                     blk_mq_run_hw_queue+0x151/0x187
[  106.245277]                     blk_mq_sched_insert_requests+0x13f/0x175
[  106.246191]                     blk_mq_flush_plug_list+0x7d6/0x81b
[  106.247044]                     blk_flush_plug_list+0x392/0x3d7
[  106.247859]                     blk_finish_plug+0x37/0x4f
[  106.248749]                     read_pages+0x3ef/0x430
[  106.249463]                     __do_page_cache_readahead+0x18e/0x2fc
[  106.250357]                     force_page_cache_readahead+0x121/0x133
[  106.251263]                     page_cache_sync_readahead+0x35f/0x3bb
[  106.252157]                     generic_file_buffered_read+0x410/0x1860
[  106.253084]                     __vfs_read+0x319/0x38f
[  106.253808]                     vfs_read+0xd2/0x19a
[  106.254488]                     ksys_read+0xb9/0x135
[  106.255186]                     do_syscall_64+0x140/0x385
[  106.255943]                     entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.256867]    INITIAL USE at:
[  106.257300]                    _raw_spin_lock+0x33/0x64
[  106.258033]                    sbitmap_get+0xd5/0x22c
[  106.258747]                    __sbitmap_queue_get+0xe8/0x177
[  106.259542]                    __blk_mq_get_tag+0x1e6/0x22d
[  106.260320]                    blk_mq_get_tag+0x1db/0x6e4
[  106.261072]                    blk_mq_get_driver_tag+0x161/0x258
[  106.261902]                    blk_mq_dispatch_rq_list+0x28e/0xd7c
[  106.262762]                    blk_mq_do_dispatch_sched+0x23a/0x287
[  106.263626]                    blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.264571]                    __blk_mq_run_hw_queue+0x137/0x17e
[  106.265409]                    __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.266302]                    blk_mq_run_hw_queue+0x151/0x187
[  106.267111]                    blk_mq_sched_insert_requests+0x13f/0x175
[  106.268028]                    blk_mq_flush_plug_list+0x7d6/0x81b
[  106.268878]                    blk_flush_plug_list+0x392/0x3d7
[  106.269694]                    blk_finish_plug+0x37/0x4f
[  106.270432]                    read_pages+0x3ef/0x430
[  106.271139]                    __do_page_cache_readahead+0x18e/0x2fc
[  106.272040]                    force_page_cache_readahead+0x121/0x133
[  106.272932]                    page_cache_sync_readahead+0x35f/0x3bb
[  106.273811]                    generic_file_buffered_read+0x410/0x1860
[  106.274709]                    __vfs_read+0x319/0x38f
[  106.275407]                    vfs_read+0xd2/0x19a
[  106.276074]                    ksys_read+0xb9/0x135
[  106.276764]                    do_syscall_64+0x140/0x385
[  106.277500]                    entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.278417]  }
[  106.278676]  ... key      at: [<ffffffff85094640>] __key.26212+0x0/0x40
[  106.279586]  ... acquired at:
[  106.280026]    lock_acquire+0x280/0x2f3
[  106.280559]    _raw_spin_lock+0x33/0x64
[  106.281101]    sbitmap_get+0xd5/0x22c
[  106.281610]    __sbitmap_queue_get+0xe8/0x177
[  106.282221]    __blk_mq_get_tag+0x1e6/0x22d
[  106.282809]    blk_mq_get_tag+0x1db/0x6e4
[  106.283368]    blk_mq_get_driver_tag+0x161/0x258
[  106.284018]    blk_mq_dispatch_rq_list+0x5b9/0xd7c
[  106.284685]    blk_mq_do_dispatch_sched+0x23a/0x287
[  106.285371]    blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.286135]    __blk_mq_run_hw_queue+0x137/0x17e
[  106.286806]    __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.287515]    blk_mq_run_hw_queue+0x151/0x187
[  106.288149]    blk_mq_sched_insert_requests+0x13f/0x175
[  106.289041]    blk_mq_flush_plug_list+0x7d6/0x81b
[  106.289912]    blk_flush_plug_list+0x392/0x3d7
[  106.290590]    blk_finish_plug+0x37/0x4f
[  106.291238]    __se_sys_io_submit+0x171/0x304
[  106.291864]    do_syscall_64+0x140/0x385
[  106.292534]    entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.293472]
[  106.293708]
[  106.293708] stack backtrace:
[  106.294456] CPU: 0 PID: 1043 Comm: fio Not tainted
4.20.0-rc3_5d2ee7122c73_for-next+ #1
[  106.295695] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
BIOS 1.10.2-2.fc27 04/01/2014
[  106.296872] Call Trace:
[  106.297246]  dump_stack+0xf0/0x191
[  106.297737]  ? show_regs_print_info+0x5/0x5
[  106.298331]  ? print_shortest_lock_dependencies+0x21d/0x245
[  106.299123]  ? print_shortest_lock_dependencies+0x21d/0x245
[  106.300049]  check_usage+0x863/0x8af
[  106.300733]  ? check_usage_forwards+0x237/0x237
[  106.301492]  ? pvclock_read_flags+0x37/0x37
[  106.302097]  ? check_redundant+0x4e/0x4e
[  106.302659]  ? commit_charge+0x570/0xa24
[  106.303221]  ? class_equal+0x11/0x1d
[  106.303737]  ? __bfs+0xff/0x3f6
[  106.304188]  ? lockdep_on+0x1e/0x1e
[  106.304709]  ? check_prevs_add+0x405/0xc15
[  106.305287]  check_prevs_add+0x405/0xc15
[  106.305972]  ? print_irq_inversion_bug.part.17+0x213/0x213
[  106.306974]  ? __pagevec_lru_add+0x13/0x13
[  106.307598]  ? pvclock_read_flags+0x37/0x37
[  106.308210]  ? zap_class+0x33b/0x33b
[  106.308720]  ? kvm_clock_read+0x14/0x23
[  106.309262]  ? kvm_sched_clock_read+0x5/0xd
[  106.309856]  ? __lock_acquire+0x171a/0x185b
[  106.310443]  __lock_acquire+0x171a/0x185b
[  106.311020]  ? debug_show_all_locks+0x354/0x354
[  106.311664]  ? module_flags+0xd7/0xd7
[  106.312190]  ? __bpf_trace_xdp_devmap_xmit+0x15/0x15
[  106.316831]  ? lock_release+0xa1/0x5b5
[  106.317395]  ? stack_access_ok+0x57/0x7c
[  106.317959]  ? check_flags+0x20a/0x20a
[  106.318488]  ? __rcu_read_unlock+0x137/0x183
[  106.319093]  ? rcu_read_unlock_special+0xb4/0xb4
[  106.319741]  ? __free_insn_slot+0x362/0x362
[  106.320327]  ? rcu_softirq_qs+0x18/0x18
[  106.320875]  ? is_bpf_text_address+0xd2/0xda
[  106.321468]  ? kernel_text_address+0x78/0x8c
[  106.322069]  ? __kernel_text_address+0x1f/0x2a
[  106.322693]  ? __list_add_valid+0x42/0x8f
[  106.323257]  lock_acquire+0x280/0x2f3
[  106.323777]  ? sbitmap_get+0xd5/0x22c
[  106.324294]  ? lock_downgrade+0x338/0x338
[  106.324860]  ? check_prevs_add+0xb3e/0xc15
[  106.325436]  ? print_irq_inversion_bug.part.17+0x213/0x213
[  106.326201]  ? find_next_zero_bit+0x27/0x88
[  106.326792]  _raw_spin_lock+0x33/0x64
[  106.327305]  ? sbitmap_get+0xd5/0x22c
[  106.327824]  sbitmap_get+0xd5/0x22c
[  106.328321]  __sbitmap_queue_get+0xe8/0x177
[  106.328911]  ? sbitmap_bitmap_show+0x2c2/0x2c2
[  106.329527]  ? kvm_sched_clock_read+0x5/0xd
[  106.330123]  __blk_mq_get_tag+0x1e6/0x22d
[  106.330692]  ? blk_mq_unique_tag+0x40/0x40
[  106.331265]  ? debug_show_all_locks+0x354/0x354
[  106.331906]  blk_mq_get_tag+0x1db/0x6e4
[  106.332447]  ? __blk_mq_tag_idle+0x44/0x44
[  106.333026]  ? wait_woken+0x152/0x152
[  106.333541]  ? pvclock_clocksource_read+0x121/0x205
[  106.334222]  ? pvclock_read_flags+0x37/0x37
[  106.334810]  ? print_irqtrace_events+0x101/0x101
[  106.335454]  ? get_lock_stats+0x23/0x51
[  106.336000]  ? lock_contended+0x65a/0x65a
[  106.336556]  ? do_raw_spin_trylock+0x156/0x1a2
[  106.337184]  blk_mq_get_driver_tag+0x161/0x258
[  106.337808]  ? blk_mq_dequeue_from_ctx+0x4c5/0x4c5
[  106.338474]  ? debug_show_all_locks+0x354/0x354
[  106.339112]  blk_mq_dispatch_rq_list+0x5b9/0xd7c
[  106.339767]  ? blk_mq_make_request+0xbd2/0xbd2
[  106.340383]  ? pvclock_clocksource_read+0x121/0x205
[  106.341065]  ? pvclock_read_flags+0x37/0x37
[  106.341661]  ? kvm_clock_read+0x14/0x23
[  106.342201]  ? kvm_clock_read+0x14/0x23
[  106.342741]  ? kvm_sched_clock_read+0x5/0xd
[  106.343322]  ? check_chain_key+0x150/0x1aa
[  106.343904]  ? check_flags+0x20a/0x20a
[  106.344428]  ? deadline_remove_request+0x1e3/0x235
[  106.345100]  ? deadline_next_request+0x77/0x77
[  106.345722]  ? do_raw_spin_unlock+0x144/0x179
[  106.346326]  ? do_raw_spin_trylock+0x1a2/0x1a2
[  106.346955]  ? preempt_count_sub+0x14/0xc4
[  106.347534]  ? _raw_spin_unlock+0x2e/0x40
[  106.348097]  ? dd_dispatch_request+0x4f9/0x540
[  106.348720]  ? deadline_fifo_request+0x159/0x159
[  106.349359]  ? pvclock_clocksource_read+0x121/0x205
[  106.350035]  ? mark_lock+0x11d/0x89d
[  106.350536]  ? pvclock_read_flags+0x37/0x37
[  106.351124]  ? print_irqtrace_events+0x101/0x101
[  106.351770]  ? rb_next_postorder+0x59/0x59
[  106.352342]  ? kvm_clock_read+0x14/0x23
[  106.352881]  ? kvm_sched_clock_read+0x5/0xd
[  106.353460]  ? check_chain_key+0x150/0x1aa
[  106.354039]  ? __lock_acquire+0xf3d/0x185b
[  106.354616]  ? debug_show_all_locks+0x354/0x354
[  106.355250]  ? __lock_acquire+0xf3d/0x185b
[  106.355829]  blk_mq_do_dispatch_sched+0x23a/0x287
[  106.356482]  ? blk_mq_sched_free_hctx_data+0xaa/0xaa
[  106.357173]  ? kvm_clock_read+0x14/0x23
[  106.357711]  ? kvm_sched_clock_read+0x5/0xd
[  106.358290]  ? check_chain_key+0x150/0x1aa
[  106.358871]  blk_mq_sched_dispatch_requests+0x379/0x3fc
[  106.359592]  ? blk_mq_sched_restart+0x2f/0x2f
[  106.360204]  ? lock_acquire+0x280/0x2f3
[  106.360743]  ? hctx_lock+0x29/0xe8
[  106.361224]  ? lock_downgrade+0x338/0x338
[  106.361792]  ? rcu_dynticks_curr_cpu_in_eqs+0xa9/0xdd
[  106.362488]  ? rcu_softirq_qs+0x18/0x18
[  106.363037]  __blk_mq_run_hw_queue+0x137/0x17e
[  106.363658]  ? hctx_lock+0xe8/0xe8
[  106.364143]  __blk_mq_delay_run_hw_queue+0x80/0x25f
[  106.364828]  blk_mq_run_hw_queue+0x151/0x187
[  106.365421]  ? blk_mq_run_work_fn+0x26/0x26
[  106.366007]  ? set_page_dirty_lock+0xd0/0x109
[  106.366611]  ? set_page_dirty+0x2dd/0x2dd
[  106.367177]  blk_mq_sched_insert_requests+0x13f/0x175
[  106.367881]  ? blk_mq_sched_insert_request+0x357/0x357
[  106.368587]  ? __lock_is_held+0x2a/0x87
[  106.369138]  blk_mq_flush_plug_list+0x7d6/0x81b
[  106.369774]  ? blkdev_direct_IO+0x69f/0x8b6
[  106.370357]  ? blk_mq_insert_requests+0x3e3/0x3e3
[  106.371013]  ? aio_poll+0x968/0x968
[  106.371503]  ? __blkdev_direct_IO_simple+0x8bf/0x8bf
[  106.372192]  ? __ia32_sys_dup3+0x44/0x44
[  106.372747]  ? debug_show_all_locks+0x354/0x354
[  106.373377]  ? preempt_count_sub+0x14/0xc4
[  106.373952]  ? _raw_spin_unlock_irqrestore+0x58/0x6b
[  106.374639]  ? rcu_lockdep_current_cpu_online+0x100/0x147
[  106.375388]  ? rcu_pm_notify+0x64/0x64
[  106.375914]  ? __lock_is_held+0x2a/0x87
[  106.376453]  ? aio_read+0x206/0x271
[  106.376948]  ? rcu_read_lock_sched_held+0x6e/0x74
[  106.377598]  ? kfree+0xaa/0x277
[  106.378050]  ? aio_read+0x206/0x271
[  106.378541]  ? kvm_clock_read+0x14/0x23
[  106.379078]  ? kvm_sched_clock_read+0x5/0xd
[  106.379661]  ? check_chain_key+0x150/0x1aa
[  106.380243]  ? ___might_sleep+0x155/0x338
[  106.380811]  blk_flush_plug_list+0x392/0x3d7
[  106.381406]  ? blk_init_request_from_bio+0xa5/0xa5
[  106.382077]  ? io_submit_one+0x686/0x8f1
[  106.382622]  ? io_submit_one+0x686/0x8f1
[  106.383175]  ? aio_fsync+0x1eb/0x1eb
[  106.383691]  ? check_flags+0x20a/0x20a
[  106.384216]  ? ___might_sleep+0x155/0x338
[  106.384781]  ? __schedule_bug+0x111/0x111
[  106.385346]  blk_finish_plug+0x37/0x4f
[  106.385878]  __se_sys_io_submit+0x171/0x304
[  106.386461]  ? io_submit_one+0x8f1/0x8f1
[  106.387017]  ? lockdep_hardirqs_on+0x26b/0x278
[  106.387632]  ? trace_hardirqs_on+0x169/0x19e
[  106.388234]  ? __bpf_trace_preemptirq_template+0x5/0x5
[  106.388955]  ? do_syscall_64+0x140/0x385
[  106.389500]  do_syscall_64+0x140/0x385
[  106.390030]  ? syscall_return_slowpath+0x291/0x291
[  106.390697]  ? trace_hardirqs_off+0x19e/0x19e
[  106.391302]  ? prepare_exit_to_usermode+0x1c4/0x1c4
[  106.391985]  ? lockdep_sys_exit+0x16/0x8d
[  106.392545]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  106.393205]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
[  106.393910] RIP: 0033:0x7f8dbd04c687
[  106.394411] Code: 00 00 00 49 83 38 00 75 ed 49 83 78 08 00 75 e6
8b 47 0c 39 47 08 75 de 31 c0 c3 0f 1f 84 00 00 00 00 00 b8 d1 00 00
00 0f 05 <c3> 0f 1f 84 00 00 00 00 00 b8 d2 00 00 00 0f 05 c3 0f 1f 84
00 00
[  106.396974] RSP: 002b:00007fff21011358 EFLAGS: 00000202 ORIG_RAX:
00000000000000d1
[  106.398018] RAX: ffffffffffffffda RBX: 00007f8d989f5670 RCX: 00007f8dbd04c687
[  106.399001] RDX: 000000000207b780 RSI: 0000000000000001 RDI: 00007f8dbe52d000
[  106.399989] RBP: 0000000000000000 R08: 0000000000000001 R09: 000000000207b4e0
[  106.400976] R10: 000000000000000c R11: 0000000000000202 R12: 00007f8d989f5670
[  106.401951] R13: 0000000000000000 R14: 000000000207b7b0 R15: 000000000205b7c0
[  204.246336] null: module loaded
[  294.406501] null: module loaded
[  396.936354] null: module loaded
[  497.202198] ================end test sanity/001: (NO_HANG, 0)================


Thanks,
Ming Lei



[Index of Archives]     [Linux RAID]     [Linux SCSI]     [Linux ATA RAID]     [IDE]     [Linux Wireless]     [Linux Kernel]     [ATH6KL]     [Linux Bluetooth]     [Linux Netdev]     [Kernel Newbies]     [Security]     [Git]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Device Mapper]

  Powered by Linux