On Tue, 2023-01-10 at 15:10 +0100, Paolo Bonzini wrote:
> On 1/10/23 13:55, David Woodhouse wrote:
> > > However, I
> > > completely forgot the sev_lock_vcpus_for_migration case, which is the
> > > exception that... well, disproves the rule.
> > >
> > But because it's an exception and rarely happens in practice, lockdep
> > didn't notice and keep me honest sooner? Can we take them in that order
> > just for fun at startup, to make sure lockdep knows?
>
> Sure, why not. Out of curiosity, is this kind of "priming" a thing
> elsewhere in the kernel
I did this:
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -461,6 +461,11 @@ void *kvm_mmu_memory_cache_alloc(struct kvm_mmu_memory_cache *mc)
static void kvm_vcpu_init(struct kvm_vcpu *vcpu, struct kvm *kvm, unsigned id)
{
mutex_init(&vcpu->mutex);
+
+ /* Ensure that lockdep knows vcpu->mutex is taken *inside* kvm->lock */
+ mutex_lock(&vcpu->mutex);
+ mutex_unlock(&vcpu->mutex);
+
vcpu->cpu = -1;
vcpu->kvm = kvm;
vcpu->vcpu_id = id;
What I got when I ran xen_shinfo_test was... not what I expected:
[13890.148203] ======================================================
[13890.148205] WARNING: possible circular locking dependency detected
[13890.148207] 6.1.0-rc4+ #1024 Tainted: G I E
[13890.148209] ------------------------------------------------------
[13890.148210] xen_shinfo_test/13326 is trying to acquire lock:
[13890.148212] ffff888107d493b0 (&gpc->lock){....}-{2:2}, at: kvm_xen_update_runstate_guest+0xf2/0x4e0 [kvm]
[13890.148285]
but task is already holding lock:
[13890.148287] ffff88887f671718 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x84/0x7c0
[13890.148295]
which lock already depends on the new lock.
[13890.148296]
the existing dependency chain (in reverse order) is:
[13890.148298]
-> #4 (&rq->__lock){-.-.}-{2:2}:
[13890.148301] __lock_acquire+0x4b4/0x940
[13890.148306] lock_acquire.part.0+0xa8/0x210
[13890.148309] _raw_spin_lock_nested+0x35/0x50
[13890.148313] raw_spin_rq_lock_nested+0x23/0x30
[13890.148318] task_fork_fair+0x45/0x170
[13890.148322] sched_cgroup_fork+0x11a/0x160
[13890.148325] copy_process+0x1139/0x1950
[13890.148329] kernel_clone+0x9b/0x390
[13890.148332] user_mode_thread+0x5b/0x80
[13890.148335] rest_init+0x1e/0x170
[13890.148338] arch_call_rest_init+0xa/0x14
[13890.148342] start_kernel+0x647/0x670
[13890.148345] secondary_startup_64_no_verify+0xd3/0xdb
[13890.148349]
-> #3 (&p->pi_lock){-.-.}-{2:2}:
[13890.148352] __lock_acquire+0x4b4/0x940
[13890.148355] lock_acquire.part.0+0xa8/0x210
[13890.148357] __raw_spin_lock_irqsave+0x44/0x60
[13890.148360] try_to_wake_up+0x69/0x360
[13890.148362] create_worker+0x129/0x1a0
[13890.148366] workqueue_init+0x14b/0x1b0
[13890.148371] kernel_init_freeable+0x95/0x122
[13890.148373] kernel_init+0x16/0x130
[13890.148375] ret_from_fork+0x22/0x30
[13890.148378]
-> #2 (&pool->lock){-.-.}-{2:2}:
[13890.148381] __lock_acquire+0x4b4/0x940
[13890.148384] lock_acquire.part.0+0xa8/0x210
[13890.148386] _raw_spin_lock+0x2f/0x40
[13890.148389] __queue_work+0x1a1/0x490
[13890.148391] queue_work_on+0x75/0x80
[13890.148394] percpu_ref_put_many.constprop.0+0xea/0xf0
[13890.148398] __mem_cgroup_uncharge_list+0x7d/0xa0
[13890.148401] release_pages+0x15b/0x590
[13890.148404] folio_batch_move_lru+0xd3/0x150
[13890.148407] lru_add_drain_cpu+0x1ce/0x270
[13890.148410] lru_add_drain+0x77/0x140
[13890.148413] do_wp_page+0x342/0x3a0
[13890.148417] __handle_mm_fault+0x3a1/0x690
[13890.148421] handle_mm_fault+0x113/0x3b0
[13890.148424] do_user_addr_fault+0x1d8/0x6b0
[13890.148427] exc_page_fault+0x6a/0xe0
[13890.148429] asm_exc_page_fault+0x22/0x30
[13890.148432]
-> #1 (lock#4){+.+.}-{2:2}:
[13890.148436] __lock_acquire+0x4b4/0x940
[13890.148439] lock_acquire.part.0+0xa8/0x210
[13890.148441] folio_mark_accessed+0x8d/0x1a0
[13890.148444] kvm_release_page_clean+0x89/0xb0 [kvm]
[13890.148485] hva_to_pfn_retry+0x296/0x2d0 [kvm]
[13890.148524] __kvm_gpc_refresh+0x18e/0x310 [kvm]
[13890.148562] kvm_xen_hvm_set_attr+0x1f5/0x2f0 [kvm]
[13890.148613] kvm_arch_vm_ioctl+0x9bf/0xd50 [kvm]
[13890.148656] kvm_vm_ioctl+0x5c1/0x7f0 [kvm]
[13890.148693] __x64_sys_ioctl+0x8a/0xc0
[13890.148696] do_syscall_64+0x3b/0x90
[13890.148701] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[13890.148704]
-> #0 (&gpc->lock){....}-{2:2}:
[13890.148708] check_prev_add+0x8f/0xc20
[13890.148710] validate_chain+0x3ba/0x450
[13890.148713] __lock_acquire+0x4b4/0x940
[13890.148715] lock_acquire.part.0+0xa8/0x210
[13890.148717] __raw_read_lock_irqsave+0x7f/0xa0
[13890.148720] kvm_xen_update_runstate_guest+0xf2/0x4e0 [kvm]
[13890.148771] kvm_arch_vcpu_put+0x1d4/0x250 [kvm]
[13890.148814] kvm_sched_out+0x2f/0x50 [kvm]
[13890.148849] prepare_task_switch+0xe7/0x3b0
[13890.148853] __schedule+0x1c9/0x7c0
[13890.148857] schedule+0x5d/0xd0
[13890.148860] xfer_to_guest_mode_handle_work+0x59/0xd0
[13890.148865] vcpu_run+0x328/0x410 [kvm]
[13890.148908] kvm_arch_vcpu_ioctl_run+0x1cd/0x640 [kvm]
[13890.148950] kvm_vcpu_ioctl+0x279/0x700 [kvm]
[13890.148986] __x64_sys_ioctl+0x8a/0xc0
[13890.148989] do_syscall_64+0x3b/0x90
[13890.148993] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[13890.148996]
other info that might help us debug this:
[13890.148997] Chain exists of:
&gpc->lock --> &p->pi_lock --> &rq->__lock
[13890.149002] Possible unsafe locking scenario:
[13890.149003] CPU0 CPU1
[13890.149004] ---- ----
[13890.149005] lock(&rq->__lock);
[13890.149007] lock(&p->pi_lock);
[13890.149009] lock(&rq->__lock);
[13890.149011] lock(&gpc->lock);
[13890.149013]
*** DEADLOCK ***
[13890.149014] 3 locks held by xen_shinfo_test/13326:
[13890.149016] #0: ffff888107d480b0 (&vcpu->mutex){+.+.}-{3:3}, at: kvm_vcpu_ioctl+0x77/0x700 [kvm]
[13890.149057] #1: ffff88887f671718 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x84/0x7c0
[13890.149064] #2: ffffc900017c5860 (&kvm->srcu){....}-{0:0}, at: kvm_arch_vcpu_put+0x2a/0x250 [kvm]
[13890.149109]
stack backtrace:
[13890.149111] CPU: 1 PID: 13326 Comm: xen_shinfo_test Tainted: G I E 6.1.0-rc4+ #1024
[13890.149115] Hardware name: Intel Corporation S2600CW/S2600CW, BIOS SE5C610.86B.01.01.0008.021120151325 02/11/2015
[13890.149116] Call Trace:
[13890.149118] <TASK>
[13890.149121] dump_stack_lvl+0x56/0x73
[13890.149126] check_noncircular+0x102/0x120
[13890.149131] check_prev_add+0x8f/0xc20
[13890.149134] ? validate_chain+0x22a/0x450
[13890.149136] ? add_chain_cache+0x10b/0x2d0
[13890.149140] validate_chain+0x3ba/0x450
[13890.149144] __lock_acquire+0x4b4/0x940
[13890.149148] lock_acquire.part.0+0xa8/0x210
[13890.149151] ? kvm_xen_update_runstate_guest+0xf2/0x4e0 [kvm]
[13890.149204] ? rcu_read_lock_sched_held+0x43/0x70
[13890.149208] ? lock_acquire+0x102/0x140
[13890.149211] __raw_read_lock_irqsave+0x7f/0xa0
[13890.149215] ? kvm_xen_update_runstate_guest+0xf2/0x4e0 [kvm]
[13890.149266] kvm_xen_update_runstate_guest+0xf2/0x4e0 [kvm]
[13890.149316] ? get_kvmclock_ns+0x52/0x90 [kvm]
[13890.149359] ? lock_acquire+0x102/0x140
[13890.149363] kvm_arch_vcpu_put+0x1d4/0x250 [kvm]
[13890.149407] kvm_sched_out+0x2f/0x50 [kvm]
[13890.149444] prepare_task_switch+0xe7/0x3b0
[13890.149449] __schedule+0x1c9/0x7c0
[13890.149454] schedule+0x5d/0xd0
[13890.149458] xfer_to_guest_mode_handle_work+0x59/0xd0
[13890.149463] vcpu_run+0x328/0x410 [kvm]
[13890.149507] kvm_arch_vcpu_ioctl_run+0x1cd/0x640 [kvm]
[13890.149551] kvm_vcpu_ioctl+0x279/0x700 [kvm]
[13890.149588] ? exc_page_fault+0xdb/0xe0
[13890.149591] ? _raw_spin_unlock_irq+0x34/0x50
[13890.149595] ? do_setitimer+0x190/0x1e0
[13890.149600] __x64_sys_ioctl+0x8a/0xc0
[13890.149604] do_syscall_64+0x3b/0x90
[13890.149607] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[13890.149611] RIP: 0033:0x7fa394a3fd1b
[13890.149614] Code: 73 01 c3 48 8b 0d 05 a1 1b 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 a0 1b 00 f7 d8 64 89 01 48
[13890.149617] RSP: 002b:00007ffe7f86c0a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[13890.149620] RAX: ffffffffffffffda RBX: 00007fa394e01000 RCX: 00007fa394a3fd1b
[13890.149622] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000007
[13890.149624] RBP: 00007fa394dc96c0 R08: 000000000041827e R09: 0000000000418234
[13890.149626] R10: 00007fa394bb936b R11: 0000000000000246 R12: 00000000018f9800
[13890.149628] R13: 000000000000000a R14: 00007fa394dffff1 R15: 00000000018f72a0
[13890.149632] </TASK>
Attachment:
smime.p7s
Description: S/MIME cryptographic signature
