Re: Bad performance since 5.9-rc1

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, 19 Nov 2020 04:05:26 +0100
Zdenek Kaspar <zkaspar82@xxxxxxxxx> wrote:

> Hi,
> 
> in my initial report (https://marc.info/?l=kvm&m=160502183220080&w=2 -
> now fixed by c887c9b9ca62c051d339b1c7b796edf2724029ed) I saw degraded
> performance going back somewhere between v5.8 - v5.9-rc1.
> 
> OpenBSD 6.8 (GENERIC.MP) guest performance (time ./test-build.sh)
> good: 0m13.54s real     0m10.51s user     0m10.96s system
> bad : 6m20.07s real    11m42.93s user     0m13.57s system
> 
> bisected to first bad commit: 6b82ef2c9cf18a48726e4bb359aa9014632f6466
> 
> git bisect log:
> # bad: [e47c4aee5bde03e7018f4fde45ba21028a8f8438] KVM: x86/mmu: Rename
> page_header() to to_shadow_page() # good:
> [01c3b2b5cdae39af8dfcf6e40fdf484ae0e812c5] KVM: SVM: Rename
> svm_nested_virtualize_tpr() to nested_svm_virtualize_tpr() git bisect
> start 'e47c4aee5bde' '01c3b2b5cdae' # bad:
> [ebdb292dac7993425c8e31e2c21c9978e914a676] KVM: x86/mmu: Batch zap MMU
> pages when shrinking the slab git bisect bad
> ebdb292dac7993425c8e31e2c21c9978e914a676 # good:
> [fb58a9c345f645f1774dcf6a36fda169253008ae] KVM: x86/mmu: Optimize MMU
> page cache lookup for fully direct MMUs git bisect good
> fb58a9c345f645f1774dcf6a36fda169253008ae # bad:
> [6b82ef2c9cf18a48726e4bb359aa9014632f6466] KVM: x86/mmu: Batch zap MMU
> pages when recycling oldest pages git bisect bad
> 6b82ef2c9cf18a48726e4bb359aa9014632f6466 # good:
> [f95eec9bed76d42194c23153cb1cc8f186bf91cb] KVM: x86/mmu: Don't put
> invalid SPs back on the list of active pages git bisect good
> f95eec9bed76d42194c23153cb1cc8f186bf91cb # first bad commit:
> [6b82ef2c9cf18a48726e4bb359aa9014632f6466] KVM: x86/mmu: Batch zap MMU
> pages when recycling oldest pages
> 
> Host machine is old Intel Core2 without EPT (TDP).
> 
> TIA, Z.

Hi, with v5.10-rc6:
get_mmio_spte: detect reserved bits on spte, addr 0xfe00d000, dump hierarchy:
------ spte 0x8000030e level 3.
------ spte 0xaf82027 level 2.
------ spte 0x2038001ffe00d407 level 1.
------------[ cut here ]------------
WARNING: CPU: 1 PID: 355 at kvm_mmu_page_fault.cold+0x42/0x4f [kvm]
...
CPU: 1 PID: 355 Comm: qemu-build Not tainted 5.10.0-rc6-amd64 #1
Hardware name:  /DG35EC, BIOS ECG3510M.86A.0118.2010.0113.1426 01/13/2010
RIP: 0010:kvm_mmu_page_fault.cold+0x42/0x4f [kvm]
Code: e2 ec 44 8b 04 24 8b 5c 24 0c 44 89 c5 89 da 83 eb 01 48 c7 c7 20 b2 65 c0 48 63 c3 48 8b 74 c4 30 e8 dd 74 e2 ec 39 dd 7e e3 <0f> 0b 41 b8 ea ff ff ff e9 27 99 ff ff 0f 0b 48 8b 54 24 10 48 83
RSP: 0018:ffffb67400653d30 EFLAGS: 00010202
RAX: 0000000000000027 RBX: 0000000000000000 RCX: ffffa271ff2976f8
RDX: 00000000ffffffd8 RSI: 0000000000000027 RDI: ffffa271ff2976f0
RBP: 0000000000000001 R08: ffffffffadd02ae8 R09: 0000000000000003
R10: 00000000ffffe000 R11: 3fffffffffffffff R12: 00000000fe00d000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
FS:  00007fc10ae3d640(0000) GS:ffffa271ff280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000002dc2000 CR4: 00000000000026e0
Call Trace:
 kvm_arch_vcpu_ioctl_run+0xbaf/0x18f0 [kvm]
 ? do_futex+0x7c4/0xb80
 kvm_vcpu_ioctl+0x203/0x520 [kvm]
 ? set_next_entity+0x5b/0x80
 ? __switch_to_asm+0x32/0x60
 ? finish_task_switch+0x70/0x260
 __x64_sys_ioctl+0x338/0x720
 ? __x64_sys_futex+0x120/0x190
 do_syscall_64+0x33/0x40
 entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x7fc10c389f6b
Code: 89 d8 49 8d 3c 1c 48 f7 d8 49 39 c4 72 b5 e8 1c ff ff ff 85 c0 78 ba 4c 89 e0 5b 5d 41 5c c3 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d d5 ae 0c 00 f7 d8 64 89 01 48
RSP: 002b:00007fc10ae3c628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007fc10c389f6b
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000012
RBP: 000055ad3767baf0 R08: 000055ad36be4850 R09: 00000000000000ff
R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
R13: 000055ad371d9800 R14: 0000000000000001 R15: 0000000000000002
---[ end trace c5f7ae690f5abcc4 ]---

without: kvm: x86/mmu: Fix get_mmio_spte() on CPUs supporting 5-level PT
I can run guest again, but with degraded performance as before.

Z.



[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux