Hi
I noticed that I am unable to resume my vms from a suspend, and that live
migrate is also broken since I started running the 4.15 rcs. I use
libvirt's managed save, the resume fails with the following dump in dmesg:
[ 75.401885] *** Guest State ***
[ 75.401888] CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7
[ 75.401888] CR4: actual=0x0000000000002050, shadow=0x0000000000000000, gh_mask=ffffffffffffe871
[ 75.401889] CR3 = 0x00000000feffc000
[ 75.401890] RSP = 0xffffffff81e03ec0 RIP = 0xffffffff8104dbe2
[ 75.401890] RFLAGS=0x00000292 DR7 = 0x0000000000000400
[ 75.401891] Sysenter RSP=0000000000000000 CS:RIP=0000:0000000000000000
[ 75.401892] CS: sel=0xf000, attr=0x0009b, limit=0x0000ffff, base=0x00000000ffff0000
[ 75.401893] DS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000
[ 75.401894] SS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000
[ 75.401895] ES: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000
[ 75.401895] FS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000
[ 75.401896] GS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000
[ 75.401896] GDTR: limit=0x0000ffff, base=0x0000000000000000
[ 75.401897] LDTR: sel=0x0000, attr=0x00082, limit=0x0000ffff, base=0x0000000000000000
[ 75.401898] IDTR: limit=0x0000ffff, base=0x0000000000000000
[ 75.401898] TR: sel=0x0000, attr=0x0008b, limit=0x0000ffff, base=0x0000000000000000
[ 75.401899] EFER = 0x0000000000000000 PAT = 0x0007040600070406
[ 75.401900] DebugCtl = 0x0000000000000000 DebugExceptions = 0x0000000000000000
[ 75.401900] Interruptibility = 00000000 ActivityState = 00000000
[ 75.401901] *** Host State ***
[ 75.401901] RIP = 0xffffffff81063bbd RSP = 0xffffc90001effd08
[ 75.401902] CS=0010 SS=0018 DS=0000 ES=0000 FS=0000 GS=0000 TR=0040
[ 75.401903] FSBase=00007fc2ebcfc700 GSBase=ffff88033f280000 TRBase=ffff88033f297b80
[ 75.401903] GDTBase=ffffffffff576000 IDTBase=ffffffffff57b000
[ 75.401904] CR0=0000000080050033 CR3=00000003113bb005 CR4=00000000001626e0
[ 75.401905] Sysenter RSP=0000000000000000 CS:RIP=0010:ffffffff8196edf0
[ 75.401906] EFER = 0x0000000000000d01 PAT = 0x0407050600070106
[ 75.401906] *** Control State ***
[ 75.401907] PinBased=0000007f CPUBased=96a1e9fa SecondaryExec=00002cf2
[ 75.401907] EntryControls=0000d1ff ExitControls=002fefff
[ 75.401908] ExceptionBitmap=00060042 PFECmask=00000000 PFECmatch=00000000
[ 75.401909] VMEntry: intr_info=80000071 errcode=00000000 ilen=00000000
[ 75.401909] VMExit: intr_info=00000000 errcode=00000000 ilen=00000000
[ 75.401910] reason=80000021 qualification=0000000000000000
[ 75.401910] IDTVectoring: info=00000000 errcode=00000000
[ 75.401911] TSC Offset = 0xffffffd310d5a99a
[ 75.401911] TPR Threshold = 0x00
[ 75.401912] EPT pointer = 0x00000002e304d05e
[ 75.401912] PLE Gap=00000080 Window=00001000
[ 75.401913] Virtual processor ID = 0x0001
qemu's log says:
KVM: entry failed, hardware error 0x80000021
If you're running a guest on an Intel machine without unrestricted mode
support, the failure can be most likely due to the guest entering an invalid
state for Intel VT. For example, the guest maybe running in big real mode
which is not supported on less recent Intel processors.
EAX=8100d9e0 EBX=81f04540 ECX=00000000 EDX=00000000
ESI=00000000 EDI=00000000 EBP=00000000 ESP=81e03ec0
EIP=8104dbe2 EFL=00000292 [--S-A--] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 00000000 0000ffff
IDT= 00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
This happens reliably with both an opensuse vm (running kernel 4.1.12) and
a windows 10 vm.
I've bisected this down to commit: f29810335965ac1f7bcb501ee2af5f039f792416,
the full bisect log is:
git bisect start
# bad: [2c1cfa49901839136e578ca516a7e230182da024] Merge tag 'usb-4.15-rc8' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb
git bisect bad 2c1cfa49901839136e578ca516a7e230182da024
# good: [bebc6082da0a9f5d47a1ea2edc099bf671058bd4] Linux 4.14
git bisect good bebc6082da0a9f5d47a1ea2edc099bf671058bd4
# good: [5d352e69c60e54b5f04d6e337a1d2bf0dbf3d94a] Merge tag 'media/v4.15-1' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media
git bisect good 5d352e69c60e54b5f04d6e337a1d2bf0dbf3d94a
# good: [f6705bf959efac87bca76d40050d342f1d212587] Merge tag 'drm-for-v4.15-amd-dc' of git://people.freedesktop.org/~airlied/linux
git bisect good f6705bf959efac87bca76d40050d342f1d212587
# good: [6b0b3bda3d77c35fa9cf20067c0b62daff98c042] Merge tag 'usb-4.15-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb
git bisect good 6b0b3bda3d77c35fa9cf20067c0b62daff98c042
# good: [d1ce8ceb8ba87034bb3c3a9c9324e77546ba8608] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
git bisect good d1ce8ceb8ba87034bb3c3a9c9324e77546ba8608
# bad: [31336ed90cdc866ac25d36851e91df532837c763] Merge branch 'core-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad 31336ed90cdc866ac25d36851e91df532837c763
# bad: [caf9a82657b313106aae8f4a35936c116a152299] Merge branch 'x86-pti-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad caf9a82657b313106aae8f4a35936c116a152299
# bad: [ead68f216110170ec729e2c4dec0aad6d38259d7] Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net
git bisect bad ead68f216110170ec729e2c4dec0aad6d38259d7
# good: [932f8c77a9cca1be544e9f0bc7dcbe8e6b7cff98] Merge tag 'mlx5-fixes-2017-12-19' of git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux
git bisect good 932f8c77a9cca1be544e9f0bc7dcbe8e6b7cff98
# bad: [9035a8961b504d0997369509ab8c6b1f0a4ee33d] Merge branch 'for-linus' of git://git.kernel.dk/linux-block
git bisect bad 9035a8961b504d0997369509ab8c6b1f0a4ee33d
# bad: [43aabca38aa9668eee3c3c1206207034614c0901] Merge tag 'kvm-arm-fixes-for-v4.15-2' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
git bisect bad 43aabca38aa9668eee3c3c1206207034614c0901
# good: [08e20a6300e106d5feb89c9e47ea479533fec46f] tools/kvm_stat: handle invalid regular expressions
git bisect good 08e20a6300e106d5feb89c9e47ea479533fec46f
# good: [0eb7c33cadf6b2f1a94e58ded8b0eb89b4eba382] KVM: arm/arm64: Fix timer enable flow
git bisect good 0eb7c33cadf6b2f1a94e58ded8b0eb89b4eba382
# good: [cf656c76614c6ec5b016233cac29738881c83c08] tools/kvm_stat: add line for totals
git bisect good cf656c76614c6ec5b016233cac29738881c83c08
# bad: [e39d200fa5bf5b94a0948db0dae44c1b73b84a56] KVM: Fix stack-out-of-bounds read in write_mmio
git bisect bad e39d200fa5bf5b94a0948db0dae44c1b73b84a56
# bad: [f29810335965ac1f7bcb501ee2af5f039f792416] KVM/x86: Check input paging mode when cs.l is set
git bisect bad f29810335965ac1f7bcb501ee2af5f039f792416
# first bad commit: [f29810335965ac1f7bcb501ee2af5f039f792416] KVM/x86: Check input paging mode when cs.l is set
Some more details: this is a thinkpad T440s, i5-4210U.
I'm available to test any fixes.
Jeremi
