Re: [PATCH v2 00/36] Add support for versioned CPU models

[Han Han <hhan@xxxxxxxxxx>]

Hi Jiri, I meet a segment fault for sev-snp testing on this libvirt branch. Here are the details:

Version:

qemu-kvm-9.1.0-4.el10.x86_64

kernel-6.12.0-30.el10.x86_64

host CPU: AMD EPYC 7313 16-Core Processor

Step:

0. Enable AMD SEV-SNP settings in BIOS and kernel. Make sure `sevctl ok` pass.

1. Start a domain with sev-snp, EPYC-Milan-v2 CPU model

domain XML:

<domain type='kvm'>

...

  <currentMemory unit='KiB'>5120000</currentMemory>
  <memtune>
    <hard_limit unit='KiB'>5120000</hard_limit>
  </memtune>
  <vcpu placement='static'>4</vcpu>

  <os>
    <type arch='x86_64' machine='pc-q35-rhel10.0.0'>hvm</type>
    <loader type='rom' stateless='yes'>/usr/share/edk2/ovmf/OVMF.amdsev.fd</loader>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <smm state='on'/>
  </features>
  <cpu mode='host-model' check='partial'>
    <model fallback='forbid'>EPYC-Milan-v2</model>
    <vendor>AMD</vendor>
    <maxphysaddr mode='passthrough' limit='43'/>
    <feature policy='require' name='x2apic'/>
    <feature policy='require' name='tsc-deadline'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='tsc_adjust'/>
    <feature policy='require' name='vaes'/>
    <feature policy='require' name='vpclmulqdq'/>
    <feature policy='require' name='spec-ctrl'/>
    <feature policy='require' name='stibp'/>
    <feature policy='require' name='arch-capabilities'/>
    <feature policy='require' name='ssbd'/>
    <feature policy='require' name='cmp_legacy'/>
    <feature policy='require' name='overflow-recov'/>
    <feature policy='require' name='succor'/>
    <feature policy='require' name='invtsc'/>
    <feature policy='require' name='stibp-always-on'/>
    <feature policy='require' name='virt-ssbd'/>
    <feature policy='require' name='amd-psfd'/>
    <feature policy='require' name='lbrv'/>
    <feature policy='require' name='tsc-scale'/>
    <feature policy='require' name='vmcb-clean'/>
    <feature policy='require' name='flushbyasid'/>
    <feature policy='require' name='pause-filter'/>
    <feature policy='require' name='pfthreshold'/>
    <feature policy='require' name='v-vmsave-vmload'/>
    <feature policy='require' name='vgif'/>
    <feature policy='require' name='no-nested-data-bp'/>
    <feature policy='require' name='lfence-always-serializing'/>
    <feature policy='require' name='null-sel-clr-base'/>
    <feature policy='require' name='rdctl-no'/>
    <feature policy='require' name='skip-l1dfl-vmentry'/>
    <feature policy='require' name='mds-no'/>
    <feature policy='require' name='pschange-mc-no'/>
    <feature policy='require' name='gds-no'/>
    <feature policy='require' name='rfds-no'/>
  </cpu>

...

  <launchSecurity type='sev-snp' kernelHashes='no'>
    <cbitpos>51</cbitpos>
    <reducedPhysBits>1</reducedPhysBits>
    <policy>0x00030000</policy>
  </launchSecurity>
</domain>

# virsh start sev-snp
error: Disconnected from qemu:///system due to end of file
error: Failed to start domain 'sev-snp'
error: End of file while reading data: Input/output error

# coredumpctl -1
TIME                         PID UID GID SIG     COREFILE EXE                 SIZE
Tue 2024-11-26 21:54:19 EST 5027   0   0 SIGSEGV present  /usr/sbin/virtqemud   1M

Backtrace:

#0  0x00007f78a4824dbf in virCPUCompareUnusable (arch=arch@entry=VIR_ARCH_X86_64, host=host@entry=0x7f78840269e0, cpu=cpu@entry=0x7f788402ba80, blockers=0x0, failIncompatible=failIncompatible@entry=true) at ../src/cpu/cpu.c:204
#1  0x00007f78904c10a6 in qemuDomainCheckCPU (arch=VIR_ARCH_X86_64, virtType=VIR_DOMAIN_VIRT_KVM, qemuCaps=0x7f7884039020 [virQEMUCaps], cpu=0x7f788402ba80, compatCPU=VIR_QEMU_CAPS_HOST_CPU_FULL, failIncompatible=<optimized out>) at ../src/qemu/qemu_domain.c:11433
#2  0x00007f7890550200 in qemuProcessUpdateGuestCPU (def=0x7f78581a0880, qemuCaps=0x7f7884039020 [virQEMUCaps], hostarch=VIR_ARCH_X86_64, flags=17) at ../src/qemu/qemu_process.c:6368
#3  qemuProcessPrepareDomain (driver=driver@entry=0x7f78340195e0, vm=vm@entry=0x7f7834352900 [virDomainObj], flags=flags@entry=17) at ../src/qemu/qemu_process.c:6800
#4  0x00007f7890556bb0 in qemuProcessStart (conn=0x7f788c428350, driver=0x7f78340195e0, vm=0x7f7834352900 [virDomainObj], updatedCPU=0x0, asyncJob=VIR_ASYNC_JOB_START, migrateFrom=0x0, migrateFd=-1, migratePath=0x0, snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_CREATE, flags=17) at ../src/qemu/qemu_process.c:8304
#5  0x00007f789057a3d8 in qemuDomainObjStart.constprop.0 (conn=0x7f788c428350, driver=0x7f78340195e0, vm=0x7f7834352900 [virDomainObj], flags=0, asyncJob=<optimized out>) at ../src/qemu/qemu_driver.c:6336
#6  0x00007f78904d0d91 in qemuDomainCreateWithFlags (dom=0x7f7884053e30, flags=0) at ../src/qemu/qemu_driver.c:6385
#7  0x00007f78a49132b9 in virDomainCreate (domain=domain@entry=0x7f7884053e30) at ../src/libvirt-domain.c:7079
#8  0x0000556939de9df1 in remoteDispatchDomainCreate (server=0x55694c7a1360 [virNetServer], msg=0x55694c7a6090, client=<optimized out>, rerr=0x7f78947fd9f0, args=0x7f788404fe70) at src/remote/remote_daemon_dispatch_stubs.h:5050
#9  remoteDispatchDomainCreateHelper (server=0x55694c7a1360 [virNetServer], client=<optimized out>, msg=0x55694c7a6090, rerr=0x7f78947fd9f0, args=0x7f788404fe70, ret=0x0) at src/remote/remote_daemon_dispatch_stubs.h:5029
#10 0x00007f78a4804f19 in virNetServerProgramDispatchCall (prog=0x55694c79a000 [virNetServerProgram], server=0x55694c7a1360 [virNetServer], client=0x55694c7aaac0 [virNetServerClient], msg=0x55694c7a6090) at ../src/rpc/virnetserverprogram.c:423
#11 virNetServerProgramDispatch (prog=0x55694c79a000 [virNetServerProgram], server=server@entry=0x55694c7a1360 [virNetServer], client=client@entry=0x55694c7aaac0 [virNetServerClient], msg=msg@entry=0x55694c7a6090) at ../src/rpc/virnetserverprogram.c:299
#12 0x00007f78a4805512 in virNetServerProcessMsg (srv=srv@entry=0x55694c7a1360 [virNetServer], client=0x55694c7aaac0 [virNetServerClient], prog=<optimized out>, msg=0x55694c7a6090) at ../src/rpc/virnetserver.c:135
#13 0x00007f78a4805621 in virNetServerHandleJob (jobOpaque=0x55694c798710, opaque=0x55694c7a1360) at ../src/rpc/virnetserver.c:155
#14 0x00007f78a473d225 in virThreadPoolWorker (opaque=<optimized out>) at ../src/util/virthreadpool.c:164
#15 0x00007f78a473c718 in virThreadHelper (data="" out>) at ../src/util/virthread.c:256
#16 0x00007f78a40dfa29 in start_thread (arg=<optimized out>) at pthread_create.c:447
#17 0x00007f78a414f4fc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Attachment: the domain XML, the qemu log, full backtrace

On Tue, Nov 26, 2024 at 2:48 PM Jiri Denemark <jdenemar@xxxxxxxxxx> wrote:

On Tue, Nov 26, 2024 at 14:36:55 +0800, Han Han wrote:
> Hi Jiri,
> I patched these patches to v10.9.0-159-ge9b8be23f7 then built and installed
> the rpms. After that, I hit an error when start a domain with host-model
> cpu:
>
> Version:
> libvirt v10.9.0-159-ge9b8be23f7 with this patch series
> qemu-kvm-9.1.0-5.el9.x86_64
> host CPU: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
>
> Here is how I build it:
> # curl
> https://patchew.org/Libvirt/cover.1732311125.git.jdenemar@xxxxxxxxxx/mbox
> -o /tmp/mbox && git am /tmp/mbox
> # meson build
> # meson dist -C build --no-tests
> # cp build/libvirt.spec ~/rpmbuild/SPECS
> # cp build/meson-dist/libvirt-10.10.0.tar.xz ~/rpmbuild/SOURCES
> # rpmbuild -ba ~/rpmbuild/SPECS/libvirt.spec --nocheck
> # dnf update ~/rpmbuild/RPMS/x86_64/libvirt*10.10.0-1* -y
>
> Then start a domain with host-model cpu:
> ➜  ~ virsh start rhel10-sebios
>
>
> error: Failed to start domain 'rhel10-sebios'
> error: XML error: failed to parse xml document
> '/usr/share/libvirt/cpu_map/x86_Denverton.xml
>
> For the virtqemud.log and the guest XML. Please check it in the attachment.

Oh yeah, I forgot to update src/cpu_map/meson.build in the last two
patches (Denverton and KnightsMill). I fixed it in my gitlab repo:

git fetch https://gitlab.com/jirkade/libvirt.git cpu-versions

Jirka

Attachment: bug.tar.gz
Description: application/gzip