On Tue, May 16, 2017 at 02:39:18PM +0200, Auger Eric wrote: > Hi Jean, Christoffer, > > On 16/05/2017 13:23, Jean-Philippe Brucker wrote: > > Hi, > > > > On 09/05/17 09:56, Christoffer Dall wrote: > >> Instead of waiting with registering KVM iodevs until the first VCPU is > >> run, we can actually create the iodevs when the redist base address is > >> set. The only downside is that we must now also check if we need to do > >> this for VCPUs which are created after creating the VGIC, because there > >> is no enforced ordering between creating the VGIC (and setting its base > >> addresses) and creating the VCPUs. > > > > This triggers a BUG(), when the order is VGIC init, then VCPU init (which > > is what kvmtool does). > > > > Issuing KVM_CREATE_VCPU after VGIC intialization produces the following calls: > > > > kvm_vm_ioctl_create_vcpu > > kvm_arch_vcpu_create > > kvm_vcpu_init > > kvm_arch_vcpu_init > > kvm_vgic_vcpu_init > > vgic_register_redist_iodev > > kvm_vcpu_get_idx > > ... no VCPU registered yet in kvm->vcpus :( > > BUG(); > > in QEMU use case, kvm_vgic_vcpu_init/vgic_register_redist_iodev does > nothing since KVM_VGIC_V3_ADDR_TYPE_REDIST was not called yet and > vgic->vgic_redist_base is undefined. Thus we postpone the redist_iodev > registration until the redist base address is set. > > In QEMU case KVM_VGIC_V3_ADDR_TYPE_REDIST is called when all the CPUs > are initialized and we were lucky. > > > > ... would later register vcpu: > > kvm->vcpus[atomic_read(&kvm->online_vcpus)] = vcpu > > > > My quick fix is to move kvm_vgic_vcpu_init into kvm_arch_vcpu_postcreate, > > but it discards the return value of kvm_vgic_vcpu_init, so I don't know > > how to do it properly. > > changing the proto of kvm_arch_vcpu_postcreate and moving the > kvm_vgic_vcpu_init there could be an alternative. I think the whole point of postcreate is a hook that can be called where it doesn't produce an error (rolling back the create at that point is pretty horrid). I'll have a closer look in the morning at what we can do - perhaps the idx thing is just a ridiculous requirement and we can do something more clever. Thanks for the heads up, and sorry about breaking stuff. -Christoffer
