On Tue, 11 May 2021 11:44:49 +0800 Yicong Yang <yangyicong@xxxxxxxxxxxxx> wrote: > [ +qemu-devel ] > > On 2021/4/30 22:29, Alex Williamson wrote: > > On Fri, 30 Apr 2021 15:57:47 +0800 > > Yicong Yang <yangyicong@xxxxxxxxxxxxx> wrote: > > > >> When I try to remove the PF driver in the host, the process will be blocked > >> if the related VF of the device is added in the Qemu as an iEP. > >> > >> here's what I got in the host: > >> > >> [root@localhost 0000:75:00.0]# rmmod hisi_zip > >> [99760.571352] vfio-pci 0000:75:00.1: Relaying device request to user (#0) > >> [99862.992099] vfio-pci 0000:75:00.1: Relaying device request to user (#10) > >> [...] > >> > >> and in the Qemu: > >> > >> estuary:/$ lspci -tv > >> -[0000:00]-+-00.0 Device 1b36:0008 > >> +-01.0 Device 1af4:1000 > >> +-02.0 Device 1af4:1009 > >> \-03.0 Device 19e5:a251 <----- the related VF device > >> estuary:/$ qemu-system-aarch64: warning: vfio 0000:75:00.1: Bus 'pcie.0' does not support hotplugging > >> qemu-system-aarch64: warning: vfio 0000:75:00.1: Bus 'pcie.0' does not support hotplugging > >> qemu-system-aarch64: warning: vfio 0000:75:00.1: Bus 'pcie.0' does not support hotplugging > >> qemu-system-aarch64: warning: vfio 0000:75:00.1: Bus 'pcie.0' does not support hotplugging > >> [...] > >> > >> The rmmod process will be blocked until I kill the Qemu process. That's the only way if I > >> want to end the rmmod. > >> > >> So my question is: is such block reasonable? If the VF devcie is occupied or doesn't > >> support hotplug in the Qemu, shouldn't we fail the rmmod and return something like -EBUSY > >> rather than make the host blocked indefinitely? > > > > Where would we return -EBUSY? pci_driver.remove() returns void. > > Without blocking, I think our only option would be to kill the user > > process. > > > > yes. the remove() callback of pci_driver doesn't provide a way to abort the process. > > >> Add the VF under a pcie root port will avoid this. Is it encouraged to always > >> add the VF under a pcie root port rather than directly add it as an iEP? > > > > Releasing a device via the vfio request interrupt is always a > > cooperative process currently, the VM needs to be configured such that > > the device is capable of being unplugged and the guest needs to respond > > to the ejection request. Thanks, > > > > Does it make sense to abort the VM creation and give some warnings if user try to > pass a vfio pci device to the Qemu and doesn't attach it to a hotpluggable > bridge? Currently I think there isn't such a mechanism in Qemu. You're essentially trying to define a usage policy and pick somewhere to impose it. I think QEMU is not the right place. There are plenty of valid assigned device configurations where the device is not hotpluggable. You therefore either need to look up in the stack if your environment demands that VM configurations should always be able to release devices at the request of the kernel, or down in the stack if you believe the kernel has an obligation to take that device if the user fails to respond to a device request. We've shied away from the latter because it generally involves killing the holding process, either directly or by closing off access to the device, where in the case of mmaps to the device, ongoing access would result in a SIGBUS to the process anyway. I wouldn't object to the kernel having a right to do this, but it's not something that has reached a high priority. Thanks, Alex
