[+cc VFIO folks and Zhenzhong (author of the commit you mention)] On Thu, Dec 08, 2022 at 09:24:31PM +0530, Major Saheb wrote: > I have a linux system running in kvm, with 6 qemu emulated NVMe > drives, as expected all of them have the same PCIe Vendor ID and > Device ID(VID: 0x1b36 DID: 0x0010). > > When I try to unbind them from the kernel NVMe driver and bind it to > vfio-pci one by one, I am getting "write error: File exists" when I > try to bind the 2nd(and other) drive to vfio-pci. > > Kernel version > > 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 > x86_64 x86_64 GNU/Linux > > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme0n1 -> ../devices/pci0000:00/0000:00:03.0/nvme/nvme0/nvme0n1 > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme1n1 -> ../devices/pci0000:00/0000:00:04.0/nvme/nvme1/nvme1n1 > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme2n1 -> ../devices/pci0000:00/0000:00:05.0/nvme/nvme2/nvme2n1 > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme3n1 -> ../devices/pci0000:00/0000:00:06.0/nvme/nvme3/nvme3n1 > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme4n1 -> ../devices/pci0000:00/0000:00:07.0/nvme/nvme4/nvme4n1 > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme5n1 -> ../devices/pci0000:00/0000:00:08.0/nvme/nvme5/nvme5n1 > > Steps for repro > ubind nvme2 from kernel NVMe driver and bind it to vfio > $ ls -l /sys/bus/pci/drivers/vfio-pci/ > lrwxrwxrwx 1 root root 0 Dec 8 13:04 0000:00:05.0 -> ../../../../devices/pci0000:00/0000:00:05.0 > --w------- 1 root root 4096 Dec 8 13:07 bind > lrwxrwxrwx 1 root root 0 Dec 8 13:07 module -> ../../../../module/vfio_pci > --w------- 1 root root 4096 Dec 8 13:04 new_id > --w------- 1 root root 4096 Dec 8 13:07 remove_id > --w------- 1 root root 4096 Dec 8 11:32 uevent > --w------- 1 root root 4096 Dec 8 13:07 unbind > > Unbind nvme3 from kernel NVMe driver > Try binding to vfio-pci > # echo "0x1b36 0x0010" > /sys/bus/pci/drivers/vfio-pci/new_id > -bash: echo: write error: File exists > > Not sure but this seems interesting > https://github.com/torvalds/linux/commit/3853f9123c185eb4018f5ccd3cdda5968efb5e10#diff-625d2827bff96bb3a019fa705d99f0b89ec32f281c38a844457b3413d9172007 > > Can some help ?
