I have a linux system running in kvm, with 6 qemu emulated NVMe drives, as expected all of them have the same PCIe Vendor ID and Device ID(VID: 0x1b36 DID: 0x0010). When I try to unbind them from the kernel NVMe driver and bind it to vfio-pci one by one, I am getting "write error: File exists" when I try to bind the 2nd(and other) drive to vfio-pci. Kernel version 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme0n1 -> ../devices/pci0000:00/0000:00:03.0/nvme/nvme0/nvme0n1 lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme1n1 -> ../devices/pci0000:00/0000:00:04.0/nvme/nvme1/nvme1n1 lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme2n1 -> ../devices/pci0000:00/0000:00:05.0/nvme/nvme2/nvme2n1 lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme3n1 -> ../devices/pci0000:00/0000:00:06.0/nvme/nvme3/nvme3n1 lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme4n1 -> ../devices/pci0000:00/0000:00:07.0/nvme/nvme4/nvme4n1 lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme5n1 -> ../devices/pci0000:00/0000:00:08.0/nvme/nvme5/nvme5n1 Steps for repro ubind nvme2 from kernel NVMe driver and bind it to vfio $ ls -l /sys/bus/pci/drivers/vfio-pci/ total 0 lrwxrwxrwx 1 root root 0 Dec 8 13:04 0000:00:05.0 -> ../../../../devices/pci0000:00/0000:00:05.0 --w------- 1 root root 4096 Dec 8 13:07 bind lrwxrwxrwx 1 root root 0 Dec 8 13:07 module -> ../../../../module/vfio_pci --w------- 1 root root 4096 Dec 8 13:04 new_id --w------- 1 root root 4096 Dec 8 13:07 remove_id --w------- 1 root root 4096 Dec 8 11:32 uevent --w------- 1 root root 4096 Dec 8 13:07 unbind Unbind nvme3 from kernel NVMe driver Try binding to vfio-pci # echo "0x1b36 0x0010" > /sys/bus/pci/drivers/vfio-pci/new_id -bash: echo: write error: File exists Not sure but this seems interesting https://github.com/torvalds/linux/commit/3853f9123c185eb4018f5ccd3cdda5968efb5e10#diff-625d2827bff96bb3a019fa705d99f0b89ec32f281c38a844457b3413d9172007 Can some help ?
