On Fri, Dec 9, 2022 at 12:50 AM Bjorn Helgaas <helgaas@xxxxxxxxxx> wrote: > > [+cc VFIO folks and Zhenzhong (author of the commit you mention)] > > On Thu, Dec 08, 2022 at 09:24:31PM +0530, Major Saheb wrote: > > I have a linux system running in kvm, with 6 qemu emulated NVMe > > drives, as expected all of them have the same PCIe Vendor ID and > > Device ID(VID: 0x1b36 DID: 0x0010). > > > > When I try to unbind them from the kernel NVMe driver and bind it to > > vfio-pci one by one, I am getting "write error: File exists" when I > > try to bind the 2nd(and other) drive to vfio-pci. > > > > Kernel version > > > > 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 > > x86_64 x86_64 GNU/Linux > > > > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme0n1 -> ../devices/pci0000:00/0000:00:03.0/nvme/nvme0/nvme0n1 > > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme1n1 -> ../devices/pci0000:00/0000:00:04.0/nvme/nvme1/nvme1n1 > > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme2n1 -> ../devices/pci0000:00/0000:00:05.0/nvme/nvme2/nvme2n1 > > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme3n1 -> ../devices/pci0000:00/0000:00:06.0/nvme/nvme3/nvme3n1 > > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme4n1 -> ../devices/pci0000:00/0000:00:07.0/nvme/nvme4/nvme4n1 > > lrwxrwxrwx 1 root root 0 Dec 8 11:32 /sys/block/nvme5n1 -> ../devices/pci0000:00/0000:00:08.0/nvme/nvme5/nvme5n1 > > > > Steps for repro > > ubind nvme2 from kernel NVMe driver and bind it to vfio > > $ ls -l /sys/bus/pci/drivers/vfio-pci/ > > lrwxrwxrwx 1 root root 0 Dec 8 13:04 0000:00:05.0 -> ../../../../devices/pci0000:00/0000:00:05.0 > > --w------- 1 root root 4096 Dec 8 13:07 bind > > lrwxrwxrwx 1 root root 0 Dec 8 13:07 module -> ../../../../module/vfio_pci > > --w------- 1 root root 4096 Dec 8 13:04 new_id > > --w------- 1 root root 4096 Dec 8 13:07 remove_id > > --w------- 1 root root 4096 Dec 8 11:32 uevent > > --w------- 1 root root 4096 Dec 8 13:07 unbind > > > > Unbind nvme3 from kernel NVMe driver > > Try binding to vfio-pci > > # echo "0x1b36 0x0010" > /sys/bus/pci/drivers/vfio-pci/new_id > > -bash: echo: write error: File exists Above operation added new id to vfio-pci, it will trigger scanning of unbind devices once by default. To bind other devices of same id, besides what Alex suggested, I think you can also use /sys/bus/pci/drivers/vfio-pci/bind interface. Thanks Zhenzhong > > > > Not sure but this seems interesting > > https://github.com/torvalds/linux/commit/3853f9123c185eb4018f5ccd3cdda5968efb5e10#diff-625d2827bff96bb3a019fa705d99f0b89ec32f281c38a844457b3413d9172007 > > > > Can some help ?
