https://bugzilla.kernel.org/show_bug.cgi?id=202055 --- Comment #5 from Alex Williamson (alex.williamson@xxxxxxxxxx) --- (In reply to Alex from comment #4) > I rebound device and there is a lspci output. It reports Count=16. > > ``` > root@localhost /home/alex # echo 0000:06:00.0 | tee > /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind > root@localhost /home/alex # echo 8086 f1a6 | tee > /sys/bus/pci/drivers/vfio-pci/new_id > 06:00.0 Non-Volatile memory controller [0108]: Intel Corporation SSD Pro > 7600p/760p/E 6100p Series [8086:f1a6] (rev 03) (prog-if 02 [NVM Express]) > Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- >Stepping- SERR+ FastB2B- DisINTx- > Capabilities: [b0] MSI-X: Enable- Count=16 Masked- > Vector table: BAR=0 offset=00002000 > PBA: BAR=0 offset=00002100 > ``` > > Resetted with vfio-pci in use. This time it reports Count=22. > ``` > root@localhost /home/alex # echo 1 | tee > /sys/bus/pci/devices/0000:06:00.0/reset > 06:00.0 Non-Volatile memory controller [0108]: Intel Corporation SSD Pro > 7600p/760p/E 6100p Series [8086:f1a6] (rev 03) (prog-if 02 [NVM Express]) > Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- >Stepping- SERR+ FastB2B- DisINTx- > Capabilities: [b0] MSI-X: Enable- Count=22 Masked- > Vector table: BAR=0 offset=00002000 > PBA: BAR=0 offset=00002100 Ok, interesting. So likely when QEMU is analyzing the device it's seeing this 22 value which is why it throws an error at the sanity test. With the nvme driver bound, we seem to get a sane number of MSI-X entries, though it still confuses me how the reporter in [1] claimed their system was making use of 17 vectors, which would mean that Count=16 is still bogus. In any case, let's see if we can figure out what we can poke on the device to make these fields within the register change. Start with the device in the state you have it above where it reports Count=22. First let's test if the vector table size is really read-only: # setpci -s 06:00.0 CAP_MSIX+2.w This should return 0016 as 0x16 is 22. Try to write it: # setpci -s 06:00.0 CAP_MSIX+2.w=10:7ff And read it back again: # setpci -s 06:00.0 CAP_MSIX+2.w Did the value change? Next, we already have memory enabled on the device, but the nvme driver also enables bus master before enabling interrupts, so let's check if setting bus master triggers a change in the MSI-X capability: # setpci -s 06:00.0 COMMAND This should report 0102 based on the lspci output, to enable bus master: # setpci -s 06:00.0 COMMAND=4:4 Does the Count value in the MSI-X capability change? (To return it back to the previous state: setpci -s 06:00.0 COMMAND=0:4) I'd really hope that one of the above helps to indicate a next step, but we could also try enabling MSI-X (with it masked), so we could try: # setpci -s 06:00.0 CAP_MSIX+2.w=c000:c000 This should cause lspci to report Enable+ and Masked+, but does the Count value change? (To return it back to the previous state: setpci -s 06:00.0 CAP_MSIX+2.w=0:c000) -- You are receiving this mail because: You are watching the assignee of the bug.
