On 21/03/17 10:27, Joao Pinto wrote: > > Hi Mark, > > Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu: >> Hi Joao, >> >> On 20/03/17 17:30, Joao Pinto wrote: >>> Hi Mark, >>> >>> I am facing a challenge in a setup I must bring up that is composed by a >>> Synopsys Root Complex connected to a MSIx-only Endpoint from Mellanox. >>> The endpoint probe fails because it is not capable of enabling msix. >>> >>> In order to get more info about it, I added some debug messages to the msix >>> enable process and got the following: >>> >>> MSIX DBG[mlx5_enable_msix]: Num. EQs: 256 >>> MSIX DBG[mlx5_enable_msix]: Num. Vectors: 4 >>> MSIX DBG[mlx5_enable_msix]: Min. Num. Vectors: 4 >>> MSIX DBG[mlx5_enable_msix]: Num. Online CPUs: 1 >>> MSIX DBG[mlx5_enable_msix]: MLX5_EQ_VEC_COMP_BASE: 3 >>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 0 >>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 1 >>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 2 >>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 3 >>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range >>> MSIX DBG[__pci_enable_msix_range]: Enabling MSIX [Nvec: 4, Affinity: 0] >>> MSIX DBG[__pci_enable_msix]: NVec: 4 >>> MSIX DBG[__pci_enable_msix]: Num. Entries: 64 >>> MSIX DBG[__pci_enable_msix]: Checking invalid entries >>> MSIX DBG[msix_capability_init]: Ensure MSI is disable >>> MSIX DBG[msix_capability_init]: Getting MSIX Capability >>> MSIX DBG[msix_capability_init]: Mapping MSIX Capability Region >>> MSIX DBG[msix_capability_init]: Setup Entries >>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 0 >>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 1 >>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 2 >>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 3 >>> MSIX DBG[msix_capability_init]: Setup IRQS >>> MSIX DBG[pci_msi_setup_msi_irqs]: Inside! >>> MSIX DBG[arch_setup_msi_irqs]: Inside! >>> MSIX DBG[msix_capability_init]: Failed pci_msi_setup_msi_irqs >>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range, result: -22 >>> mlx5_core 0000:01:00.0: enable msix failed >>> >>> The conclusion I came up was that the msix setup in the endpoint fails because >>> in the pcie-designware-host.c we are rejecting msix as you can see here: >>> >>> /* MSI-X interrupts are not supported */ >>> if (type == PCI_CAP_ID_MSIX) >>> return -EINVAL; >>> >>> Did you ever came across this problem? >> >> I don't have access to any system using this IP, unfortunately. This >> looks like an arbitrary limitation of the host controller driver, as >> there isn't much difference between MSI and MSI-X at the MSI controller >> level, other than the reduced number of interrupts. >> >> Unfortunately, this driver seems to rely on rather old APIs, and could >> use some care and attention. I'm pretty sure that converting this driver >> to the generic MSI API would solve this problem entirely, but I'm afraid >> I don't have the bandwidth to do so. > > I volunteer to do it, no problem, but I will need some guidance and examples. > Could you give me some hints to get started? A very simple example using the current API is pcie-altera-msi.c (some others are much more intricate), and should give you a good picture of how PCI/MSI, generic MSI and MSI controllers interact. Start with understanding how the various layers interact via their own IRQ domains, and implement their own IRQ chip. Happy to answer questions (I'm sure you'll have some). Thanks, M. -- Jazz is not dead. It just smells funny...
