Às 10:41 AM de 3/21/2017, Marc Zyngier escreveu: > On 21/03/17 10:27, Joao Pinto wrote: >> >> Hi Mark, >> >> Às 6:16 PM de 3/20/2017, Marc Zyngier escreveu: >>> Hi Joao, >>> >>> On 20/03/17 17:30, Joao Pinto wrote: >>>> Hi Mark, >>>> >>>> I am facing a challenge in a setup I must bring up that is composed by a >>>> Synopsys Root Complex connected to a MSIx-only Endpoint from Mellanox. >>>> The endpoint probe fails because it is not capable of enabling msix. >>>> >>>> In order to get more info about it, I added some debug messages to the msix >>>> enable process and got the following: >>>> >>>> MSIX DBG[mlx5_enable_msix]: Num. EQs: 256 >>>> MSIX DBG[mlx5_enable_msix]: Num. Vectors: 4 >>>> MSIX DBG[mlx5_enable_msix]: Min. Num. Vectors: 4 >>>> MSIX DBG[mlx5_enable_msix]: Num. Online CPUs: 1 >>>> MSIX DBG[mlx5_enable_msix]: MLX5_EQ_VEC_COMP_BASE: 3 >>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 0 >>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 1 >>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 2 >>>> MSIX DBG[mlx5_enable_msix]: Filling msix array... entry = 3 >>>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range >>>> MSIX DBG[__pci_enable_msix_range]: Enabling MSIX [Nvec: 4, Affinity: 0] >>>> MSIX DBG[__pci_enable_msix]: NVec: 4 >>>> MSIX DBG[__pci_enable_msix]: Num. Entries: 64 >>>> MSIX DBG[__pci_enable_msix]: Checking invalid entries >>>> MSIX DBG[msix_capability_init]: Ensure MSI is disable >>>> MSIX DBG[msix_capability_init]: Getting MSIX Capability >>>> MSIX DBG[msix_capability_init]: Mapping MSIX Capability Region >>>> MSIX DBG[msix_capability_init]: Setup Entries >>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 0 >>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 1 >>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 2 >>>> MSIX DBG[msix_setup_entries]: Configuring MSIX entry 3 >>>> MSIX DBG[msix_capability_init]: Setup IRQS >>>> MSIX DBG[pci_msi_setup_msi_irqs]: Inside! >>>> MSIX DBG[arch_setup_msi_irqs]: Inside! >>>> MSIX DBG[msix_capability_init]: Failed pci_msi_setup_msi_irqs >>>> MSIX DBG[mlx5_enable_msix]: Enabling PCI MSIX Range, result: -22 >>>> mlx5_core 0000:01:00.0: enable msix failed >>>> >>>> The conclusion I came up was that the msix setup in the endpoint fails because >>>> in the pcie-designware-host.c we are rejecting msix as you can see here: >>>> >>>> /* MSI-X interrupts are not supported */ >>>> if (type == PCI_CAP_ID_MSIX) >>>> return -EINVAL; >>>> >>>> Did you ever came across this problem? >>> >>> I don't have access to any system using this IP, unfortunately. This >>> looks like an arbitrary limitation of the host controller driver, as >>> there isn't much difference between MSI and MSI-X at the MSI controller >>> level, other than the reduced number of interrupts. >>> >>> Unfortunately, this driver seems to rely on rather old APIs, and could >>> use some care and attention. I'm pretty sure that converting this driver >>> to the generic MSI API would solve this problem entirely, but I'm afraid >>> I don't have the bandwidth to do so. >> >> I volunteer to do it, no problem, but I will need some guidance and examples. >> Could you give me some hints to get started? > > A very simple example using the current API is pcie-altera-msi.c (some > others are much more intricate), and should give you a good picture of > how PCI/MSI, generic MSI and MSI controllers interact. > > Start with understanding how the various layers interact via their own > IRQ domains, and implement their own IRQ chip. Happy to answer questions > (I'm sure you'll have some). > > Thanks, > > M. > Thanks for the guidance Marc! Joao
