On Mon, May 25, 2020 at 08:32:44AM -0700, Bart Van Assche wrote: > On 2020-05-24 21:09, Ming Lei wrote: > > On Sat, May 23, 2020 at 08:19:58AM -0700, Bart Van Assche wrote: > >> On 2020-05-21 19:39, Ming Lei wrote: > >>> You may argue that two hw queue may share single managed interrupt, that > >>> is possible if driver plays the trick. But if driver plays the trick in > >>> this way, it is driver's responsibility to guarantee that the managed > >>> irq won't be shutdown if either of the two hctxs are active, such as, > >>> making sure that hctx->cpumask + hctx->cpumask <= this managed interrupt's affinity. > >>> It is definitely one strange enough case, and this patch doesn't > >>> suppose to cover this strange case. But, this patch won't break this > >>> case. Also just be curious, do you have such in-tree case? and are you > >>> sure the driver uses managed interrupt? > >> > >> I'm concerned about the block drivers that use RDMA (NVMeOF, SRP, iSER, > >> ...). The functions that accept an interrupt vector argument > >> (comp_vector), namely ib_alloc_cq() and ib_create_cq(), can be used in > > > > PCI_IRQ_AFFINITY isn't used in RDMA driver, so RDMA NIC uses non-managed > > irq. > > Something doesn't add up ... > > On a system with eight CPU cores and a ConnectX-3 RDMA adapter (mlx4 > driver; uses request_irq()) I ran the following test: > * Query the affinity of all mlx4 edge interrupts (mlx4-1@0000:01:00.0 .. > mlx4-16@0000:01:00.0). > * Offline CPUs 6 and 7. > * Query CPU affinity again. > > As one can see below the affinity of the mlx4 interrupts was modified. > Does this mean that the interrupt core manages more than only interrupts > registered with PCI_IRQ_AFFINITY? Interrupts registered with PCI_IRQ_AFFINITY are assigned their cpu affinity mask at creation time and it never changes for the lifetime of that interrupt. Interrupts not registerd with that option can have their affinity modified in various paths.
