On Thu, Feb 26, 2009 at 11:15:25PM -0700, Grant Grundler wrote: > ... > > +3. Why use MSIs? > > + > > +There are three reasons why using MSIs can give an advantage over > > +traditional pin-based interrupts. > ... > > +PCI devices can only support a single pin-based interrupt per function. > > Related to this is a 4th reason: distribute workload across CPUs > and enables construction of efficient, multi-queue devices. > Care to mention that? That's true for MSI-X, but not for MSIs in general. Workload is already distributed across CPUs with round-robin interrupts. I'm inclined to leave out this level of detail. > > +The MSI-X capability is much more flexible than the MSI capability. > > +It supports up to 2048 interrupts, each of which can be separately > > +assigned. > > Nothing describes "assignment" below or what is meant by "assigned". > My guess is you wanted to differentiate MSIX from MSI with: > ... and each MSIX can be directed at a different CPU. I think 'each of which can be controlled separately' might work better. For example, they're individually maskable which isn't (necessarily) true of plain MSI. > > +4.5 Considerations when using MSIs > > + > > +4.5.1 Choosing between MSI-X and MSI > > + > > +If your device supports both MSI-X and MSI capabilities, you should use > > +the MSI-X facilities in preference to the MSI facilities. As mentioned > > +above, MSI-X supports any number of interrupts between 1 and 2048. > > +In constrast, MSI is restricted to a maximum of 32 interrupts (and > > +must be a power of two). In addition, the MSI interrupt vectors must > > +be allocated consecutively, so the system may not be able to allocate > > +as many vectors for MSI as it could for MSI-X. On some platforms, MSI > > +interrupts must all be targetted at the same set of CPUs whereas MSI-X > > +interrupts can all be targetted at different CPUs. > > The description for MSI is correct. But Linux will only allocate one > MSI as noted in an earlier section. This section implies more could > be allocated when using MSI and that won't happen. > > IIRC, for AHCI perf you were working on a patch to change that and > it should probably update this text at the same time when the > behavior changes. Did you see this is patch 1/6? ;-) I removed the description of pci_enable_msi_block() from this patch, but missed updating this paragraph. By patch 6/6, this paragraph is true. > > +5. MSI quirks > > + > > +Several PCI chipsets or devices are known not to support MSIs. > > +The PCI stack provides three ways to disable MSIs: > > + > > +1. globally > > +2. on all devices behind a specific bridge > > +3. on a single device > ... > > +5.3. Disabling MSIs on a single device > > + > > +Some devices are known to have faulty MSI implementations. Usually this > > +is handled in the individual device driver but occasionally it's necessary > > +to handle this with a quirk. Some drivers have an option to disable MSIs; > > +this is deprecated. > > "this" is ambiguous. My guess is "quirks to disable MSI for a device is > deprecated" since recently some drivers have added module parameters to > disable MSI. Having an option to disable MSI is deprecated. That doesn't mean that individual driver authors aren't selfish and short-sighted. > > +5.4. Finding why MSIs are disabled on a device > > + > > +From the above three sections, you can see that there are many reasons > > +why MSIs may not be enabled for a given device. Your first step should > > +be to examine your dmesg carefully to determine whether MSIs are enabled > > +for your machine. You should also check your .config to be sure you > > +have enabled CONFIG_PCI_MSI. > > Should mention "fgrep MSI /proc/interrupts" to see if any devices have > MSI in use? Yes, you're right. > > +Then, lspci -t gives the list of bridges above a device. Reading > > +/sys/bus/pci/devices/*/msi_bus will tell you whether MSI are enabled (1) > > +or disabled (0). If 0 is found in any of the msi_bus files belonging > > +to bridges between the PCI root and the device, MSIs are disabled. > > + > > +It is also worth checking whether the device driver supports MSIs. > > Suggestions on how to check? 'eg has calls to pci_enable_msi(), pci_enable_msix() or pci_enable_msi_block()'? > Conversely, one can easily check if the driver has MSI disabled by default > and MSI can be enabled. e.g. use "modinfo mvsas" to check driver parameters. I'm not going to give examples of bad practise. > Reviewed-by: Grant Grundler <grundler@xxxxxxxxxxxxxxxxx> Thanks. -- Matthew Wilcox Intel Open Source Technology Centre "Bill, look, we understand that you're interested in selling us this operating system, but compare it to ours. We can't possibly take such a retrograde step." -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
