On Tuesday 20 January 2009, Rafael J. Wysocki wrote: > On Tuesday 20 January 2009, Matthew Wilcox wrote: > > On Mon, Jan 19, 2009 at 10:45:18AM +0100, Rafael J. Wysocki wrote: > > > On Monday 19 January 2009, Matthew Wilcox wrote: > > > > > > > > It seems that the PCI code goes out of its way to avoid doing any memory > > > > allocation during the suspend path. For example, there's > > > > pci_allocate_cap_save_buffers() which is called (indirectly) from > > > > pci_device_add(). > > > > > > > > This is a bit of a problem for MSI. I can get the per-interrupt > > > > overhead down to 4 bytes (ie the unsigned int that is the Linux > > > > interrupt number), but if I can't allocate memory during suspend, that > > > > bloats up to 16 bytes per interrupt. The maximum number of interrupts > > > > allocated per devfn is 2048, so that's an extra 24k of ram that either > > > > needs to be allocated all the time, or at suspend time (and then > > > > released at resume time). > > > > > > > > Matthew Garrett tells me there's no reason to avoid memory allocation at > > > > suspend time, indeed it's run twice, once with interrupts on and once > > > > with them off, precisely to permit memory allocations. So why does PCI > > > > try so hard? Is it just an accident of history? > > > > > > AFAICS, it is done so we don't have to allocate memory with interrupts > > > disabled, because that would have to be done with GFP_ATOMIC. > > > pci_allocate_cap_save_buffers() is there exactly for this reason. > > > > You recently wrote: > > > > > > > Nope. pci_prepare_to_sleep() is called during suspend, from > > > > > pci_pm_default_suspend(), with interrupts on. > > > > Why can't we allocate memory in pci_prepare_to_sleep()? > > Hm, in fact we can. Well, this is not correct. :-( Although interrupts are enabled when we call pci_save_state(), we can't use GFP_KERNEL for allocating memory in there, because all storage devices may have already been suspended at this point. While we can use GFP_NOIO, I don't think it'll help much. I think the solution could be to allocate memory from the ->prepare() callback of the PCI bus type driver. Also, I thought a bit about what we'd need to store permanently in RAM to be able to handle MSI-X and it seemed to me that an array of allocated interrupt vectors (the number of entries equal to the number of allocated vectors) and a bitmap of allocated MSI-X table entries would be sufficient. If you already have some patches reworking the MSI-X suspend-resume, please point me to them if possible. Thanks, Rafael -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
