On Fri, Jul 10, 2009 at 09:56:29AM +0200, Rob van de Voort wrote: > > Hello, > > I have the following question regarding the (re-)allocation of dma-able > memory for a PCIe device. I wish to know on which CPU / Memory > controller the dma-able memory will be allocated. There are several kinds of dma-able memory. Coherent/consistent memory (pci_alloc_consistent, dma_alloc_coherent) This is usually allocated on the node where the device is connected to, if the platform reports it correctly using ACPI and there is enough memory available on that node. In some kernels there were bugs with this. All other memory, later mapped with pci_map_*/dma_map_*: This is allocated according to the current numa memory policy, unless you override it (alloc_pages_node etc.) > The system runs a 64bit linux kernel. Furthermore a PCIe device which > can only use 32 bit addressing is connected to the PCIe bus. I want to GFP_DMA32 allocations are typically only on the first node, unless your first node has less than 3GB of memory or so. The only thing you can do for 4GB limited allocation devices is to enable VT-d and allocate it elsewhere (alloc_pages_node et.al.) and then let it be remapped by pci_map*/dma_map* into the first 4GB. > P.S I'd be grateful if people could point me in the direction of > resources I could read. Thus far I have gone over the PCI / DMA http://halobates.de/numaapi3.pdf has a general overview of NUMA policy, although from the user space perspective. -Andi -- ak@xxxxxxxxxxxxxxx -- Speaking for myself only. -- To unsubscribe from this list: send the line "unsubscribe linux-numa" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
