On Thu, Mar 1, 2018 at 12:34 PM, Benjamin Herrenschmidt <benh@xxxxxxxxxxx> wrote: > On Thu, 2018-03-01 at 11:21 -0800, Dan Williams wrote: >> On Wed, Feb 28, 2018 at 7:56 PM, Benjamin Herrenschmidt >> <benh@xxxxxxxxxxx> wrote: >> > On Thu, 2018-03-01 at 14:54 +1100, Benjamin Herrenschmidt wrote: >> > > On Wed, 2018-02-28 at 16:39 -0700, Logan Gunthorpe wrote: >> > > > Hi Everyone, >> > > >> > > >> > > So Oliver (CC) was having issues getting any of that to work for us. >> > > >> > > The problem is that acccording to him (I didn't double check the latest >> > > patches) you effectively hotplug the PCIe memory into the system when >> > > creating struct pages. >> > > >> > > This cannot possibly work for us. First we cannot map PCIe memory as >> > > cachable. (Note that doing so is a bad idea if you are behind a PLX >> > > switch anyway since you'd ahve to manage cache coherency in SW). >> > >> > Note: I think the above means it won't work behind a switch on x86 >> > either, will it ? >> >> The devm_memremap_pages() infrastructure allows placing the memmap in >> "System-RAM" even if the hotplugged range is in PCI space. So, even if >> it is an issue on some configurations, it's just a simple adjustment >> to where the memmap is placed. > > But what happens with that PCI memory ? Is it effectively turned into > nromal memory (ie, usable for normal allocations, potentially used to > populate user pages etc...) or is it kept aside ? > > Also on ppc64, the physical addresses of PCIe make it so far appart > that there's no way we can map them into the linear mapping at the > normal offset of PAGE_OFFSET + (pfn << PAGE_SHIFT), so things like > page_address or virt_to_page cannot work as-is on PCIe addresses. Ah ok, I'd need to look at the details. I had been assuming that sparse-vmemmap could handle such a situation, but that could indeed be a broken assumption.
