Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Mon, May 01, 2017 at 02:56:34PM -0700, John Hubbard wrote:
On 05/01/2017 02:04 PM, Reza Arbab wrote:
On Mon, May 01, 2017 at 01:41:55PM -0700, John Hubbard wrote:
1. A way to move pages between NUMA nodes, both virtual address and physical address-based, from kernel mode.

Jérôme's migrate_vma() and migrate_dma() should have this covered, including DMA-accelerated copy.

Yes, that's good. I wasn't sure from this discussion here if either or both of those would be used, but now I see.

Are those APIs ready for moving pages between NUMA nodes? As there is no NUMA node id in the API, are we relying on the pages' membership (using each page and updating which node it is on)?

Yes. Those APIs work by callback. The alloc_and_copy() function you provide will be called at the appropriate point in the migration. Yours would allocate from a specific destination node, and copy using DMA.

5. Something to handle the story of bringing NUMA nodes online and putting them back offline, given that they require a device driver that may not yet have been loaded. There are a few minor missing bits there.

This has been prototyped with the driver doing memory hotplug/hotremove. Could you elaborate a little on what you feel is missing?


We just worked through how to deal with this in our driver, and I remember feeling worried about the way NUMA nodes can only be put online via a user space action (through sysfs). It seemed like you'd want to do that from kernel as well, when a device driver gets loaded.

That's true. I don't think we have a way to online/offline from a driver. To online, the alternatives are memhp_auto_online (incapable of doing online_movable), or udev rules (not ideal in this driver controlled memory use case). To offline, nothing that I know of.

I was also uneasy about user space trying to bring a node online before the associated device driver was loaded, and I think it would be nice to be sure that that whole story is looked at.

The theme here is that driver load/unload is, today, independent from the NUMA node online/offline, and that's a problem. Not a huge one, though, just worth enumerating here.

--
Reza Arbab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>



[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]
  Powered by Linux