Hi ; not quite sure where to ask so will start here... Some context first. I'm inquiring in the context of mckernel[1], a lightweight kernel that works next to linux (basically offlines a few/most cores, reserve some memory and have boot a second OS on that to run HPC applications). Being brutally honest here, this is mostly research and anyone here looking into it will probably scream, but I might as well try not to add too many more reasons to do so.... One of the mecanisms here is that sometimes we want to access the mckernel memory from linux (either from the process that spawned the mckernel side process or from a driver in linux), and to do that we have mapped the mckernel side virtual memory range to that process so it can page fault. The (horrible) function doing that can be found here[2], rus_vm_fault - sends a message to the other side to identify the physical address corresponding from what we had reserved earlier and map it quite manually. We could know at this point if it had been a huge page (very likely) or not; I'm observing a huge difference of performance with some interconnect if I add a huge kludge emulating huge pages here (directly manipulating the process' page table) so I'd very much like to use huge pages when we know a huge page has been mapped on the other side. What I'd like to know is: - we know (assuming the other side isn't too bugged, but if it is we're fucked up anyway) exactly what huge-page-sized physical memory range has been mapped on the other side, is there a way to manually gather the pages corresponding and merge them into a huge page? - from what I understand that does not seem possible/recommended, the way to go being to have a userland process get huge pages and pass these to a device (ioctl or something); but I assume that means said process needs to keep on running all the time that memory is required? If the page fault needs to split the page (because the other side handed a "small" page so we can only map a regular page here), can it be merged back into a huge page for the next time this physical region is used? [1] https://github.com/RIKEN-SysSoft/mckernel [2] https://github.com/RIKEN-SysSoft/mckernel/blob/development/executer/kernel/mcctrl/syscall.c#L538 Any input will be appreciated, -- Dominique Martinet
