On 1/27/25 2:33 PM, Andrew Morton wrote:
On Fri, 24 Jan 2025 15:54:34 -0800 Anthony Yznaga <anthony.yznaga@xxxxxxxxxx> wrote:Memory pages shared between processes require page table entries (PTEs) for each process. Each of these PTEs consume some of the memory and as long as the number of mappings being maintained is small enough, this space consumed by page tables is not objectionable. When very few memory pages are shared between processes, the number of PTEs to maintain is mostly constrained by the number of pages of memory on the system. As the number of shared pages and the number of times pages are shared goes up, amount of memory consumed by page tables starts to become significant. This issue does not apply to threads. Any number of threads can share the same pages inside a process while sharing the same PTEs. Extending this same model to sharing pages across processes can eliminate this issue for sharing across processes as well. ... API === mshare does not introduce a new API. It instead uses existing APIs to implement page table sharing. The steps to use this feature are: 1. Mount msharefs on /sys/fs/mshare - mount -t msharefs msharefs /sys/fs/mshare 2. mshare regions have alignment and size requirements. Start address for the region must be aligned to an address boundary and be a multiple of fixed size. This alignment and size requirement can be obtained by reading the file /sys/fs/mshare/mshare_info which returns a number in text format. mshare regions must be aligned to this boundary and be a multiple of this size. 3. For the process creating an mshare region: a. Create a file on /sys/fs/mshare, for example - fd = open("/sys/fs/mshare/shareme", O_RDWR|O_CREAT|O_EXCL, 0600); b. Establish the starting address and size of the region struct mshare_info minfo; minfo.start = TB(2); minfo.size = BUFFER_SIZE; ioctl(fd, MSHAREFS_SET_SIZE, &minfo) c. Map some memory in the region struct mshare_create mcreate; mcreate.addr = TB(2); mcreate.size = BUFFER_SIZE; mcreate.offset = 0; mcreate.prot = PROT_READ | PROT_WRITE; mcreate.flags = MAP_ANONYMOUS | MAP_SHARED | MAP_FIXED; mcreate.fd = -1; ioctl(fd, MSHAREFS_CREATE_MAPPING, &mcreate)I'm not really understanding why step a exists. It's basically an mmap() so why can't this be done within step d?
One way to think of it is that step d establishes a window to the mshare region and the objects mapped within it.
Discussions on earlier iterations of mshare pushed back strongly on introducing special casing in the mmap path to redirect mmaps that fell within an mshare region to map into an mshare mm. Even then it gets messier for munmap, i.e. does an unmap of the whole range mean unmap the window or unmap the objects within it.
d. Map the mshare region into the process mmap((void *)TB(2), BUF_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); e. Write and read to mshared region normally. 4. For processes attaching an mshare region: a. Open the file on msharefs, for example - fd = open("/sys/fs/mshare/shareme", O_RDWR); b. Get information about mshare'd region from the file: struct mshare_info minfo; ioctl(fd, MSHAREFS_GET_SIZE, &minfo); c. Map the mshare'd region into the process mmap(minfo.start, minfo.size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); 5. To delete the mshare region - unlink("/sys/fs/mshare/shareme");The userspace intergace is the thing we should initially consider. I'm having ancient memories of hugetlbfs. Over time it was seen that hugetlbfs was too standalone and huge pages became more (and more (and more (and more))) integrated into regular MM code. Can we expect a similar evolution with pte-shared memory and if so, is this the correct interface to be starting out with?
I don't know. This is an approach that has been refined through a number of discussions, but I'm certainly open to alternatives.
Anthony
