Forgot the link. [1] https://github.com/westerndigitalcorporation/hmmap Take care, Adam On Thu, 2019-02-21 at 15:11 -0800, Adam Manzanares wrote: > Hello, > > I would like to attend the LSF/MM Summit 2019. I'm interested in > several MM topics that are mentioned below as well as Zoned Block > Devices and any io determinism topics that come up in the storage > track. > > I have been working on a caching layer, hmmap (heterogeneous memory > map) [1], for emerging NVM and it is in spirit close to the page > cache. The key difference being that the backend device and caching > layer of hmmap is pluggable. In addition, hmmap supports DAX and > write > protection, which I believe are key features for emerging NVMs that > may > have write/read asymmetry as well as write endurance constraints. > Lastly we can leverage hardware, such as a DMA engine, when moving > pages between the cache while also allowing direct access if the > device > is capable. > > I am proposing that as an alternative to using NVMs as a NUMA node > we expose the NVM through the page cache or a viable alternative and > have userspace applications mmap the NVM and hand out memory with > their favorite userspace memory allocator. > > This would isolate the NVMs to only applications that are well aware > of the performance implications of accessing NVM. I believe that all > of this work could be solved with the NUMA node approach, but the two > approaches are seeming to blur together. > > The main points I would like to discuss are: > > * Is the page cache model a viable alternative to NVM as a NUMA NODE? > * Can we add more flexibility to the page cache? > * Should we force separation of NVM through an explicit mmap? > > I believe this discussion could be merged with NUMA, memory hierarchy > and device memory, Use NVDIMM as NUMA node and NUMA API, or memory > reclaim with NUMA balancing. > > Here are some performance numbers of hmmap (in development): > > All numbers are collected on a 4GiB hmmap device with a 128MiB cache. > For the mmap tests I used cgroups to limit the page cache usage to > 128MiB. All results are an average of 10 runs. W and R access the > entire device with all threads segregated in the address space. RR > reads the entire device randomly 8 bytes at a time and is limited to > 8MiB of data accessed. > > hmmap brd vs. mmap of brd > > hmmap mmap > > Threads W R RR W R RR > > 1 7.21 5.39 5.04 6.80 5.63 5.23 > 2 5.19 3.87 3.74 4.66 3.33 3.20 > 4 3.65 2.95 3.07 3.53 2.26 2.18 > 8 4.52 3.43 3.59 4.30 1.98 1.88 > 16 5.00 3.85 3.98 4.92 2.00 1.99 > > > > Memory Backend Test (Dax capable) > > hmmap hmmap-dax hmmap-wrprotect > > Threads W R RR W R RR W R RR > > 1 6.29 4.94 4.37 2.54 1.36 0.16 7.12 2.13 0.73 > 2 4.62 3.63 3.57 1.41 0.69 0.08 5.06 1.14 0.41 > 4 3.45 2.97 3.11 0.77 0.36 0.04 3.66 0.63 0.25 > 8 4.10 3.53 3.71 0.44 0.19 0.02 4.03 0.35 0.17 > 16 4.60 3.98 4.04 0.34 0.16 0.02 4.52 0.27 0.14 > > > Thanks, > Adam > > > >
