Thank you dan for kind reminder of the submission. I filled out the form with topic suggestions and required attendees. Hopefully, we can elaborate the topics with wider opinions revisiting previous kernel designs related. >Please be sure to log this in the submission spreadsheet as well. From the >CFP: > >--- > >1) Fill out the following Google form to request attendance and >suggest any topics > > https://forms.gle/VKVXjWGBHZbnsz226 > >In previous years we have accidentally missed people's attendance >requests because they either didn't cc lsf-pc@ or we simply missed them >in the flurry of emails we get. Our community is large and our >volunteers are busy, filling this out will help us make sure we don't >miss anybody. > > >Kyungsan Kim wrote: >> CXL is a promising technology that leads to fundamental changes in >> computing architecture. To facilitate adoption and widespread of CXL >> memory, we are developing a memory tiering solution, called >> SMDK[1][2]. Using SMDK and CXL RAM device, our team has been working >> with industry and academic partners over last year. Also, thanks to >> many researcher's effort, CXL adoption stage is gradually moving >> forward from basic enablement to real-world composite usecases. At >> this moment, based on the researches and experiences gained working on >> SMDK, we would like to suggest a session at LSF/MM/BFP this year to >> propose possible Linux MM changes with a brief of SMDK. >> >> Adam Manzanares kindly adviced me that it is preferred to discuss >> implementation details on given problem and consensus at LSF/MM/BFP. >> Considering the adoption stage of CXL technology, however, let me >> suggest a design level discussion on the two MM expansions of SMDK >> this year. When we have design consensus with participants, we want >> to continue follow-up discussions with additional implementation >> details, hopefully. >> >> >> 1. A new zone, ZONE_EXMEM We added ZONE_EXMEM to manage CXL RAM >> device(s), separated from ZONE_NORMAL for usual DRAM due to the three >> reasons below. >> >> 1) a CXL RAM has many different characteristics with conventional DRAM >> because a CXL device inherits and expands PCIe specification. ex) >> frequency range, pluggability, link speed/width negotiation, >> host/device flow control, power throttling, channel-interleaving >> methodology, error handling, and etc. It is likely that the primary >> usecase of CXL RAM would be System RAM. However, to deal with the >> hardware differences properly, different MM algorithms are needed >> accordingly. >> >> 2) Historically, zone has been expanded by reflecting the evolution of >> CPU, IO, and memory devices. ex) ZONE_DMA(32), ZONE_HIGHMEM, >> ZONE_DEVICE, and ZONE_MOVABLE. Each zone applies different MM >> algorithms such as page reclaim, compaction, migration, and >> fragmentation. At first, we tried reuse of existing zones, >> ZONE_DEVICE and ZONE_MOVABLE, for CXL RAM purpose. However, the >> purpose and implementation of the zones are not fit for CXL RAM. >> >> 3) Industry is preparing a CXL-capable system that connects dozens of >> CXL devices in a server system. When a CXL device becomes a separate >> node, an administrator/programmer needs to be aware of and manually >> control all nodes using 3rd party software, such as numactl and >> libnuma. ZONE_EXMEM allows the assemble of CXL RAM devices into the >> single ZONE_EXMEM zone, and provides an abstraction to userspace by >> seamlessly managing the devices. Also, the zone is able to interleave >> assembled devices in a software way to lead to aggregated bandwidth. >> We would like to suggest if it is co-existable with HW interleaving >> like SW/HW raid0. To help understanding, please refer to the node >> partition part of the picture[3]. >> >> >> 2. User/Kernelspace Programmable Interface In terms of a memory >> tiering solution, it is typical that the solution attempts to locate >> hot data on near memory, and cold data on far memory as accurately as >> possible.[4][5][6][7] We noticed that the hot/coldness of data is >> determined by the memory access pattern of running application and/or >> kernel context. Hence, a running context needs a near/far memory >> identifier to determine near/far memory. When CXL RAM(s) is >> manipulated as a NUMA node, a node id can be function as a CXL >> identifier more or less. However, the node id has limitation in that >> it is an ephemeral information that dynamically varies according to >> online status of CXL topology and system socket. In this sense, we >> provides programmable interfaces for userspace and kernelspace context >> to explicitly (de)allocate memory from DRAM and CXL RAM regardless of >> a system change. Specifically, MAP_EXMEM and GFP_EXMEM flags were >> added to mmap() syscall and kmalloc() siblings, respectively. >> >> Thanks to Adam Manzanares for reviewing this CFP thoroughly. >> >> >> [1]SMDK: https://github.com/openMPDK/SMDK >> [2]SMT: Software-defined Memory Tiering for Heterogeneous Computing systems with CXL Memory Expander, https://ieeexplore.ieee.org/document/10032695 >> [3]SMDK node partition: https://github.com/OpenMPDK/SMDK/wiki/2.-SMDK-Architecture#memory-partition >> [4]TMO: Transparent Memory Offloading in Datacenters, https://dl.acm.org/doi/10.1145/3503222.3507731 >> [5]TPP: Transparent Page Placement for CXL-Enabled Tiered Memory, https://arxiv.org/abs/2206.02878 >> [6]Pond: CXL-Based Memory Pooling Systems for Cloud Platforms, https://dl.acm.org/doi/10.1145/3575693.3578835 >> [7]Hierarchical NUMA: https://blog.linuxplumbersconf.org/2017/ocw/system/presentations/4656/original/Hierarchical_NUMA_Design_Plumbers_2017.pdf >
