On Mon 06-05-24 20:37:19, David Rientjes wrote: > Hi all, > > I think it would be very worthwhile to have a block set aside for > discussion on locally attached memory tiering extensions at LSF/MM/BPF > 2024. > > Primarily interested in discussing Linux enlightenment for CXL 1.1 and > later type-3 memory expansion devices (CXL.mem). I think we could touch > on CXL 2.0 and later memory pooling architectures if we have time and > there is interest, but the primary focus here would be local attached. > > Based on the premise for a Memory Tiering Working Group[1], there is > widespread interest in the foundational topics for generally useful Linux > enlightenment: > > - Decoupling CPU balancing from memory balancing (or obsoleting CPU > balancing entirely) > > + John Hubbard notes this would be useful for GPUs: > > a) GPUs have their own processors that are invisible to the kernel's > NUMA "which tasks are active on which NUMA nodes" calculations, > and > > b) Similar to where CXL is generally going, we have already built > fully memory-coherent hardware, which include memory-only NUMA > nodes. > > - In-kernel hot memory abstraction, informed by hardware hinting drivers > (incl some architectures like Power10), usable as a NUMA Balancing > backend for promotion and other areas of the kernel like transparent > hugepage utilization > > - NUMA and memory tiering enlightenment for accelerators, such as for > optimal use of GPU memory, extremely important for a cloud provider > (hint hint :) > > - Asynchronous memory promotion independent of task_numa_fault() while > considering the cost of page migration (due to identifying cold memory) > > - What the role of userspace plays in this decision-making and how we can > extend the default policy and mechanisms in the kernel to allow for it > if necessary > > Additional topics that you find interesting are also very helpful! > > I'm biased toward a generally useful solution that would leverage the > kernel as the ultimate source of truth for page hotness that can be > extended for multiple use caes, one of which is memory tiering support. > But certainly if there are other approaches, we can discuss that as well. > > A few main goals from this discussion: > > - Ensure that proposals address, or can be extended to address, the > emerging needs of the various use cases that users may have > > - Surface any constraints that stakeholders may find to be prohibitive > for support in the core MM subsystem > > - Alignment and division of work for developers who are actively looking > to contribute to this area Do you think having 2 contigious slots would be sufficient for these topics? > As I'm just one of many stakeholders for this discussion, I'd nominate > Michal Hocko to moderate it if he's willing to do so. Sure I can help out with that. -- Michal Hocko SUSE Labs
