>> Gregory Price <gregory.price@xxxxxxxxxxxx> writes: >> >>> On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote: >>>> Gregory Price <gregory.price@xxxxxxxxxxxx> writes: >>>> >>>> [snip] >>>> >>>>> 2. During the migration process, the memory needs to be forced not to be >>>>> migrated to another node by other means (tiering software, swap, >>>>> etc). The obvious way of doing this would be to migrate and >>>>> temporarily pin the page... but going back to problem #1 we see that >>>>> ZONE_MOVABLE and Pinning are mutually exclusive. So that's >>>>> troublesome. >>>> >>>> Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to >>>> avoid move pages out of CXL.mem node? Now, there are gaps in tiering, >>>> but I think it is fixable. >>>> >>>> Best Regards, >>>> Huang, Ying >>>> >>>> [snip] >>> >>> That feels like a hack/bodge rather than a proper solution to me. >>> >>> Maybe this is an affirmative argument for the creation of an EXMEM >>> zone. >> >> Let's start with requirements. What is the requirements for a new zone >> type? > >I'm stills scratching my head regarding this. I keep hearing all >different kind of statements that just add more confusions "we want it >to be hotunpluggable" "we want to allow for long-term pinning memory" >"but we still want it to be movable" "we want to place some unmovable >allocations on it". Huh? > >Just to clarify: ZONE_MOVABLE allows for pinning. It just doesn't allow >for long-term pinning of memory. > >For good reason, because long-term pinning of memory is just the worst >(memory waste, fragmentation, overcommit) and instead of finding new >ways to *avoid* long-term pinnings, we're coming up with advanced >concepts to work-around the fundamental property of long-term pinnings. > >We want all memory to be long-term pinnable and we want all memory to be >movable/hotunpluggable. That's not going to work. Looks there is misunderstanding about ZONE_EXMEM argument. Pinning and plubbability is mutual exclusive so it can not happen at the same time. What we argue is ZONE_EXMEM does not "confine movability". an allocation context can determine the movability attribute. Even one unmovable allocation will make the entire CXL DRAM unpluggable. When you see ZONE_EXMEM just on movable/unmoable aspect, we think it is the same with ZONE_NORMAL, but ZONE_EXMEM works on an extended memory, as of now CXL DRAM. Then why ZONE_EXMEM is, ZONE_EXMEM considers not only the pluggability aspect, but CXL identifier for user/kenelspace API, the abstraction of multiple CXL DRAM channels, and zone unit algorithm for CXL HW characteristics. The last one is potential at the moment, though. As mentioned in ZONE_EXMEM thread, we are preparing slides to explain experiences and proposals. It it not final version now[1]. [1] https://github.com/OpenMPDK/SMDK/wiki/93.-%5BLSF-MM-BPF-TOPIC%5D-SMDK-inspired-MM-changes-for-CXL >If you'd ask me today, my prediction is that ZONE_EXMEM is not going to >happen. > >-- >Thanks, > >David / dhildenb
