On 12.04.23 13:10, Kyungsan Kim wrote:
Gregory Price <gregory.price@xxxxxxxxxxxx> writes:
On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote:
Gregory Price <gregory.price@xxxxxxxxxxxx> writes:
[snip]
2. During the migration process, the memory needs to be forced not to be
migrated to another node by other means (tiering software, swap,
etc). The obvious way of doing this would be to migrate and
temporarily pin the page... but going back to problem #1 we see that
ZONE_MOVABLE and Pinning are mutually exclusive. So that's
troublesome.
Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to
avoid move pages out of CXL.mem node? Now, there are gaps in tiering,
but I think it is fixable.
Best Regards,
Huang, Ying
[snip]
That feels like a hack/bodge rather than a proper solution to me.
Maybe this is an affirmative argument for the creation of an EXMEM
zone.
Let's start with requirements. What is the requirements for a new zone
type?
I'm stills scratching my head regarding this. I keep hearing all
different kind of statements that just add more confusions "we want it
to be hotunpluggable" "we want to allow for long-term pinning memory"
"but we still want it to be movable" "we want to place some unmovable
allocations on it". Huh?
Just to clarify: ZONE_MOVABLE allows for pinning. It just doesn't allow
for long-term pinning of memory.
For good reason, because long-term pinning of memory is just the worst
(memory waste, fragmentation, overcommit) and instead of finding new
ways to *avoid* long-term pinnings, we're coming up with advanced
concepts to work-around the fundamental property of long-term pinnings.
We want all memory to be long-term pinnable and we want all memory to be
movable/hotunpluggable. That's not going to work.
Looks there is misunderstanding about ZONE_EXMEM argument.
Pinning and plubbability is mutual exclusive so it can not happen at the same time.
What we argue is ZONE_EXMEM does not "confine movability". an allocation context can determine the movability attribute.
Even one unmovable allocation will make the entire CXL DRAM unpluggable.
When you see ZONE_EXMEM just on movable/unmoable aspect, we think it is the same with ZONE_NORMAL,
but ZONE_EXMEM works on an extended memory, as of now CXL DRAM.
Then why ZONE_EXMEM is, ZONE_EXMEM considers not only the pluggability aspect, but CXL identifier for user/kenelspace API,
the abstraction of multiple CXL DRAM channels, and zone unit algorithm for CXL HW characteristics.
The last one is potential at the moment, though.
As mentioned in ZONE_EXMEM thread, we are preparing slides to explain experiences and proposals.
It it not final version now[1].
[1] https://github.com/OpenMPDK/SMDK/wiki/93.-%5BLSF-MM-BPF-TOPIC%5D-SMDK-inspired-MM-changes-for-CXL
Yes, hopefully we can discuss at LSF/MM also the problems we are trying
to solve instead of focusing on one solution. [did not have the time to
look at the slides yet, sorry]
--
Thanks,
David / dhildenb
