>On 24.03.23 10:09, Kyungsan Kim wrote: >> Thank you David Hinderbrand for your interest on this topic. >> >>>> >>>>> Kyungsan Kim wrote: >>>>> [..] >>>>>>> In addition to CXL memory, we may have other kind of memory in the >>>>>>> system, for example, HBM (High Bandwidth Memory), memory in FPGA card, >>>>>>> memory in GPU card, etc. I guess that we need to consider them >>>>>>> together. Do we need to add one zone type for each kind of memory? >>>>>> >>>>>> We also don't think a new zone is needed for every single memory >>>>>> device. Our viewpoint is the sole ZONE_NORMAL becomes not enough to >>>>>> manage multiple volatile memory devices due to the increased device >>>>>> types. Including CXL DRAM, we think the ZONE_EXMEM can be used to >>>>>> represent extended volatile memories that have different HW >>>>>> characteristics. >>>>> >>>>> Some advice for the LSF/MM discussion, the rationale will need to be >>>>> more than "we think the ZONE_EXMEM can be used to represent extended >>>>> volatile memories that have different HW characteristics". It needs to >>>>> be along the lines of "yes, to date Linux has been able to describe DDR >>>>> with NUMA effects, PMEM with high write overhead, and HBM with improved >>>>> bandwidth not necessarily latency, all without adding a new ZONE, but a >>>>> new ZONE is absolutely required now to enable use case FOO, or address >>>>> unfixable NUMA problem BAR." Without FOO and BAR to discuss the code >>>>> maintainability concern of "fewer degress of freedom in the ZONE >>>>> dimension" starts to dominate. >>>> >>>> One problem we experienced was occured in the combination of hot-remove and kerelspace allocation usecases. >>>> ZONE_NORMAL allows kernel context allocation, but it does not allow hot-remove because kernel resides all the time. >>>> ZONE_MOVABLE allows hot-remove due to the page migration, but it only allows userspace allocation. >>>> Alternatively, we allocated a kernel context out of ZONE_MOVABLE by adding GFP_MOVABLE flag. >> >>> That sounds like a bad hack :) . >> I consent you. >> >>>> In case, oops and system hang has occasionally occured because ZONE_MOVABLE can be swapped. >>>> We resolved the issue using ZONE_EXMEM by allowing seletively choice of the two usecases. >> >>> I once raised the idea of a ZONE_PREFER_MOVABLE [1], maybe that's >>> similar to what you have in mind here. In general, adding new zones is >>> frowned upon. >> >> Actually, we have already studied your idea and thought it is similar with us in 2 aspects. >> 1. ZONE_PREFER_MOVABLE allows a kernelspace allocation using a new zone >> 2. ZONE_PREFER_MOVABLE helps less fragmentation by splitting zones, and ordering allocation requests from the zones. >> >> We think ZONE_EXMEM also helps less fragmentation. >> Because it is a separated zone and handles a page allocation as movable by default. > >So how is it different that it would justify a different (more confusing >IMHO) name? :) Of course, names don't matter that much, but I'd be >interested in which other aspect that zone would be "special". FYI for the first time I named it as ZONE_CXLMEM, but we thought it would be needed to cover other extended memory types as well. So I changed it as ZONE_EXMEM. We also would like to point out a "special" zone aspeact, which is different from ZONE_NORMAL for tranditional DDR DRAM. Of course, a symbol naming is important more or less to represent it very nicely, though. Do you prefer ZONE_SPECIAL? :) > >-- >Thanks, > >David / dhildenb >
