RE(3): FW: [LSF/MM/BPF TOPIC] SMDK inspired MM changes for CXL

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



>On 24.03.23 10:27, Kyungsan Kim wrote:
>>> On 24.03.23 10:09, Kyungsan Kim wrote:
>>>> Thank you David Hinderbrand for your interest on this topic.
>>>>
>>>>>>
>>>>>>> Kyungsan Kim wrote:
>>>>>>> [..]
>>>>>>>>> In addition to CXL memory, we may have other kind of memory in the
>>>>>>>>> system, for example, HBM (High Bandwidth Memory), memory in FPGA card,
>>>>>>>>> memory in GPU card, etc.  I guess that we need to consider them
>>>>>>>>> together.  Do we need to add one zone type for each kind of memory?
>>>>>>>>
>>>>>>>> We also don't think a new zone is needed for every single memory
>>>>>>>> device.  Our viewpoint is the sole ZONE_NORMAL becomes not enough to
>>>>>>>> manage multiple volatile memory devices due to the increased device
>>>>>>>> types.  Including CXL DRAM, we think the ZONE_EXMEM can be used to
>>>>>>>> represent extended volatile memories that have different HW
>>>>>>>> characteristics.
>>>>>>>
>>>>>>> Some advice for the LSF/MM discussion, the rationale will need to be
>>>>>>> more than "we think the ZONE_EXMEM can be used to represent extended
>>>>>>> volatile memories that have different HW characteristics". It needs to
>>>>>>> be along the lines of "yes, to date Linux has been able to describe DDR
>>>>>>> with NUMA effects, PMEM with high write overhead, and HBM with improved
>>>>>>> bandwidth not necessarily latency, all without adding a new ZONE, but a
>>>>>>> new ZONE is absolutely required now to enable use case FOO, or address
>>>>>>> unfixable NUMA problem BAR." Without FOO and BAR to discuss the code
>>>>>>> maintainability concern of "fewer degress of freedom in the ZONE
>>>>>>> dimension" starts to dominate.
>>>>>>
>>>>>> One problem we experienced was occured in the combination of hot-remove and kerelspace allocation usecases.
>>>>>> ZONE_NORMAL allows kernel context allocation, but it does not allow hot-remove because kernel resides all the time.
>>>>>> ZONE_MOVABLE allows hot-remove due to the page migration, but it only allows userspace allocation.
>>>>>> Alternatively, we allocated a kernel context out of ZONE_MOVABLE by adding GFP_MOVABLE flag.
>>>>
>>>>> That sounds like a bad hack :) .
>>>> I consent you.
>>>>
>>>>>> In case, oops and system hang has occasionally occured because ZONE_MOVABLE can be swapped.
>>>>>> We resolved the issue using ZONE_EXMEM by allowing seletively choice of the two usecases.
>>>>
>>>>> I once raised the idea of a ZONE_PREFER_MOVABLE [1], maybe that's
>>>>> similar to what you have in mind here. In general, adding new zones is
>>>>> frowned upon.
>>>>
>>>> Actually, we have already studied your idea and thought it is similar with us in 2 aspects.
>>>> 1. ZONE_PREFER_MOVABLE allows a kernelspace allocation using a new zone
>>>> 2. ZONE_PREFER_MOVABLE helps less fragmentation by splitting zones, and ordering allocation requests from the zones.
>>>>
>>>> We think ZONE_EXMEM also helps less fragmentation.
>>>> Because it is a separated zone and handles a page allocation as movable by default.
>>>
>>> So how is it different that it would justify a different (more confusing
>>> IMHO) name? :) Of course, names don't matter that much, but I'd be
>>> interested in which other aspect that zone would be "special".
>>
>> FYI for the first time I named it as ZONE_CXLMEM, but we thought it would be needed to cover other extended memory types as well.
>> So I changed it as ZONE_EXMEM.
>> We also would like to point out a "special" zone aspeact, which is different from ZONE_NORMAL for tranditional DDR DRAM.
>> Of course, a symbol naming is important more or less to represent it very nicely, though.
>> Do you prefer ZONE_SPECIAL? :)
>
>I called it ZONE_PREFER_MOVABLE. If you studied that approach there must
>be a good reason to name it differently?
>

The intention of ZONE_EXMEM is a separated logical management dimension originated from the HW diffrences of extended memory devices.
Althought the ZONE_EXMEM considers the movable and frementation aspect, it is not all what ZONE_EXMEM considers.
So it is named as it.




[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [NTFS 3]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [NTFS 3]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux