On Wed, May 25, 2022 at 4:48 AM Jonathan Cameron <Jonathan.Cameron@xxxxxxxxxx> wrote: > > On Wed, 25 May 2022 17:47:33 +1000 > Alistair Popple <apopple@xxxxxxxxxx> wrote: > > > Wei Xu <weixugc@xxxxxxxxxx> writes: > > > > > On Tue, May 24, 2022 at 6:27 AM Aneesh Kumar K.V > > > <aneesh.kumar@xxxxxxxxxxxxx> wrote: > > >> > > >> Wei Xu <weixugc@xxxxxxxxxx> writes: > > >> > > >> > On Wed, May 18, 2022 at 5:00 AM Jonathan Cameron > > >> > <Jonathan.Cameron@xxxxxxxxxx> wrote: > > >> >> > > >> >> On Wed, 18 May 2022 00:09:48 -0700 > > >> >> Wei Xu <weixugc@xxxxxxxxxx> wrote: > > >> > > >> ... > > >> > > >> > Nice :) > > >> >> > > >> >> Initially I thought this was over complicated when compared to just leaving space, but > > >> >> after a chat with Hesham just now you have us both convinced that this is an elegant solution. > > >> >> > > >> >> Few corners probably need fleshing out: > > >> >> * Use of an allocator for new tiers. Flat number at startup, or new one on write of unique > > >> >> value to set_memtier perhaps? Also whether to allow drivers to allocate (I think > > >> >> we should). > > >> >> * Multiple tiers with same rank. My assumption is from demotion path point of view you > > >> >> fuse them (treat them as if they were a single tier), but keep them expressed > > >> >> separately in the sysfs interface so that the rank can be changed independently. > > >> >> * Some guidance on what values make sense for given rank default that might be set by > > >> >> a driver. If we have multiple GPU vendors, and someone mixes them in a system we > > >> >> probably don't want the default values they use to result in demotion between them. > > >> >> This might well be a guidance DOC or appropriate set of #define > > >> > > > >> > All of these are good ideas, though I am afraid that these can make > > >> > tier management too complex for what it's worth. > > >> > > > >> > How about an alternative tier numbering scheme that uses major.minor > > >> > device IDs? For simplicity, we can just start with 3 major tiers. > > >> > New tiers can be inserted in-between using minor tier IDs. > > >> > > >> > > >> What drives the creation of a new memory tier here? Jonathan was > > >> suggesting we could do something similar to writing to set_memtier for > > >> creating a new memory tier. > > >> > > >> $ echo "memtier128" > sys/devices/system/node/node1/set_memtier > > >> > > >> But I am wondering whether we should implement that now. If we keep > > >> "rank" concept and detach tier index (memtier0 is the memory tier with > > >> index 0) separate from rank, I assume we have enough flexibility for a > > >> future extension that will allow us to create a memory tier from userspace > > >> and assigning it a rank value that helps the device to be placed before or > > >> after DRAM in demotion order. > > >> > > >> ie, For now we will only have memtier0, memtier1, memtier2. We won't add > > >> dynamic creation of memory tiers and the above memory tiers will have > > >> rank value 0, 1, 2 according with demotion order 0 -> 1 -> 2. > > > > > > Great. So the consensus is to go with the "rank" approach. The above > > > sounds good to me as a starting point. > > > > The rank approach seems good to me too. > > Rank is good, but I do slightly worry about accidentally defining ABI > that people care about with the particular numbers used for the initial ranks. > > Maybe just x100 on all of them to allow things in between with no change to > this initial set of 3? So 0, 100, 200 I strongly support this, which is also my original intention for rank values. I'd suggest to even remove 0 to avoid it becoming a special value that userspace depends on. > Jonathan > > > > > - Alistair > > > > >> -aneesh >
