On Thu, May 26, 2022 at 2:32 AM Jonathan Cameron <Jonathan.Cameron@xxxxxxxxxx> wrote: > > On Wed, 25 May 2022 10:27:42 -0700 > Wei Xu <weixugc@xxxxxxxxxx> wrote: > > > On Wed, May 25, 2022 at 3:01 AM Aneesh Kumar K V > > <aneesh.kumar@xxxxxxxxxxxxx> wrote: > > > > > > On 5/25/22 2:33 PM, Ying Huang wrote: > > > > On Tue, 2022-05-24 at 22:32 -0700, Wei Xu wrote: > > > >> On Tue, May 24, 2022 at 1:24 AM Ying Huang <ying.huang@xxxxxxxxx> wrote: > > > >>> > > > >>> On Tue, 2022-05-24 at 00:04 -0700, Wei Xu wrote: > > > >>>> On Thu, May 19, 2022 at 8:06 PM Ying Huang <ying.huang@xxxxxxxxx> wrote: > > > >>>>> > > > > > > ... > > > > > > > > > > > OK. Just to confirm. Does this mean that we will have fixed device ID, > > > > for example, > > > > > > > > GPU memtier255 > > > > DRAM (with CPU) memtier0 > > > > PMEM memtier1 > > > > > > > > When we add a new memtier, it can be memtier254, or memter2? The rank > > > > value will determine the real demotion order. > > > > > > > > I think you may need to send v3 to make sure everyone is at the same > > > > page. > > > > > > > > > > What we have implemented which we will send as RFC shortly is below. > > > > > > cd /sys/dekvaneesh@ubuntu-guest:~$ cd /sys/devices/system/ > > > kvaneesh@ubuntu-guest:/sys/devices/system$ pwd > > > /sys/devices/system > > > kvaneesh@ubuntu-guest:/sys/devices/system$ ls > > > clockevents clocksource container cpu edac memory memtier mpic > > > node power > > > kvaneesh@ubuntu-guest:/sys/devices/system$ cd memtier/ > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ pwd > > > /sys/devices/system/memtier > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ ls > > > default_rank max_rank memtier1 power uevent > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat default_rank > > > 1 > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat max_rank > > > 3 > > > > For flexibility, we don't want max_rank to be interpreted as the > > number of memory tiers. Also, we want to leave spaces in rank values > > to allow new memtiers to be inserted when needed. So I'd suggest to > > make max_rank a much larger value (e.g. 255). > > > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cd memtier1/ > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ ls > > > nodelist power rank subsystem uevent > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat nodelist > > > 0-3 > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat rank > > > 1 > > > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cd > > > ../../node/node1/ > > > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ cat memtier > > > 1 > > > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ > > > root@ubuntu-guest:/sys/devices/system/node/node1# echo 0 > memtier > > > root@ubuntu-guest:/sys/devices/system/node/node1# cat memtier > > > 0 > > > root@ubuntu-guest:/sys/devices/system/node/node1# cd ../../memtier/ > > > root@ubuntu-guest:/sys/devices/system/memtier# ls > > > default_rank max_rank memtier0 memtier1 power uevent > > > root@ubuntu-guest:/sys/devices/system/memtier# cd memtier0/ > > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat nodelist > > > 1 > > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat rank > > > 0 > > > > It looks like the example here demonstrates the dynamic creation of > > memtier0. If so, how is the rank of memtier0 determined? If we want > > to support creating new memtiers at runtime, I think an explicit > > interface that specifies both device ID and rank is preferred to avoid > > implicit dependencies between device IDs and ranks. > > Why make device ID explicit - it's meaningless I think? > How about a creation interface that is simply writing the rank value > to create a new one? The only race I can see would be to get > two parallel attempts to create a new tier with the same rank. > That seems unlikely to matter unless we support changing rank later. > > Two attempts to create the same device ID tier seems more likely to > cause fiddly races. That's right: Device ID is not needed when creating a new memtier. It should be enough to provide only a rank value. > Jonathan > > > > > > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# echo 4 > rank > > > bash: rank: Permission denied > > > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# > >
