On 9/5/22 10:43 AM, Huang, Ying wrote: > Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx> writes: > >> On 9/5/22 7:22 AM, Huang, Ying wrote: >>> Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx> writes: >>> >>>> On 9/2/22 2:34 PM, Huang, Ying wrote: >>>>> Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx> writes: >>>>> >>>>>> On 9/2/22 1:27 PM, Huang, Ying wrote: >>>>>>> Wei Xu <weixugc@xxxxxxxxxx> writes: >>>>>>> >>>>>>>> On Thu, Sep 1, 2022 at 11:44 PM Aneesh Kumar K V >>>>>>>> <aneesh.kumar@xxxxxxxxxxxxx> wrote: >>>>>>>>> >>>>>>>>> On 9/2/22 12:10 PM, Huang, Ying wrote: >>>>>>>>>> Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx> writes: >>>>>>>>>> >>>>>>>>>>> On 9/2/22 11:42 AM, Huang, Ying wrote: >>>>>>>>>>>> Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx> writes: >>>>>>>>>>>> >>>>>>>>>>>>> On 9/2/22 11:10 AM, Huang, Ying wrote: >>>>>>>>>>>>>> Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx> writes: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 9/2/22 10:39 AM, Wei Xu wrote: >>>>>>>>>>>>>>>> On Thu, Sep 1, 2022 at 5:33 PM Huang, Ying <ying.huang@xxxxxxxxx> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx> writes: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On 9/1/22 12:31 PM, Huang, Ying wrote: >>>>>>>>>>>>>>>>>>> "Aneesh Kumar K.V" <aneesh.kumar@xxxxxxxxxxxxx> writes: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> This patch adds /sys/devices/virtual/memory_tiering/ where all memory tier >>>>>>>>>>>>>>>>>>>> related details can be found. All allocated memory tiers will be listed >>>>>>>>>>>>>>>>>>>> there as /sys/devices/virtual/memory_tiering/memory_tierN/ >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The nodes which are part of a specific memory tier can be listed via >>>>>>>>>>>>>>>>>>>> /sys/devices/virtual/memory_tiering/memory_tierN/nodes >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I think "memory_tier" is a better subsystem/bus name than >>>>>>>>>>>>>>>>>>> memory_tiering. Because we have a set of memory_tierN devices inside. >>>>>>>>>>>>>>>>>>> "memory_tier" sounds more natural. I know this is subjective, just my >>>>>>>>>>>>>>>>>>> preference. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I missed replying to this earlier. I will keep memory_tiering as subsystem name in v4 >>>>>>>>>>>>>>> because we would want it to a susbsystem where all memory tiering related details can be found >>>>>>>>>>>>>>> including memory type in the future. This is as per discussion >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://lore.kernel.org/linux-mm/CAAPL-u9TKbHGztAF=r-io3gkX7gorUunS2UfstudCWuihrA=0g@xxxxxxxxxxxxxx >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't think that it's a good idea to mix 2 types of devices in one >>>>>>>>>>>>>> subsystem (bus). If my understanding were correct, that breaks the >>>>>>>>>>>>>> driver core convention. >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> All these are virtual devices .I am not sure i follow what you mean by 2 types of devices. >>>>>>>>>>>>> memory_tiering is a subsystem that represents all the details w.r.t memory tiering. It shows >>>>>>>>>>>>> details of memory tiers and can possibly contain details of different memory types . >>>>>>>>>>>> >>>>>>>>>>>> IMHO, memory_tier and memory_type are 2 kind of devices. They have >>>>>>>>>>>> almost totally different attributes (sysfs file). So, we should create >>>>>>>>>>>> 2 buses for them. Each has its own attribute group. "virtual" itself >>>>>>>>>>>> isn't a subsystem. >>>>>>>>>>> >>>>>>>>>>> Considering both the details are related to memory tiering, wouldn't it be much simpler we consolidate >>>>>>>>>>> them within the same subdirectory? I am still not clear why you are suggesting they need to be in different >>>>>>>>>>> sysfs hierarchy. It doesn't break any driver core convention as you mentioned earlier. >>>>>>>>>>> >>>>>>>>>>> /sys/devices/virtual/memory_tiering/memory_tierN >>>>>>>>>>> /sys/devices/virtual/memory_tiering/memory_typeN >>>>>>>>>> >>>>>>>>>> I think we should add >>>>>>>>>> >>>>>>>>>> /sys/devices/virtual/memory_tier/memory_tierN >>>>>>>>>> /sys/devices/virtual/memory_type/memory_typeN >>>>>>>>>> >>>>>>>>> >>>>>>>>> I am trying to find if there is a technical reason to do the same? >>>>>>>>> >>>>>>>>>> I don't think this is complex. Devices of same bus/subsystem should >>>>>>>>>> have mostly same attributes. This is my understanding of driver core >>>>>>>>>> convention. >>>>>>>>>> >>>>>>>>> >>>>>>>>> I was not looking at this from code complexity point. Instead of having multiple directories >>>>>>>>> with details w.r.t memory tiering, I was looking at consolidating the details >>>>>>>>> within the directory /sys/devices/virtual/memory_tiering. (similar to all virtual devices >>>>>>>>> are consolidated within /sys/devics/virtual/). >>>>>>>>> >>>>>>>>> -aneesh >>>>>>>> >>>>>>>> Here is an example of /sys/bus/nd/devices (I know it is not under >>>>>>>> /sys/devices/virtual, but it can still serve as a reference): >>>>>>>> >>>>>>>> ls -1 /sys/bus/nd/devices >>>>>>>> >>>>>>>> namespace2.0 >>>>>>>> namespace3.0 >>>>>>>> ndbus0 >>>>>>>> nmem0 >>>>>>>> nmem1 >>>>>>>> region0 >>>>>>>> region1 >>>>>>>> region2 >>>>>>>> region3 >>>>>>>> >>>>>>>> So I think it is not unreasonable if we want to group memory tiering >>>>>>>> related interfaces within a single top directory. >>>>>>> >>>>>>> Thanks for pointing this out. My original understanding of driver core >>>>>>> isn't correct. >>>>>>> >>>>>>> But I still think it's better to separate instead of mixing memory_tier >>>>>>> and memory_type. Per my understanding, memory_type shows information >>>>>>> (abstract distance, latency, bandwidth, etc.) of memory types (and >>>>>>> nodes), it can be useful even without memory tiers. That is, memory >>>>>>> types describes the physical characteristics, while memory tier reflects >>>>>>> the policy. >>>>>>> >>>>>> >>>>>> The latency and bandwidth details are already exposed via >>>>>> >>>>>> /sys/devices/system/node/nodeY/access0/initiators/ >>>>>> >>>>>> Documentation/admin-guide/mm/numaperf.rst >>>>>> >>>>>> That is the interface that libraries like libmemkind will look at for finding >>>>>> details w.r.t latency/bandwidth >>>>> >>>>> Yes. Only with that, it's still inconvenient to find out which nodes >>>>> belong to same memory type (has same performance, same topology, managed >>>>> by same driver, etc). So memory types can still provide useful >>>>> information even without memory tiering. >>>>> >>>> >>>> I am not sure i quiet follow what to conclude from your reply. I used the subsystem name >>>> "memory_tiering" so that all memory tiering related information can be consolidated there. >>>> I guess you agreed to the above part that we can consolidated things like that. >>> >>> I just prefer to separate memory_tier and memory_type sysfs directories >>> personally. Because memory_type describes the physical memory types and >>> performance, while memory_tier is more about the policy to group >>> memory_types. >>> >> IMHO we can decide on that based on why we end up adding memory_type details to sysfs. If that >> is only for memory tier modification from userspace we can look at adding that in the memory tiering >> sysfs hierarchy. >> >> Also since we have precedence of consolidating things within a sysfs hierarchy as explained in previous emails, >> I think we should keep "memory_tiering" as sysfs subsystem name? I hope we can get an agreement on that >> for now? > > I prefer to separate memory_tier and memory_type, so the subsystem name > should be "memory_tier". You prefer to consolidate memory_tier and > memory_type, so the subsystem name should be "memory_tiering". > > The main reason behind my idea is that memory_type isn't tied with > memory tiering directly. It describes some hardware property. Even if > we don't use memory tiering, we can still use that to classify the > memory devices in the system. > > Why do you want to consolidate them? To reduce one directory from > sysfs? > So that it is much intuitive for user to got to memory_tiering sysfs hierarchy to change the memory tier levels. As I mentioned earlier the reason for consolidating things is to accommodate the possibility of supporting changing abstract distance of a memory type so that we can change the memory tier assignment of that specific memory type. I don't see any other reason we would want to expose memory type to userspace as of now. > I want to get opinions from other people too. > > Best Regards, > Huang, Ying -aneesh
