Re: [PATCH v6 04/13] mm/demotion/dax/kmem: Set node's memory tier to MEMORY_TIER_PMEM

[Aneesh Kumar K V <aneesh.kumar@xxxxxxxxxxxxx>]

On 6/13/22 12:29 PM, Ying Huang wrote:
> On Fri, 2022-06-10 at 19:22 +0530, Aneesh Kumar K.V wrote:
> > By default, all nodes are assigned to DEFAULT_MEMORY_TIER which
> > is the memory tier designated for nodes with DRAM
> >
> > Set dax kmem device node's tier to MEMORY_TIER_PMEM. MEMORY_TIER_PMEM
> > is assigned a default rank value of 100 and appears below DEFAULT_MEMORY_TIER
> > in demotion order.
> >
> > Signed-off-by: Jagdish Gediya <jvgediya@xxxxxxxxxxxxx>
> > Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@xxxxxxxxxxxxx>
> > ---
> >   drivers/dax/kmem.c           |  4 ++
> >   include/linux/memory-tiers.h |  1 +
> >   mm/memory-tiers.c            | 78 ++++++++++++++++++++++++++++++++++++
> >   3 files changed, 83 insertions(+)
> >
> > diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
> > index a37622060fff..0cb3de3d138f 100644
> > --- a/drivers/dax/kmem.c
> > +++ b/drivers/dax/kmem.c
> > @@ -11,6 +11,7 @@
> >   #include <linux/fs.h>
> >   #include <linux/mm.h>
> >   #include <linux/mman.h>
> > +#include <linux/memory-tiers.h>
> >   #include "dax-private.h"
> >   #include "bus.h"
> >   
> >
> > @@ -147,6 +148,9 @@ static int dev_dax_kmem_probe(struct dev_dax *dev_dax)
> >   
> >
> >   	dev_set_drvdata(dev, data);
> >   
> >
> > +#ifdef CONFIG_TIERED_MEMORY
> > +	node_create_and_set_memory_tier(numa_node, MEMORY_TIER_PMEM);
> > +#endif
> >   	return 0;
> >   
> >
> >   err_request_mem:
> > diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h
> > index 44c3c3b16a36..e102ec73ab80 100644
> > --- a/include/linux/memory-tiers.h
> > +++ b/include/linux/memory-tiers.h
> > @@ -18,6 +18,7 @@
> >   #define MAX_MEMORY_TIERS  3
> >   
> >
> >   extern bool numa_demotion_enabled;
> > +int node_create_and_set_memory_tier(int node, int tier);
> >   #else
> >   #define numa_demotion_enabled	false
> >   
> >
> > diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c
> > index c3123a457d90..00d393a5a628 100644
> > --- a/mm/memory-tiers.c
> > +++ b/mm/memory-tiers.c
> > @@ -67,6 +67,84 @@ static struct memory_tier *register_memory_tier(unsigned int tier,
> >   	return memtier;
> >   }
> >   
> >
> > +static struct memory_tier *__node_get_memory_tier(int node)
> > +{
> > +	struct memory_tier *memtier;
> > +
> > +	list_for_each_entry(memtier, &memory_tiers, list) {
> > +		if (node_isset(node, memtier->nodelist))
> > +			return memtier;
> > +	}
> > +	return NULL;
> > +}
> > +
>
> I suggest to add NODE_DATA(nid)->mem_tier before this patch.  That is,
> part of [9/13].  That will make code much simpler and easier to
> review.

I would still like to keep introducing that NODE_DATA()->memtier later, 
so that the everything before that can be reviewed with simpler locking. 
__node_get_memory_tier() get fixed in patch 9 where all the details of 
that lockless access is documented. This also enables one to look at the 
history later.

> And, in addition to dax_kmem, whenever a normal node is onlined, we
> need to add it to the default memory tier.  I found this is done in
> [5/13].  IMHO, we should move that part before this patch.

Sure will move the hotplug callback earlier.


-aneesh