On 4/25/22 7:27 PM, Jonathan Cameron wrote:
On Mon, 25 Apr 2022 16:45:38 +0530
Jagdish Gediya <jvgediya@xxxxxxxxxxxxx> wrote:
....
$ numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7
node 0 size: 14272 MB
node 0 free: 13392 MB
node 1 cpus:
node 1 size: 2028 MB
node 1 free: 1971 MB
node distances:
node 0 1
0: 10 40
1: 40 10
1) without N_DEMOTION_TARGETS patch series, 1 is demotion target
for 0 even when 1 is DRAM node and there is no demotion targets for 1.
I'm not convinced the distinction between DRAM and persistent memory is
valid. There will definitely be systems with a large pool
of remote DRAM (and potentially no NV memory) where the right choice
is to demote to that DRAM pool.
Basing the decision on whether the memory is from kmem or
normal DRAM doesn't provide sufficient information to make the decision.
$ cat /sys/bus/nd/devices/dax0.0/target_node
2
$
# cd /sys/bus/dax/drivers/
:/sys/bus/dax/drivers# ls
device_dax kmem
:/sys/bus/dax/drivers# cd device_dax/
:/sys/bus/dax/drivers/device_dax# echo dax0.0 > unbind
:/sys/bus/dax/drivers/device_dax# echo dax0.0 > ../kmem/new_id
:/sys/bus/dax/drivers/device_dax# numactl -H
available: 3 nodes (0-2)
node 0 cpus: 0 1 2 3 4 5 6 7
node 0 size: 14272 MB
node 0 free: 13380 MB
node 1 cpus:
node 1 size: 2028 MB
node 1 free: 1961 MB
node 2 cpus:
node 2 size: 0 MB
node 2 free: 0 MB
node distances:
node 0 1 2
0: 10 40 80
1: 40 10 80
2: 80 80 10
2) Once this new node brought online, without N_DEMOTION_TARGETS
patch series, 1 is demotion target for 0 and 2 is demotion target
for 1.
With this patch series applied,
1) No demotion target for either 0 or 1 before dax device is online
I'd argue that is wrong. At this state you have a tiered memory system
be it one with just DRAM. Using it as such is correct behavior that
we should not be preventing. Sure some usecases wouldn't want that
arrangement but some do want it.
I missed this in my earlier reply. Are you suggesting that we would want
Node 1 (DRAM only memory numa node) to act as demotion target for Node
0? Any reason why we would want to do that? That is clearly opposite of
what we are trying to do here. IMHO node using Node1 as demotion target
for Node0 is a better default?
-aneesh
