Re: numa_alloc_onnode does not allocate on node passed as argument

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Elena,

Thank you very much for your quick reply! numa_set_strict(1) and numa_set_strict(0) both result in the wrong output. I did not change the default policy.

numa_get_membind returns 1 for all nodes before and after numa_run_on_node.
numa_get_interleave_mask returns 0 for all nodes.
numa_get_run_node_mask is all 1s before and 0010 after numa_run_on_node.

The machine config (the CPUs are all Intel(R) Xeon(R) CPU E5-4657L v2 @ 2.40GHz):

$ numactl --hardware
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 48 49 50 51 52 53 54 55 56 57 58 59
node 0 size: 262093 MB
node 0 free: 966 MB
node 1 cpus: 12 13 14 15 16 17 18 19 20 21 22 23 60 61 62 63 64 65 66 67 68 69 70 71
node 1 size: 262144 MB
node 1 free: 82 MB
node 2 cpus: 24 25 26 27 28 29 30 31 32 33 34 35 72 73 74 75 76 77 78 79 80 81 82 83
node 2 size: 262144 MB
node 2 free: 102 MB
node 3 cpus: 36 37 38 39 40 41 42 43 44 45 46 47 84 85 86 87 88 89 90 91 92 93 94 95
node 3 size: 262144 MB
node 3 free: 113 MB
node distances:
node   0   1   2   3 
  0:  10  20  30  20 
  1:  20  10  20  30 
  2:  30  20  10  20 
  3:  20  30  20  10 

Thanks again,
Andres

> On 22 Oct 2014, at 06:12, Elena Ufimtseva <ufimtseva@xxxxxxxxx> wrote:
> 
> On Tue, Oct 21, 2014 at 11:47 PM, Andres Nötzli <noetzli@xxxxxxxxxxxx> wrote:
>> Hi everyone,
>> 
>> I am experiencing a weird problem. When using numa_alloc_onnode repeatedly to allocate memory, it does not allocate memory on the node passed as an argument.
>> 
>> Sample code:
>> #include <numa.h>
>> #include <numaif.h>
>> #include <iostream>
>> using namespace std;
>> 
>> void find_memory_node_for_addr(void* ptr) {
>>   int numa_node = -1;
>>   if(get_mempolicy(&numa_node, NULL, 0, ptr, MPOL_F_NODE | MPOL_F_ADDR) < 0)
>>      cout << "WARNING: get_mempolicy failed" << endl;
>>   cout << numa_node << endl;
>> }
>> 
>> int main() {
>>   int64_t* x;
>>   int64_t n = 5000;
>>   //numa_set_preferred(1);
>> 
>>   numa_run_on_node(2);
>>   for(int i = 0; i < 20; i++) {
>>      size_t s = n * sizeof(int64_t);
>>      x = (int64_t*)numa_alloc_onnode(s, 1);
>>      for(int j = 0; j < n; j++)
>>         x[j] = j + i;
>>      find_memory_node_for_addr(x);
>>   }
>> 
>>   return 0;
>> }
>> 
>> Output:
>> 1
>> 1
>> 1
>> 2
>> 1
>> 2
>> 1
>> 2
>> 1
>> 2
>> 1
>> 2
>> 1
>> 2
>> 1
>> 2
>> 1
>> 2
>> 1
>> 2
>> 
>> When uncommenting the line "numa_set_preferred(1);”, the output is all 1s as expected. Am I doing something wrong? Have you seen similar issues?
>> 
>> I am running Ubuntu 12.04.5 LTS:
>> $ cat /proc/version
>> Linux version 3.2.0-29-generic (buildd@allspice) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012
>> 
>> I am using libnuma 2.0.10 but I’ve had the same problem with 2.0.8~rc3-1.
>> 
>> Thank you very much,
>> Andres
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-numa" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> Hi Andres
> 
> Can you try to use strict policy by calling numa_set_strict?
> 
> If you comment out setting the preferred node, the default policy is
> in action (I assume you did no change it, not for the process, not
> system wide) which is preferred also.
> But here you set preferred to a specific node and manual says, the
> default for process is to allocate on the node it runs.
> So I wonder what is the cpu affinity for this process looks like...
> Also maybe just to confirm you can check the policy from within your
> running code?
> 
> Can you also post the machine NUMA config?
> 
> -- 
> Elena


--
To unsubscribe from this list: send the line "unsubscribe linux-numa" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Kernel]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]     [Devices]

  Powered by Linux