On 6/14/18 2:32 PM, Adam Manzanares wrote: > > > On 6/14/18 9:09 AM, Hannes Reinecke wrote: >> On Thu, 14 Jun 2018 09:33:35 -0600 >> Jens Axboe <axboe@xxxxxxxxx> wrote: >> >>> On 6/14/18 9:29 AM, Hannes Reinecke wrote: >>>> On Thu, 14 Jun 2018 08:47:33 -0600 >>>> Jens Axboe <axboe@xxxxxxxxx> wrote: >>>> >>>>> On 6/14/18 7:38 AM, Hannes Reinecke wrote: >>>>>> For performance reasons we should be able to allocate all memory >>>>>> from a given NUMA node, so this patch adds a new parameter >>>>>> 'rd_numa_node' to allow the user to specify the NUMA node id. >>>>>> When restricing fio to use the same NUMA node I'm seeing a >>>>>> performance boost of more than 200%. >>>>> >>>>> Looks fine to me. One comment. >>>>> >>>>>> @@ -342,6 +343,10 @@ static int max_part = 1; >>>>>> module_param(max_part, int, 0444); >>>>>> MODULE_PARM_DESC(max_part, "Num Minors to reserve between >>>>>> devices"); >>>>>> +static int rd_numa_node = NUMA_NO_NODE; >>>>>> +module_param(rd_numa_node, int, 0444); >>>>>> +MODULE_PARM_DESC(rd_numa_node, "NUMA node number to allocate RAM >>>>>> disk on."); >>>>> >>>>> This could feasibly be 0644, as there would be nothing wrong with >>>>> altering this at runtime. >>>>> >>>> >>>> While we could it would not change the allocation of _existing_ ram >>>> devices, making behaviour rather unpredictable. >>>> Hence I did decide against it (and yes, I actually thought about >>>> it). >>>> >>>> But if you insist ... >>> >>> Right, it would just change new allocations. Probably not a common use >>> case, but there's really nothing that prevents it from being feasible. >>> >>> Next question - what does the memory allocator do if we run out of >>> memory on the given node? Should we punt to a different node if that >>> happens? Slower, but functional, seems preferable to not being able >>> to get memory. >>> >> >> Hmm. That I haven't considered; yes, that really sounds like an idea. >> Will be sending an updated patch. > > Will numactl ... modprobe brd ... solve this problem? It won't, pages are allocated as needed. -- Jens Axboe
