On 6/14/18 1:37 PM, Jens Axboe wrote: > On 6/14/18 2:32 PM, Adam Manzanares wrote: >> >> >> On 6/14/18 9:09 AM, Hannes Reinecke wrote: >>> On Thu, 14 Jun 2018 09:33:35 -0600 >>> Jens Axboe <axboe@xxxxxxxxx> wrote: >>> >>>> On 6/14/18 9:29 AM, Hannes Reinecke wrote: >>>>> On Thu, 14 Jun 2018 08:47:33 -0600 >>>>> Jens Axboe <axboe@xxxxxxxxx> wrote: >>>>> >>>>>> On 6/14/18 7:38 AM, Hannes Reinecke wrote: >>>>>>> For performance reasons we should be able to allocate all memory >>>>>>> from a given NUMA node, so this patch adds a new parameter >>>>>>> 'rd_numa_node' to allow the user to specify the NUMA node id. >>>>>>> When restricing fio to use the same NUMA node I'm seeing a >>>>>>> performance boost of more than 200%. >>>>>> >>>>>> Looks fine to me. One comment. >>>>>> >>>>>>> @@ -342,6 +343,10 @@ static int max_part = 1; >>>>>>> module_param(max_part, int, 0444); >>>>>>> MODULE_PARM_DESC(max_part, "Num Minors to reserve between >>>>>>> devices"); >>>>>>> +static int rd_numa_node = NUMA_NO_NODE; >>>>>>> +module_param(rd_numa_node, int, 0444); >>>>>>> +MODULE_PARM_DESC(rd_numa_node, "NUMA node number to allocate RAM >>>>>>> disk on."); >>>>>> >>>>>> This could feasibly be 0644, as there would be nothing wrong with >>>>>> altering this at runtime. >>>>>> >>>>> >>>>> While we could it would not change the allocation of _existing_ ram >>>>> devices, making behaviour rather unpredictable. >>>>> Hence I did decide against it (and yes, I actually thought about >>>>> it). >>>>> >>>>> But if you insist ... >>>> >>>> Right, it would just change new allocations. Probably not a common use >>>> case, but there's really nothing that prevents it from being feasible. >>>> >>>> Next question - what does the memory allocator do if we run out of >>>> memory on the given node? Should we punt to a different node if that >>>> happens? Slower, but functional, seems preferable to not being able >>>> to get memory. >>>> >>> >>> Hmm. That I haven't considered; yes, that really sounds like an idea. >>> Will be sending an updated patch. >> >> Will numactl ... modprobe brd ... solve this problem? > > It won't, pages are allocated as needed. > Then how about a numactl ... dd /dev/ram ... after the modprobe.
