On 6/14/18 9:09 AM, Hannes Reinecke wrote: > On Thu, 14 Jun 2018 09:33:35 -0600 > Jens Axboe <axboe@xxxxxxxxx> wrote: > >> On 6/14/18 9:29 AM, Hannes Reinecke wrote: >>> On Thu, 14 Jun 2018 08:47:33 -0600 >>> Jens Axboe <axboe@xxxxxxxxx> wrote: >>> >>>> On 6/14/18 7:38 AM, Hannes Reinecke wrote: >>>>> For performance reasons we should be able to allocate all memory >>>>> from a given NUMA node, so this patch adds a new parameter >>>>> 'rd_numa_node' to allow the user to specify the NUMA node id. >>>>> When restricing fio to use the same NUMA node I'm seeing a >>>>> performance boost of more than 200%. >>>> >>>> Looks fine to me. One comment. >>>> >>>>> @@ -342,6 +343,10 @@ static int max_part = 1; >>>>> module_param(max_part, int, 0444); >>>>> MODULE_PARM_DESC(max_part, "Num Minors to reserve between >>>>> devices"); >>>>> +static int rd_numa_node = NUMA_NO_NODE; >>>>> +module_param(rd_numa_node, int, 0444); >>>>> +MODULE_PARM_DESC(rd_numa_node, "NUMA node number to allocate RAM >>>>> disk on."); >>>> >>>> This could feasibly be 0644, as there would be nothing wrong with >>>> altering this at runtime. >>>> >>> >>> While we could it would not change the allocation of _existing_ ram >>> devices, making behaviour rather unpredictable. >>> Hence I did decide against it (and yes, I actually thought about >>> it). >>> >>> But if you insist ... >> >> Right, it would just change new allocations. Probably not a common use >> case, but there's really nothing that prevents it from being feasible. >> >> Next question - what does the memory allocator do if we run out of >> memory on the given node? Should we punt to a different node if that >> happens? Slower, but functional, seems preferable to not being able >> to get memory. >> > > Hmm. That I haven't considered; yes, that really sounds like an idea. > Will be sending an updated patch. Will numactl ... modprobe brd ... solve this problem? > > Cheers, > > Hannes >
