On Thu, 14 Jun 2018 09:33:35 -0600 Jens Axboe <axboe@xxxxxxxxx> wrote: > On 6/14/18 9:29 AM, Hannes Reinecke wrote: > > On Thu, 14 Jun 2018 08:47:33 -0600 > > Jens Axboe <axboe@xxxxxxxxx> wrote: > > > >> On 6/14/18 7:38 AM, Hannes Reinecke wrote: > >>> For performance reasons we should be able to allocate all memory > >>> from a given NUMA node, so this patch adds a new parameter > >>> 'rd_numa_node' to allow the user to specify the NUMA node id. > >>> When restricing fio to use the same NUMA node I'm seeing a > >>> performance boost of more than 200%. > >> > >> Looks fine to me. One comment. > >> > >>> @@ -342,6 +343,10 @@ static int max_part = 1; > >>> module_param(max_part, int, 0444); > >>> MODULE_PARM_DESC(max_part, "Num Minors to reserve between > >>> devices"); > >>> +static int rd_numa_node = NUMA_NO_NODE; > >>> +module_param(rd_numa_node, int, 0444); > >>> +MODULE_PARM_DESC(rd_numa_node, "NUMA node number to allocate RAM > >>> disk on."); > >> > >> This could feasibly be 0644, as there would be nothing wrong with > >> altering this at runtime. > >> > > > > While we could it would not change the allocation of _existing_ ram > > devices, making behaviour rather unpredictable. > > Hence I did decide against it (and yes, I actually thought about > > it). > > > > But if you insist ... > > Right, it would just change new allocations. Probably not a common use > case, but there's really nothing that prevents it from being feasible. > > Next question - what does the memory allocator do if we run out of > memory on the given node? Should we punt to a different node if that > happens? Slower, but functional, seems preferable to not being able > to get memory. > Hmm. That I haven't considered; yes, that really sounds like an idea. Will be sending an updated patch. Cheers, Hannes
