On Thu, Jun 14, 2018 at 02:47:39PM -0600, Jens Axboe wrote:
> >>> Will numactl ... modprobe brd ... solve this problem?
> >>
> >> It won't, pages are allocated as needed.
> >>
> >
> > Then how about a numactl ... dd /dev/ram ... after the modprobe.
>
> Yes of course, or you could do that for every application that ends
> up in the path of the doing IO to it. The point of the option is to
> just make it explicit, and not have to either NUMA pin each task,
> or prefill all possible pages.
>
It's certainly possible from userspace using dd and numactl setting the
desired memory policy. mmtests has the following snippet when setting
up a benchmark using brd to deal with both NUMA artifacts and variable
performance due to first faults early in the lifetime of a benchmark.
modprobe brd rd_size=$((TESTDISK_RD_SIZE/1024))
if [ "$TESTDISK_RD_PREALLOC" == "yes" ]; then
if [ "$TESTDISK_RD_PREALLOC_NODE" != "" ]; then
tmp_prealloc_cmd="numactl -N $TESTDISK_RD_PREALLOC_NODE"
else
tmp_prealloc_cmd="numactl -i all"
fi
$tmp_prealloc_cmd dd if=/dev/zero of=/dev/ram0 bs=1M &>/dev/null
fi
(Haven't actually validated this in a long time but it worked at some point)
First option allocates just from one node, the other interleaves between
everything. Any combination of nodes or policies can be used and this was
very simple, but it's what was needed at the time. The question is how
far do you want to go with supporting policies within the module?
One option would be to keep this very simple like the patch suggests so users
get the hint that it's even worth considering and then point at a document
on how to do more complex policies from userspace at device creation time.
Another is simply to document the hazard that the locality of memory is
controlled by the memory policy of the first task that touches it.
--
Mel Gorman
SUSE Labs
