On Mon, 14 Aug 2017 13:31:30 +0800 Aaron Lu <aaron.lu@xxxxxxxxx> wrote:
> If the system has more than one swap device and swap device has the node
> information, we can make use of this information to decide which swap
> device to use in get_swap_pages() to get better performance.
>
> The current code uses a priority based list, swap_avail_list, to decide
> which swap device to use and if multiple swap devices share the same
> priority, they are used round robin. This patch changes the previous
> single global swap_avail_list into a per-numa-node list, i.e. for each
> numa node, it sees its own priority based list of available swap devices.
> Swap device's priority can be promoted on its matching node's swap_avail_list.
>
> The current swap device's priority is set as: user can set a >=0 value,
> or the system will pick one starting from -1 then downwards. The priority
> value in the swap_avail_list is the negated value of the swap device's
> due to plist being sorted from low to high. The new policy doesn't change
> the semantics for priority >=0 cases, the previous starting from -1 then
> downwards now becomes starting from -2 then downwards and -1 is reserved
> as the promoted value.
>
> ...
>
> On a 2 node Skylake EP machine with 64GiB memory, two 170GB SSD drives
> are used as swap devices with each attached to a different node, the
> result is:
>
> runtime=30m/processes=32/total test size=128G/each process mmap region=4G
> kernel throughput
> vanilla 13306
> auto-binding 15169 +14%
>
> runtime=30m/processes=64/total test size=128G/each process mmap region=2G
> kernel throughput
> vanilla 11885
> auto-binding 14879 25%
>
Sounds nice.
> ...
>
> --- /dev/null
> +++ b/Documentation/vm/swap_numa.txt
> @@ -0,0 +1,18 @@
> +If the system has more than one swap device and swap device has the node
> +information, we can make use of this information to decide which swap
> +device to use in get_swap_pages() to get better performance.
> +
> +The current code uses a priority based list, swap_avail_list, to decide
> +which swap device to use and if multiple swap devices share the same
> +priority, they are used round robin. This change here replaces the single
> +global swap_avail_list with a per-numa-node list, i.e. for each numa node,
> +it sees its own priority based list of available swap devices. Swap
> +device's priority can be promoted on its matching node's swap_avail_list.
> +
> +The current swap device's priority is set as: user can set a >=0 value,
> +or the system will pick one starting from -1 then downwards. The priority
> +value in the swap_avail_list is the negated value of the swap device's
> +due to plist being sorted from low to high. The new policy doesn't change
> +the semantics for priority >=0 cases, the previous starting from -1 then
> +downwards now becomes starting from -2 then downwards and -1 is reserved
> +as the promoted value.
Could we please add a little "user guide" here? Tell people how to set
up their system to exploit this? Sample /etc/fstab entries, perhaps?
>
> ...
>
> +static int __init swapfile_init(void)
> +{
> + int nid;
> +
> + swap_avail_heads = kmalloc(nr_node_ids * sizeof(struct plist_head), GFP_KERNEL);
> + if (!swap_avail_heads)
> + return -ENOMEM;
Well, a kmalloc failure at __init time is generally considered "can't
happen", but if it _does_ happen, the system will later oops, I think.
Can we do something nicer here?
> + for_each_node(nid)
> + plist_head_init(&swap_avail_heads[nid]);
> +
> + return 0;
> +}
> +subsys_initcall(swapfile_init);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>
