On Fri, 21 Jul 2023, Peter Zijlstra wrote:
Extend the futex2 interface to be numa aware.
Sorry to be chiming in that late but it seems that this is useful to
mitigate NUMA issues also for our platform.
When FUTEX2_NUMA is not set, the node is simply an extention of the
hash, such that traditional futexes are still interleaved over the
nodes.
Could we follow NUMA policies like with other metadata allocations during
systen call processing? If there is no NUMA task policy then the futex
should be placed on the local NUMA node.
That way the placement of the futex can be controlled by the tasks memory
policy. We could skip the FUTEX2_NUMA option.
@@ -114,10 +137,29 @@ late_initcall(fail_futex_debugfs);
*/
struct futex_hash_bucket *futex_hash(union futex_key *key)
{
- u32 hash = jhash2((u32 *)key, offsetof(typeof(*key), both.offset) / 4,
+ u32 hash = jhash2((u32 *)key,
+ offsetof(typeof(*key), both.offset) / sizeof(u32),
key->both.offset);
+ int node = key->both.node;
- return &futex_queues[hash & (futex_hashsize - 1)];
+ if (node == -1) {
+ /*
+ * In case of !FLAGS_NUMA, use some unused hash bits to pick a
+ * node -- this ensures regular futexes are interleaved across
+ * the nodes and avoids having to allocate multiple
+ * hash-tables.
+ *
+ * NOTE: this isn't perfectly uniform, but it is fast and
+ * handles sparse node masks.
+ */
+ node = (hash >> futex_hashshift) % nr_node_ids;
+ if (!node_possible(node)) {
+ node = find_next_bit_wrap(node_possible_map.bits,
+ nr_node_ids, node);
+ }
Use memory allocation policies here instead?
