For practical use-cases it is beneficial to prefetch the next freelist object in bulk allocation loop. Micro benchmarking show approx 1 cycle change: bulk - prev-patch - this patch 1 - 49 cycles(tsc) - 49 cycles(tsc) - increase in cycles:0 2 - 30 cycles(tsc) - 31 cycles(tsc) - increase in cycles:1 3 - 23 cycles(tsc) - 25 cycles(tsc) - increase in cycles:2 4 - 20 cycles(tsc) - 22 cycles(tsc) - increase in cycles:2 8 - 18 cycles(tsc) - 19 cycles(tsc) - increase in cycles:1 16 - 17 cycles(tsc) - 18 cycles(tsc) - increase in cycles:1 30 - 18 cycles(tsc) - 17 cycles(tsc) - increase in cycles:-1 32 - 18 cycles(tsc) - 19 cycles(tsc) - increase in cycles:1 34 - 23 cycles(tsc) - 24 cycles(tsc) - increase in cycles:1 48 - 21 cycles(tsc) - 22 cycles(tsc) - increase in cycles:1 64 - 20 cycles(tsc) - 21 cycles(tsc) - increase in cycles:1 128 - 27 cycles(tsc) - 27 cycles(tsc) - increase in cycles:0 158 - 30 cycles(tsc) - 30 cycles(tsc) - increase in cycles:0 250 - 37 cycles(tsc) - 37 cycles(tsc) - increase in cycles:0 Note, benchmark done with slab_nomerge to keep it stable enough for accurate comparison. Signed-off-by: Jesper Dangaard Brouer <brouer@xxxxxxxxxx> --- mm/slub.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/slub.c b/mm/slub.c index c25717ab3b5a..5af75a618b91 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2951,6 +2951,7 @@ bool kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size, goto error; c = this_cpu_ptr(s->cpu_slab); + prefetch_freepointer(s, c->freelist); continue; /* goto for-loop */ } @@ -2960,6 +2961,7 @@ bool kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size, goto error; c->freelist = get_freepointer(s, object); + prefetch_freepointer(s, c->freelist); p[i] = object; /* kmem_cache debug support */ -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>