On Thu, 03 Dec 2015 16:57:31 +0100 Jesper Dangaard Brouer <brouer@xxxxxxxxxx> wrote:
> diff --git a/mm/slab.c b/mm/slab.c
> index 4765c97ce690..3354489547ec 100644
> --- a/mm/slab.c
> +++ b/mm/slab.c
> @@ -3420,9 +3420,59 @@ void kmem_cache_free_bulk(struct kmem_cache *s, size_t size, void **p)
> EXPORT_SYMBOL(kmem_cache_free_bulk);
>
> int kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size,
> - void **p)
> + void **p)
> {
> - return __kmem_cache_alloc_bulk(s, flags, size, p);
> + size_t i;
> +
> + flags &= gfp_allowed_mask;
> + lockdep_trace_alloc(flags);
> +
> + if (slab_should_failslab(s, flags))
> + return 0;
> +
> + s = memcg_kmem_get_cache(s, flags);
> +
> + cache_alloc_debugcheck_before(s, flags);
> +
> + local_irq_disable();
> + for (i = 0; i < size; i++) {
> + void *objp = __do_cache_alloc(s, flags);
> +
> + // this call could be done outside IRQ disabled section
> + objp = cache_alloc_debugcheck_after(s, flags, objp, _RET_IP_);
Profiling with SLAB mem debugging on (CONFIG_DEBUG_SLAB=y), this call
cache_alloc_debugcheck_after() is the most expensive call, well
actually the underlying check_poison_obj() call.
Thus, it might be a good idea to, place it outside the IRQ disabled section?
It might make the code look a little strange, but I can try and we can
see how ugly that makes the code look (and the compiler still have to
be able to remove the code in-case no debugging enabled).
> +
> + if (unlikely(!objp))
> + goto error;
> +
> + prefetchw(objp);
> + p[i] = objp;
> + }
> + local_irq_enable();
> +
> + /* Kmemleak and kmemcheck outside IRQ disabled section */
> + for (i = 0; i < size; i++) {
> + void *x = p[i];
> +
> + kmemleak_alloc_recursive(x, s->object_size, 1, s->flags, flags);
> + kmemcheck_slab_alloc(s, flags, x, s->object_size);
> + }
> +
> + /* Clear memory outside IRQ disabled section */
> + if (unlikely(flags & __GFP_ZERO))
> + for (i = 0; i < size; i++)
> + memset(p[i], 0, s->object_size);
[...]
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>