On Tue, 16 Jun 2015 14:48:40 -0700
Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> wrote:
> On Mon, 15 Jun 2015 17:52:07 +0200 Jesper Dangaard Brouer <brouer@xxxxxxxxxx> wrote:
>
> > From: Christoph Lameter <cl@xxxxxxxxx>
> >
> > [NOTICE: Already in AKPM's quilt-queue]
> >
> > First piece: acceleration of retrieval of per cpu objects
> >
> > If we are allocating lots of objects then it is advantageous to disable
> > interrupts and avoid the this_cpu_cmpxchg() operation to get these objects
> > faster.
> >
> > Note that we cannot do the fast operation if debugging is enabled, because
> > we would have to add extra code to do all the debugging checks. And it
> > would not be fast anyway.
> >
> > Note also that the requirement of having interrupts disabled
> > avoids having to do processor flag operations.
> >
> > Allocate as many objects as possible in the fast way and then fall back to
> > the generic implementation for the rest of the objects.
> >
> > ...
> >
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -2759,7 +2759,32 @@ EXPORT_SYMBOL(kmem_cache_free_bulk);
> > bool kmem_cache_alloc_bulk(struct kmem_cache *s, gfp_t flags, size_t size,
> > void **p)
> > {
> > - return kmem_cache_alloc_bulk(s, flags, size, p);
> > + if (!kmem_cache_debug(s)) {
> > + struct kmem_cache_cpu *c;
> > +
> > + /* Drain objects in the per cpu slab */
> > + local_irq_disable();
> > + c = this_cpu_ptr(s->cpu_slab);
> > +
> > + while (size) {
> > + void *object = c->freelist;
> > +
> > + if (!object)
> > + break;
> > +
> > + c->freelist = get_freepointer(s, object);
> > + *p++ = object;
> > + size--;
> > +
> > + if (unlikely(flags & __GFP_ZERO))
> > + memset(object, 0, s->object_size);
> > + }
> > + c->tid = next_tid(c->tid);
> > +
> > + local_irq_enable();
>
> It might be worth adding
>
> if (!size)
> return true;
>
> here. To avoid the pointless call to __kmem_cache_alloc_bulk().
The pointless call did present a measurable performance hit (2ns), and
I've removed it in the next patches, which fixes the error/exit path.
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>