On Wed, 10 Dec 2014 13:51:32 -0600 (CST) Christoph Lameter <cl@xxxxxxxxx> wrote: > On Wed, 10 Dec 2014, Jesper Dangaard Brouer wrote: > > > One of the building blocks for achieving this speedup is a cmpxchg > > based Lock-Free queue that supports bulking, named alf_queue for > > Array-based Lock-Free queue. By bulking elements (pointers) from the > > queue, the cost of the cmpxchg (approx 8 ns) is amortized over several > > elements. > > This is a bit of an issue since the design of the SLUB allocator is such > that you should pick up an object, apply some processing and then take the > next one. The fetching of an object warms up the first cacheline and this > is tied into the way free objects are linked in SLUB. > > So a bulk fetch from SLUB will not that effective and cause the touching > of many cachelines if we are dealing with just a few objects. If we are > looking at whole slab pages with all objects then SLUB can be effective > since we do not have to build up the linked pointer structure in each > page. SLAB has a different architecture there and a bulk fetch there is > possible without touching objects even for small sets since the freelist > management is separate from the objects. > > If you do this bulking then you will later access cache cold objects? > Doesnt that negate the benefit that you gain? Or are these objects written > to by hardware and therefore by necessity cache cold? Cache warmup is a concern, but perhaps it's the callers responsibility to prefetch for their use-case. For qmempool I do have patches that prefetch elems when going from the sharedq to the localq (per CPU), but I didn't see much gain, and I could prove my point (of being faster than slab) without it. And I would use/need the slab bulk interface to add elems to sharedq which I consider semi-cache cold. > We could provide a faster bulk alloc/free function. > > int kmem_cache_alloc_array(struct kmem_cache *s, gfp_t flags, > size_t objects, void **array) I like it :-) > and this could be optimized by each slab allocator to provide fast > population of objects in that array. We then assume that the number of > objects is in the hundreds or so right? I'm already seeing a benefit with 16 packets alloc/free "bulking". On RX we have a "budget" of 64 packets/descriptors (taken from the NIC RX ring) that need SKBs. On TX packets are put into the TX ring, and later at TX completion the TX ring is cleaned up, as many as 256 (as e.g. in the ixgbe driver). Scientific articles on userspace networking (like netmap) report that they need at least 8 packet bulking to see wirespeed 10G at 64 bytes. > The corresponding free function > > void kmem_cache_free_array(struct kmem_cache *s, > size_t objects, void **array) > > > I think the queue management of the array can be improved by using a > similar technique as used the SLUB allocator using the cmpxchg_local. > cmpxchg_local is much faster than a full cmpxchg and we are operating on > per cpu structures anyways. So the overhead could still be reduced. I think you missed that the per cpu localq is already not using cmpxchg (it is a SPSC queue). The sharedq (MPMC queue) does need and use the locked cmpxchg. -- Best regards, Jesper Dangaard Brouer MSc.CS, Sr. Network Kernel Developer at Red Hat Author of http://www.iptv-analyzer.org LinkedIn: http://www.linkedin.com/in/brouer -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>