* Davidlohr Bueso <davidlohr@xxxxxx> wrote: > On Mon, 2013-11-11 at 12:47 -0800, Davidlohr Bueso wrote: > > On Mon, 2013-11-11 at 13:04 +0100, Ingo Molnar wrote: > > > * Michel Lespinasse <walken@xxxxxxxxxx> wrote: > > > > > > > On Sun, Nov 10, 2013 at 8:12 PM, Davidlohr Bueso <davidlohr@xxxxxx> wrote: > > > > > 2) Oracle Data mining (4K pages) > > > > > +------------------------+----------+------------------+---------+ > > > > > | mmap_cache type | hit-rate | cycles (billion) | stddev | > > > > > +------------------------+----------+------------------+---------+ > > > > > | no mmap_cache | - | 63.35 | 0.20207 | > > > > > | current mmap_cache | 65.66% | 19.55 | 0.35019 | > > > > > | mmap_cache+largest VMA | 71.53% | 15.84 | 0.26764 | > > > > > | 4 element hash table | 70.75% | 15.90 | 0.25586 | > > > > > | per-thread mmap_cache | 86.42% | 11.57 | 0.29462 | > > > > > +------------------------+----------+------------------+---------+ > > > > > > > > > > This workload sure makes the point of how much we can benefit of > > > > > caching the vma, otherwise find_vma() can cost more than 220% extra > > > > > cycles. We clearly win here by having a per-thread cache instead of > > > > > per address space. I also tried the same workload with 2Mb hugepages > > > > > and the results are much more closer to the kernel build, but with the > > > > > per-thread vma still winning over the rest of the alternatives. > > > > > > > > > > All in all I think that we should probably have a per-thread vma > > > > > cache. Please let me know if there is some other workload you'd like > > > > > me to try out. If folks agree then I can cleanup the patch and send it > > > > > out. > > > > > > > > Per thread cache sounds interesting - with per-mm caches there is a real > > > > risk that some modern threaded apps pay the cost of cache updates > > > > without seeing much of the benefit. However, how do you cheaply handle > > > > invalidations for the per thread cache ? > > > > > > The cheapest way to handle that would be to have a generation counter for > > > the mm and to couple cache validity to a specific value of that. > > > 'Invalidation' is then the free side effect of bumping the generation > > > counter when a vma is removed/moved. > > Wouldn't this approach make us invalidate all vmas even when we > just want to do it for one? [...] Yes. If it's implemented as some sort of small, vma-size-weighted LRU, then all these 'different' caches go away and there's just this single LRU cache with a handful of entries cached. This cache is then invalidated on munmap() et al. Which should be fine, mmap()/munmap() is a slowpath relative to find_vma(). Thanks, Ingo -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>