On Mon, 20 Jul 2015 11:54:15 +0900
Joonsoo Kim <iamjoonsoo.kim@xxxxxxx> wrote:
> On Thu, Jul 16, 2015 at 11:57:56AM +0200, Jesper Dangaard Brouer wrote:
> >
> > On Wed, 15 Jul 2015 18:02:39 +0200 Jesper Dangaard Brouer <brouer@xxxxxxxxxx> wrote:
> >
> > > Results:
> > [...]
> > > bulk- Fallback - Bulk API
> > > 1 - 64 cycles(tsc) 16.144 ns - 47 cycles(tsc) 11.931 - improved 26.6%
> > > 2 - 57 cycles(tsc) 14.397 ns - 29 cycles(tsc) 7.368 - improved 49.1%
> > > 3 - 55 cycles(tsc) 13.797 ns - 24 cycles(tsc) 6.003 - improved 56.4%
> > > 4 - 53 cycles(tsc) 13.500 ns - 22 cycles(tsc) 5.543 - improved 58.5%
> > > 8 - 52 cycles(tsc) 13.008 ns - 20 cycles(tsc) 5.047 - improved 61.5%
> > > 16 - 51 cycles(tsc) 12.763 ns - 20 cycles(tsc) 5.015 - improved 60.8%
> > > 30 - 50 cycles(tsc) 12.743 ns - 20 cycles(tsc) 5.062 - improved 60.0%
> > > 32 - 51 cycles(tsc) 12.908 ns - 20 cycles(tsc) 5.089 - improved 60.8%
> > > 34 - 87 cycles(tsc) 21.936 ns - 28 cycles(tsc) 7.006 - improved 67.8%
> > > 48 - 79 cycles(tsc) 19.840 ns - 31 cycles(tsc) 7.755 - improved 60.8%
> > > 64 - 86 cycles(tsc) 21.669 ns - 68 cycles(tsc) 17.203 - improved 20.9%
> > > 128 - 101 cycles(tsc) 25.340 ns - 72 cycles(tsc) 18.195 - improved 28.7%
> > > 158 - 112 cycles(tsc) 28.152 ns - 73 cycles(tsc) 18.372 - improved 34.8%
> > > 250 - 110 cycles(tsc) 27.727 ns - 73 cycles(tsc) 18.430 - improved 33.6%
> >
> >
> > Something interesting happens, when I'm tuning the SLAB/slub cache...
> >
> > I was thinking what happens if I "give" the slub more per CPU partial
> > pages. In my benchmark 250 is my "max" bulk working set.
> >
> > Tuning SLAB/slub for 256 bytes object size, by tuning SLUB saying each
> > CPU partial should be allowed to contain 256 objects (cpu_partial).
> >
> > sudo sh -c 'echo 256 > /sys/kernel/slab/:t-0000256/cpu_partial'
> >
> > And adjusting 'min_partial' affects __slab_free() by avoiding removing
> > partial if node->nr_partial >= s->min_partial. Thus, in our test
> > min_partial=9 result in keeping 9 pages 32 * 9 = 288 objects in the
> >
> > sudo sh -c 'echo 9 > /sys/kernel/slab/:t-0000256/min_partial'
> > sudo grep -H . /sys/kernel/slab/:t-0000256/*
> >
> > First notice the normal fastpath is: 47 cycles(tsc) 11.894 ns
> >
> > Patch03-TUNED-run01:
> > bulk- Fallback - Bulk-API
> > 1 - 63 cycles(tsc) 15.866 ns - 46 cycles(tsc) 11.653 ns - improved 27.0%
> > 2 - 56 cycles(tsc) 14.137 ns - 28 cycles(tsc) 7.106 ns - improved 50.0%
> > 3 - 54 cycles(tsc) 13.623 ns - 23 cycles(tsc) 5.845 ns - improved 57.4%
> > 4 - 53 cycles(tsc) 13.345 ns - 21 cycles(tsc) 5.316 ns - improved 60.4%
> > 8 - 51 cycles(tsc) 12.960 ns - 20 cycles(tsc) 5.187 ns - improved 60.8%
> > 16 - 50 cycles(tsc) 12.743 ns - 20 cycles(tsc) 5.091 ns - improved 60.0%
> > 30 - 80 cycles(tsc) 20.153 ns - 28 cycles(tsc) 7.054 ns - improved 65.0%
> > 32 - 82 cycles(tsc) 20.621 ns - 33 cycles(tsc) 8.392 ns - improved 59.8%
> > 34 - 80 cycles(tsc) 20.125 ns - 32 cycles(tsc) 8.046 ns - improved 60.0%
> > 48 - 91 cycles(tsc) 22.887 ns - 30 cycles(tsc) 7.655 ns - improved 67.0%
> > 64 - 85 cycles(tsc) 21.362 ns - 36 cycles(tsc) 9.141 ns - improved 57.6%
> > 128 - 101 cycles(tsc) 25.481 ns - 33 cycles(tsc) 8.286 ns - improved 67.3%
> > 158 - 103 cycles(tsc) 25.909 ns - 36 cycles(tsc) 9.179 ns - improved 65.0%
> > 250 - 105 cycles(tsc) 26.481 ns - 39 cycles(tsc) 9.994 ns - improved 62.9%
> >
> > Notice how ALL of the bulk sizes now are faster than the 47 cycles of
> > the normal slub fastpath. This is amazing!
> >
> > A little strangely, the tuning didn't seem to help the fallback version.
>
> Hello,
>
> Looks very nice.
Thanks :-)
> I have some questions about your benchmark and result.
>
> 1. Does the slab is merged?
> - Your above result shows that fallback bulk for 30, 32 takes longer
> than fallback bulk for 16. This is strange result because fallback
> bulk allocation/free for 16, 30, 32 should happens only on cpu cache.
I guess it depends on how "used/full" the page is... some other
subsystem can hold on to objects...
> If the slab is merged, you should turn off merging to get precise
> result.
Yes, I think it is merged... how do I turn off merging?
Before adjusting/tuning the SLAB.
$ sudo grep -H . /sys/kernel/slab/:t-0000256/{cpu_partial,min_partial,order,objs_per_slab}
/sys/kernel/slab/:t-0000256/cpu_partial:13
/sys/kernel/slab/:t-0000256/min_partial:5
/sys/kernel/slab/:t-0000256/order:1
/sys/kernel/slab/:t-0000256/objs_per_slab:32
Run01: non-tuned
1 - 64 cycles(tsc) 16.092 ns - 47 cycles(tsc) 11.886 ns
2 - 57 cycles(tsc) 14.258 ns - 28 cycles(tsc) 7.226 ns
3 - 54 cycles(tsc) 13.626 ns - 23 cycles(tsc) 5.822 ns
4 - 53 cycles(tsc) 13.328 ns - 20 cycles(tsc) 5.185 ns
8 - 93 cycles(tsc) 23.301 ns - 49 cycles(tsc) 12.406 ns
16 - 83 cycles(tsc) 20.902 ns - 37 cycles(tsc) 9.418 ns
30 - 77 cycles(tsc) 19.400 ns - 30 cycles(tsc) 7.748 ns
32 - 79 cycles(tsc) 19.938 ns - 30 cycles(tsc) 7.751 ns
34 - 80 cycles(tsc) 20.215 ns - 35 cycles(tsc) 8.907 ns
48 - 85 cycles(tsc) 21.391 ns - 24 cycles(tsc) 6.219 ns
64 - 93 cycles(tsc) 23.272 ns - 67 cycles(tsc) 16.874 ns
128 - 101 cycles(tsc) 25.407 ns - 72 cycles(tsc) 18.097 ns
158 - 105 cycles(tsc) 26.319 ns - 72 cycles(tsc) 18.164 ns
250 - 107 cycles(tsc) 26.783 ns - 72 cycles(tsc) 18.246 ns
Run02: non-tuned
1 - 63 cycles(tsc) 15.864 ns - 46 cycles(tsc) 11.672 ns
2 - 56 cycles(tsc) 14.153 ns - 28 cycles(tsc) 7.119 ns
3 - 54 cycles(tsc) 13.681 ns - 23 cycles(tsc) 5.846 ns
4 - 53 cycles(tsc) 13.354 ns - 20 cycles(tsc) 5.141 ns
8 - 51 cycles(tsc) 12.970 ns - 19 cycles(tsc) 4.954 ns
16 - 51 cycles(tsc) 12.763 ns - 20 cycles(tsc) 5.003 ns
30 - 51 cycles(tsc) 12.760 ns - 20 cycles(tsc) 5.065 ns
32 - 80 cycles(tsc) 20.045 ns - 37 cycles(tsc) 9.311 ns
34 - 73 cycles(tsc) 18.454 ns - 27 cycles(tsc) 6.773 ns
48 - 82 cycles(tsc) 20.544 ns - 35 cycles(tsc) 8.973 ns
64 - 87 cycles(tsc) 21.809 ns - 60 cycles(tsc) 15.167 ns
128 - 103 cycles(tsc) 25.772 ns - 63 cycles(tsc) 15.874 ns
158 - 104 cycles(tsc) 26.215 ns - 61 cycles(tsc) 15.433 ns
250 - 107 cycles(tsc) 26.926 ns - 60 cycles(tsc) 15.058 ns
Notice the variation is fairly high between runs... :-(
> 3. For more precise test setup, how about setting cpu affinity?
Sure, starting to use test cmd:
sudo taskset -c 1 modprobe slab_bulk_test01 && rmmod slab_bulk_test01 && sudo dmesg
Code:
https://github.com/netoptimizer/prototype-kernel/blob/master/kernel/mm/slab_bulk_test01.c
For these runs I've also disabled HT (Hyper Threading) in the BIOS, as
this tuned out to be a big disturbance for my network testing use-case.
(ps. I've hacked together a use-case in ixgbe/skbuff.c, but only TX complete
bulk-free which shows improvement of 3ns and 16ns with this slab
tuning, once I also implement alloc-bulk I should get a better boost).
> 2. Could you show result with only tuning min_partial?
> - I guess that much improvement for Bulk-API comes from disappearing
> slab page allocation/free cost rather than tuning cpu_partial.
Sure, there are some more runs:
sudo sh -c 'echo 9 > /sys/kernel/slab/:t-0000256/min_partial'
Run03: tuned min_partial=9
1 - 63 cycles(tsc) 15.910 ns - 46 cycles(tsc) 11.720 ns
2 - 57 cycles(tsc) 14.318 ns - 29 cycles(tsc) 7.266 ns
3 - 55 cycles(tsc) 13.762 ns - 23 cycles(tsc) 5.937 ns
4 - 53 cycles(tsc) 13.459 ns - 20 cycles(tsc) 5.211 ns
8 - 51 cycles(tsc) 13.001 ns - 19 cycles(tsc) 4.821 ns
16 - 51 cycles(tsc) 12.772 ns - 20 cycles(tsc) 5.016 ns
30 - 84 cycles(tsc) 21.135 ns - 28 cycles(tsc) 7.047 ns
32 - 83 cycles(tsc) 20.887 ns - 28 cycles(tsc) 7.133 ns
34 - 81 cycles(tsc) 20.454 ns - 28 cycles(tsc) 7.024 ns
48 - 86 cycles(tsc) 21.662 ns - 32 cycles(tsc) 8.121 ns
64 - 92 cycles(tsc) 23.027 ns - 52 cycles(tsc) 13.033 ns
128 - 97 cycles(tsc) 24.270 ns - 51 cycles(tsc) 12.865 ns
158 - 105 cycles(tsc) 26.290 ns - 53 cycles(tsc) 13.435 ns
250 - 106 cycles(tsc) 26.545 ns - 54 cycles(tsc) 13.607 ns
Run04: tuned min_partial=9
1 - 64 cycles(tsc) 16.123 ns - 47 cycles(tsc) 11.906 ns
2 - 57 cycles(tsc) 14.267 ns - 28 cycles(tsc) 7.235 ns
3 - 54 cycles(tsc) 13.691 ns - 23 cycles(tsc) 5.916 ns
4 - 53 cycles(tsc) 13.470 ns - 21 cycles(tsc) 5.278 ns
8 - 51 cycles(tsc) 12.991 ns - 19 cycles(tsc) 4.815 ns
16 - 50 cycles(tsc) 12.651 ns - 19 cycles(tsc) 4.840 ns
30 - 81 cycles(tsc) 20.282 ns - 35 cycles(tsc) 8.835 ns
32 - 77 cycles(tsc) 19.327 ns - 29 cycles(tsc) 7.403 ns
34 - 77 cycles(tsc) 19.438 ns - 31 cycles(tsc) 7.879 ns
48 - 85 cycles(tsc) 21.367 ns - 34 cycles(tsc) 8.563 ns
64 - 87 cycles(tsc) 21.830 ns - 55 cycles(tsc) 13.820 ns
128 - 109 cycles(tsc) 27.445 ns - 56 cycles(tsc) 14.152 ns
158 - 102 cycles(tsc) 25.576 ns - 60 cycles(tsc) 15.120 ns
250 - 108 cycles(tsc) 27.069 ns - 58 cycles(tsc) 14.534 ns
Looking at Run04 the win was not so big...
Also adjust cpu_partial:
sudo sh -c 'echo 256 > /sys/kernel/slab/:t-0000256/cpu_partial'
$ sudo grep -H . /sys/kernel/slab/:t-0000256/{cpu_partial,min_partial,order,objs_per_slab}
/sys/kernel/slab/:t-0000256/cpu_partial:256
/sys/kernel/slab/:t-0000256/min_partial:9
/sys/kernel/slab/:t-0000256/order:1
/sys/kernel/slab/:t-0000256/objs_per_slab:32
Run05: also tuned cpu_partial=256
1 - 63 cycles(tsc) 15.867 ns - 46 cycles(tsc) 11.656 ns
2 - 56 cycles(tsc) 14.229 ns - 28 cycles(tsc) 7.131 ns
3 - 54 cycles(tsc) 13.587 ns - 23 cycles(tsc) 5.760 ns
4 - 53 cycles(tsc) 13.287 ns - 20 cycles(tsc) 5.081 ns
8 - 51 cycles(tsc) 12.935 ns - 19 cycles(tsc) 4.953 ns
16 - 50 cycles(tsc) 12.707 ns - 20 cycles(tsc) 5.074 ns
30 - 79 cycles(tsc) 19.927 ns - 28 cycles(tsc) 7.057 ns
32 - 79 cycles(tsc) 19.977 ns - 31 cycles(tsc) 7.762 ns
34 - 79 cycles(tsc) 19.800 ns - 33 cycles(tsc) 8.392 ns
48 - 93 cycles(tsc) 23.316 ns - 35 cycles(tsc) 8.777 ns
64 - 92 cycles(tsc) 23.144 ns - 33 cycles(tsc) 8.449 ns
128 - 97 cycles(tsc) 24.268 ns - 35 cycles(tsc) 8.943 ns
158 - 106 cycles(tsc) 26.606 ns - 40 cycles(tsc) 10.067 ns
250 - 109 cycles(tsc) 27.385 ns - 51 cycles(tsc) 12.957 ns
Run06: also tuned cpu_partial=256
1 - 63 cycles(tsc) 15.952 ns - 46 cycles(tsc) 11.710 ns
2 - 57 cycles(tsc) 14.309 ns - 29 cycles(tsc) 7.261 ns
3 - 54 cycles(tsc) 13.703 ns - 23 cycles(tsc) 5.858 ns
4 - 53 cycles(tsc) 13.394 ns - 20 cycles(tsc) 5.161 ns
8 - 52 cycles(tsc) 13.013 ns - 19 cycles(tsc) 4.809 ns
16 - 94 cycles(tsc) 23.734 ns - 49 cycles(tsc) 12.376 ns
30 - 88 cycles(tsc) 22.221 ns - 35 cycles(tsc) 8.933 ns
32 - 101 cycles(tsc) 25.319 ns - 41 cycles(tsc) 10.437 ns
34 - 98 cycles(tsc) 24.711 ns - 41 cycles(tsc) 10.485 ns
48 - 96 cycles(tsc) 24.119 ns - 41 cycles(tsc) 10.479 ns
64 - 100 cycles(tsc) 25.223 ns - 39 cycles(tsc) 9.766 ns
128 - 100 cycles(tsc) 25.078 ns - 34 cycles(tsc) 8.602 ns
158 - 102 cycles(tsc) 25.673 ns - 38 cycles(tsc) 9.645 ns
250 - 110 cycles(tsc) 27.560 ns - 40 cycles(tsc) 10.046 ns
(p.s. I'm currently on vacation for 3 weeks...)
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>