After reading Joe Mario's comments[0] and blog post[1] (which conveniently includes a source file to demonstrate the cacheline problems[2]), it looks like it would be useful to create a teuthology test that does something similar to the current mempool test[3] while c2c collects information. There could be two runs: * N threads looping on shard[0].bytes += value * N threads looping on shard[pick_a_shard()].bytes += value And comparing c2c reports should show the run with sharding is better. Does that sound like a reasonable approach or do you think there is a simpler way to verify the optimization works as expected ? Cheers [0] https://github.com/ceph/ceph/pull/9431 [1] https://joemario.github.io/blog/2016/09/01/c2c-blog/ [2] https://github.com/joemario/perf-c2c-usage-files/blob/master/false_sharing_example.c [3] https://github.com/ceph/ceph/blob/3fb62b1a4c5d2a51db3f2b6276c7c48b3d8da201/src/test/test_mempool.cc#L405 On 17/03/2021 10:16, Loïc Dachary wrote: > Hi Adam, > > Thanks for the pointer: very informative indeed :-) perf c2c seems to be a good tool and works on my laptop. I'll have to dig deeper to figure out what it does exactly. To be continued! > > Cheers > > On 17/03/2021 08:48, Adam Kupczyk wrote: >> Loic, >> >> Here is some discussion on cache false sharing that you may find valuable: https://github.com/ceph/ceph/pull/9431 <https://github.com/ceph/ceph/pull/9431> . >> Maybe it is possible to turn perf c2c into some systemic check? >> >> Best regards, >> Adam >> >> On Tue, 16 Mar 2021 at 22:59, Loïc Dachary <loic@xxxxxxxxxxx <mailto:loic@xxxxxxxxxxx>> wrote: >> >> >> >> On 16/03/2021 22:01, Mark Nelson wrote: >> > Hi Loïc, >> > >> > >> > I don't have any tests specifically for this, but looking at #39057, I wonder if something could be crafted using the check_shard_select test that Adam wrote as a template. What do you think? >> That sounds like a good idea. And maybe using cachegrind[0] also? >> >> [0] https://www.valgrind.org/docs/manual/cg-manual.html <https://www.valgrind.org/docs/manual/cg-manual.html> >> > >> > >> > Mark >> > >> > >> > On 3/16/21 3:52 PM, Loïc Dachary wrote: >> >> Hi Mark, >> >> >> >> While trying to figure out a random failure in the mempool tests[0] introduced when fixing a bug in how mempool selects shards holding the byte count of a given pool[1] earlier this year, I was intrigued by this "cache line ping pong" problem[2]. And I wonder if you have some kind of benchmark, somewhere in your toolbox, that someone could use to demonstrate the problem. Maybe such a code could be adapted to show the benefit of the optimization implemented in mempool? >> >> >> >> Cheers >> >> >> >> [0] https://tracker.ceph.com/issues/49781#note-9 <https://tracker.ceph.com/issues/49781#note-9> >> >> [1] https://github.com/ceph/ceph/pull/39057/files <https://github.com/ceph/ceph/pull/39057/files> >> >> [2] https://www.drdobbs.com/parallel/understanding-and-avoiding-memory-issues/212400410 <https://www.drdobbs.com/parallel/understanding-and-avoiding-memory-issues/212400410> >> >> >> >> >> >> _______________________________________________ >> >> Dev mailing list -- dev@xxxxxxx <mailto:dev@xxxxxxx> >> >> To unsubscribe send an email to dev-leave@xxxxxxx <mailto:dev-leave@xxxxxxx> >> > _______________________________________________ >> > Dev mailing list -- dev@xxxxxxx <mailto:dev@xxxxxxx> >> > To unsubscribe send an email to dev-leave@xxxxxxx <mailto:dev-leave@xxxxxxx> >> >> -- >> Loïc Dachary, Artisan Logiciel Libre >> >> >> _______________________________________________ >> Dev mailing list -- dev@xxxxxxx <mailto:dev@xxxxxxx> >> To unsubscribe send an email to dev-leave@xxxxxxx <mailto:dev-leave@xxxxxxx> >> >> >> _______________________________________________ >> Dev mailing list -- dev@xxxxxxx >> To unsubscribe send an email to dev-leave@xxxxxxx -- Loïc Dachary, Artisan Logiciel Libre _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
