Hi Sage, While Abhishek (cc'ed) was working on backporting a bug fix in obj_bench[1], he ended up pulling more and more commits in an attempt to have it apply cleanly. The goal is to backport these two commits from http://tracker.ceph.com/issues/12947 9bcf5f0 tools: fix race condition in seq/rand bench 55a6f9e bufferlist: implement bufferlist::invalidate_crc() After looking at the modification history of obj_bencher.cc, going from 9bcf5f0 back to hammer, it turns out there are two bug fixes that would be nice to have in hammer ( cd0f2b7 obj_bencher: does not accumulate bandwidth that is zero + e3d62a9 common: make rados bench return correctly errno ) and one feature (3944264 obj_bencher: add IOPS metric calculation). The rest is refactoring and cleanup, reason why the backports are difficult. I think we could backport all commits from 9bcf5f0 tools: fix race condition in seq/rand bench down to 069d95e obj_bencher: fix indents Leaving out the documentation commit. Abhishek tested that it applies cleanly, compiles and passes make check. Luckily all these commits are limited to the obj_bencher.{cc,h}, rest_bench.cc and rados.cc files. More importantly, although this series look like a lot, in reality it does not change much. Not only does it make this backport simpler because there are no conflicts to resolve, I think it would also make future backports in this area easier because they will not stumble on cosmetic changes. Do you see a problem to proceed in this direction ? Cheers [1] https://github.com/ceph/ceph/pull/5812 $ git --no-pager log --full-diff --no-merges --stat --oneline ceph/hammer..ceph/master src/common/obj_bencher.cc 08296dc rados: make 'rados bench' support json format output Fixes: #12864 rados bench add '[--format json]' and '[-o | --output outfile]' support. output option only take effect in json format. now we can use the bench result draw performance graph easily. src/common/obj_bencher.cc | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------ src/common/obj_bencher.h | 9 ++++++++ src/test/test_rados_tool.sh | 5 +++++ src/tools/rados/rados.cc | 24 ++++++++++++++++++++ 4 files changed, 148 insertions(+), 14 deletions(-) 4eaa9ea fix print error of rados bench src/common/obj_bencher.cc | 10 ++++------ src/common/obj_bencher.h | 9 +++++---- 2 files changed, 9 insertions(+), 10 deletions(-) f4d2032 c++11: disambiguate std::isnan src/common/obj_bencher.cc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) 90fdbbf common, tools, test: Add "rados purge" feature to remove all objects from a pool src/common/obj_bencher.cc | 24 ++++++++++++------------ src/common/obj_bencher.h | 10 +++++++--- src/test/test_rados_tool.sh | 7 +++++++ src/tools/rados/rados.cc | 37 +++++++++++++++++++++++++++++++++++-- src/tools/rest_bench.cc | 7 ++++--- 5 files changed, 65 insertions(+), 20 deletions(-) 9bcf5f0 tools: fix race condition in seq/rand bench src/common/obj_bencher.cc | 80 ++++++++++++++++++++++++++++++++++++++++++++------------------------------------ 1 file changed, 44 insertions(+), 36 deletions(-) 813897c obj_bencher: remove trailing space src/common/obj_bencher.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) 0f7aeee obj_bencher: check run_name and prefix for empty string instead of NULL src/common/obj_bencher.cc | 12 ++++++------ src/common/obj_bencher.h | 4 ++-- src/tools/rest_bench.cc | 4 ++-- 3 files changed, 10 insertions(+), 10 deletions(-) ca6abca tools: add --no-verify option to rados bench src/common/obj_bencher.cc | 67 ++++++++++++++++++++++++++++++++++++++++++------------------------- src/common/obj_bencher.h | 6 +++--- src/tools/rados/rados.cc | 11 ++++++++++- 3 files changed, 55 insertions(+), 29 deletions(-) b894fc7 tools: Don't delete, recreate and re-fill buffers in rados bench. src/common/obj_bencher.cc | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) fe79bab common: fix ObjBencher::aio_bench signature src/common/obj_bencher.cc | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) 5c6190d common: rados bench data.finished = 0 src/common/obj_bencher.cc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) 1ac7279 common: remove unused maxObjectsToCreate src/common/obj_bencher.cc | 22 +++++++++------------- src/common/obj_bencher.h | 4 ++-- src/tools/rados/rados.cc | 2 +- 3 files changed, 12 insertions(+), 16 deletions(-) cd0f2b7 obj_bencher: does not accumulate bandwidth that is zero src/common/obj_bencher.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) a87ac4d obj_bencher: aio_bench - rename op_size to object_size src/common/obj_bencher.cc | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) 1d1d0aa obj_bencher: remove excess 'object_size = op_size' src/common/obj_bencher.cc | 2 -- 1 file changed, 2 deletions(-) 31d16e9 obj_bencher: remove 'trans_size' as obsolete src/common/obj_bencher.cc | 5 ++--- src/common/obj_bencher.h | 1 - 2 files changed, 2 insertions(+), 4 deletions(-) e3d62a9 common: make rados bench return correctly errno. src/common/obj_bencher.cc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) 3944264 obj_bencher: add IOPS metric calculation src/common/obj_bencher.cc | 28 ++++++++++++++++++++++++++++ src/common/obj_bencher.h | 3 +++ 2 files changed, 31 insertions(+) ddb422f obj_bencher: cosmetic display fixes src/common/obj_bencher.cc | 30 ++++++++++++------------------ 1 file changed, 12 insertions(+), 18 deletions(-) e360bfd obj_bencher: generalize vec_stddev function src/common/obj_bencher.cc | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) 069d95e obj_bencher: fix indents src/common/obj_bencher.cc | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------------------------- 1 file changed, 58 insertions(+), 58 deletions(-) c1e792d doc: update doc with latest code doc/dev/osd_internals/map_message_handling.rst | 4 ++-- doc/dev/osd_internals/snaps.rst | 4 ++-- src/common/obj_bencher.cc | 3 +-- src/include/cephfs/libcephfs.h | 18 +++++++++--------- src/include/rados/librados.h | 38 +++++++++++++++++++------------------- src/include/rados/librados.hpp | 15 +++++++-------- src/include/radosstriper/libradosstriper.h | 8 ++++---- src/include/str_map.h | 2 +- src/mds/CInode.h | 4 ++-- src/mon/Elector.h | 2 +- src/mon/Monitor.h | 4 ++-- src/mon/MonitorStore.h | 3 ++- src/mon/PGMap.cc | 1 + src/mon/Paxos.h | 9 +++++---- src/mon/PaxosService.h | 8 ++++---- src/msg/Messenger.h | 12 ++++++------ src/msg/async/AsyncMessenger.h | 5 +---- src/msg/simple/SimpleMessenger.h | 2 +- src/os/CollectionIndex.h | 2 +- src/os/ObjectStore.h | 6 +++--- src/osd/OSDCap.h | 1 - src/osd/osd_types.h | 8 ++++---- src/test/messenger/xio_dispatcher.h | 2 +- 23 files changed, 79 insertions(+), 82 deletions(-) -- Loïc Dachary, Artisan Logiciel Libre
Attachment:
signature.asc
Description: OpenPGP digital signature
