Re: 4 failed, 298 passed in dzafman-2013-09-23_17:50:06-rados-wip-5862-testing-basic-plana

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The osd.4 crash in 13443 is bug #5951:

2013-09-23 21:23:28.378428 1034e700  0 filestore(/var/lib/ceph/osd/ceph-4)  error (17) File exists not handled on operation 20 (6579.0.0, or op 0, counting from 0)
2013-09-23 21:23:28.862204 1034e700  0 filestore(/var/lib/ceph/osd/ceph-4) unexpected error code
2013-09-23 21:23:28.864816 1034e700  0 filestore(/var/lib/ceph/osd/ceph-4)  transaction dump:
{ "ops": [
        { "op_num": 0,
          "op_name": "mkcoll",
          "collection": "4.6_head"},
        { "op_num": 1,
          "op_name": "collection_setattr",
          "collection": "4.6_head",
          "name": "info",
          "length": 1},
        { "op_num": 2,
          "op_name": "omap_setkeys",
          "collection": "meta",
          "oid": "16ef7597\/infos\/head\/\/-1",
          "attr_lens": { "4.6_biginfo": 125,
              "4.6_epoch": 4,
              "4.6_info": 576}},
        { "op_num": 3,
          "op_name": "touch",
          "collection": "meta",
          "oid": "1039d44e\/pglog_4.6\/0\/\/-1"},
        { "op_num": 4,
          "op_name": "omap_rmkeys",
          "collection": "meta",
          "oid": "1039d44e\/pglog_4.6\/0\/\/-1"},
        { "op_num": 5,
          "op_name": "omap_setkeys",
          "collection": "meta",
          "oid": "1039d44e\/pglog_4.6\/0\/\/-1",
          "attr_lens": {}}]}

2013-09-23 21:23:28.959220 1a282700  5 osd.4 pg_epoch: 424 pg[32.0( empty local-les=424 n=0 ec=96 les/c 400/400 419/419/419) [4,1] r=0 lpr=419 pi=364-418/6 mlcod 0'0 active] enter
 Started/Primary/Active/Activating
2013-09-23 21:23:29.116768 1034e700 -1 os/FileStore.cc: In function 'unsigned int FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t, int)' thread 1034e700 time 2013-0
 9-23 21:23:28.920055
os/FileStore.cc: 2461: FAILED assert(0 == "unexpected error")

 ceph version 0.69-220-g4f7526a (4f7526a785692795ee29f7101b8b18482b4c6e11)
 1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0xffc) [0x72473c]
 2: (FileStore::_do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long, ThreadPool::TPHandle*)+0x71) [0x72b241]
 3: (FileStore::_do_op(FileStore::OpSequencer*, ThreadPool::TPHandle&)+0x291) [0x72b4f1]
 4: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0x93da36]
 5: (ThreadPool::WorkThread::entry()+0x10) [0x93f840]
 6: (()+0x7e9a) [0x503be9a]
 7: (clone()+0x6d) [0x6c71ccd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

David Zafman
Senior Developer
http://www.inktank.com

On Sep 24, 2013, at 12:03 PM, Sage Weil <sage@xxxxxxxxxxx> wrote:

> On Tue, 24 Sep 2013, David Zafman wrote:
>> 
>> Rados suite test run results for wip-5862.  2 "scrub mismatch" from mon
>> (known problem).  2 are valgrind issues found with mds and osd. 
> 
> What is the osd valgrind failure?  And the osd.4 crash on 13443?
> 
> (Note that the teuthology.log will include message about valgrind issues 
> found in the mds log, but does not generate an actual error about it.)
> 
> Thanks!
> sage
> 
> 
>> 
>> David Zafman
>> Senior Developer
>> http://www.inktank.com
>> 
>> Begin forwarded message:
>> 
>>      From: teuthworker@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> Subject: 4 failed, 298 passed in
>> dzafman-2013-09-23_17:50:06-rados-wip-5862-testing-basic-plana
>> Date: September 23, 2013 10:48:00 PM PDT
>> To: david.zafman@xxxxxxxxxxx
>> 
>> Test Run:
>> dzafman-2013-09-23_17:50:06-rados-wip-5862-testing-basic-plana
>> =================================================================
>> logs:  http://qa-proxy.ceph.com/teuthology/dzafman-2013-09-23_17:50:06-rados-wip
>> -5862-testing-basic-plana/
>> failed: 4
>> hung:   0
>> passed: 298
>> 
>> Failed
>> =================================================================
>> [13187]  rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/snaps-few-objects.yaml}
>> -----------------------------------------------------------------
>> time:   2095s
>> log:   http://qa-proxy.ceph.com/teuthology/dzafman-2013-09-23_17:50:06-rados-wi
>> p-5862-testing-basic-plana/13187/
>> 
>>    "2013-09-23 18:22:37.805204 mon.0 10.214.132.24:6789/0 514 : [ERR]
>> scrub
>>    mismatch" in cluster log
>> 
>> [13449]  rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_api_tests.yaml
>> validater/valgrind.yaml}
>> -----------------------------------------------------------------
>> time:   1067s
>> log:   http://qa-proxy.ceph.com/teuthology/dzafman-2013-09-23_17:50:06-rados-wi
>> p-5862-testing-basic-plana/13449/
>> 
>>    saw valgrind issues
>> 
>> [13443]  rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_api_tests.yaml
>> validater/valgrind.yaml}
>> -----------------------------------------------------------------
>> time:   1307s
>> log:   http://qa-proxy.ceph.com/teuthology/dzafman-2013-09-23_17:50:06-rados-wi
>> p-5862-testing-basic-plana/13443/
>> 
>>    timed out waiting for admin_socket to appear after osd.4 restart
>> 
>> [13227]  rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/snaps-few-objects.yaml}
>> -----------------------------------------------------------------
>> time:   2374s
>> log:   http://qa-proxy.ceph.com/teuthology/dzafman-2013-09-23_17:50:06-rados-wi
>> p-5862-testing-basic-plana/13227/
>> 
>>    "2013-09-23 18:41:57.370588 mon.0 10.214.132.34:6789/0 20 : [ERR]
>> scrub
>>    mismatch" in cluster log
>> 
>> 
>> Passed
>> =================================================================
>> [13329] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/readwrite.yaml}
>> time:    1327s
>> 
>> [13414] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1926s
>> 
>> [13278] rados/singleton/{all/mon-thrasher.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    811s
>> 
>> [13368] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    322s
>> 
>> [13359] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1529s
>> 
>> [13403] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1651s
>> 
>> [13402] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1878s
>> 
>> [13405] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    836s
>> 
>> [13404] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    243s
>> 
>> [13407] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1249s
>> 
>> [13358] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/radosbench.yaml}
>> time:    3144s
>> 
>> [13360] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1994s
>> 
>> [13361] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1894s
>> 
>> [13362] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    276s
>> 
>> [13363] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    966s
>> 
>> [13364] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2300s
>> 
>> [13365] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml}
>> time:    1374s
>> 
>> [13366] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2202s
>> 
>> [13367] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1945s
>> 
>> [13386] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    279s
>> 
>> [13387] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    988s
>> 
>> [13295] rados/singleton-nomsgr/{all/objectcacher-stress.yaml}
>> time:    1144s
>> 
>> [13243] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2400s
>> 
>> [13242] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    445s
>> 
>> [13168] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/rados_5925.yaml}
>> time:    200s
>> 
>> [13169] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1455s
>> 
>> [13247] rados/multimon/{clusters/21.yaml msgr-failures/many.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    186s
>> 
>> [13246] rados/multimon/{clusters/21.yaml msgr-failures/few.yaml
>> tasks/mon_recovery.yaml}
>> time:    888s
>> 
>> [13245] rados/multimon/{clusters/21.yaml msgr-failures/few.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    191s
>> 
>> [13244] rados/multimon/{clusters/21.yaml msgr-failures/few.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    171s
>> 
>> [13162] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml tasks/rados_workunit_loadgen_mix.yaml}
>> time:    1454s
>> 
>> [13163] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml tasks/rados_workunit_loadgen_mostlyread.yaml}
>> time:    6072s
>> 
>> [13160] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml tasks/rados_stress_watch.yaml}
>> time:    469s
>> 
>> [13161] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml tasks/rados_workunit_loadgen_big.yaml}
>> time:    2034s
>> 
>> [13166] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    795s
>> 
>> [13167] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2350s
>> 
>> [13164] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/rados_5925.yaml}
>> time:    271s
>> 
>> [13165] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1082s
>> 
>> [13199] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2158s
>> 
>> [13412] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2124s
>> 
>> [13413] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml}
>> time:    1461s
>> 
>> [13410] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    285s
>> 
>> [13198] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    378s
>> 
>> [13416] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    261s
>> 
>> [13417] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/rados_api_tests.yaml}
>> time:    912s
>> 
>> [13389] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1405s
>> 
>> [13415] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1909s
>> 
>> [13319] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    2043s
>> 
>> [13323] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/readwrite.yaml}
>> time:    1258s
>> 
>> [13419] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/readwrite.yaml}
>> time:    1244s
>> 
>> [13318] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1883s
>> 
>> [13157] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml tasks/rados_api_tests.yaml}
>> time:    1056s
>> 
>> [13156] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml tasks/rados_workunit_loadgen_mostlyread.yaml}
>> time:    6006s
>> 
>> [13155] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml tasks/rados_workunit_loadgen_mix.yaml}
>> time:    1477s
>> 
>> [13154] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml tasks/rados_workunit_loadgen_big.yaml}
>> time:    2115s
>> 
>> [13153] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml tasks/rados_stress_watch.yaml}
>> time:    449s
>> 
>> [13152] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml tasks/rados_python.yaml}
>> time:    461s
>> 
>> [13151] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml tasks/rados_cls_all.yaml}
>> time:    360s
>> 
>> [13150] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml tasks/rados_api_tests.yaml}
>> time:    1063s
>> 
>> [13248] rados/multimon/{clusters/21.yaml msgr-failures/many.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    203s
>> 
>> [13314] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    286s
>> 
>> [13317] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml workloads/readwrite.yaml}
>> time:    1199s
>> 
>> [13316] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2858s
>> 
>> [13311] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1359s
>> 
>> [13310] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2978s
>> 
>> [13159] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml tasks/rados_python.yaml}
>> time:    545s
>> 
>> [13158] rados/basic/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml tasks/rados_cls_all.yaml}
>> time:    537s
>> 
>> [13395] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/readwrite.yaml}
>> time:    1420s
>> 
>> [13193] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1305s
>> 
>> [13397] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1918s
>> 
>> [13396] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1910s
>> 
>> [13258] rados/multimon/{clusters/6.yaml msgr-failures/few.yaml
>> tasks/mon_recovery.yaml}
>> time:    400s
>> 
>> [13259] rados/multimon/{clusters/6.yaml msgr-failures/many.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    171s
>> 
>> [13393] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/rados_api_tests.yaml}
>> time:    876s
>> 
>> [13219] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2475s
>> 
>> [13254] rados/multimon/{clusters/3.yaml msgr-failures/many.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    152s
>> 
>> [13255] rados/multimon/{clusters/3.yaml msgr-failures/many.yaml
>> tasks/mon_recovery.yaml}
>> time:    404s
>> 
>> [13256] rados/multimon/{clusters/6.yaml msgr-failures/few.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    162s
>> 
>> [13257] rados/multimon/{clusters/6.yaml msgr-failures/few.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    177s
>> 
>> [13250] rados/multimon/{clusters/3.yaml msgr-failures/few.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    173s
>> 
>> [13251] rados/multimon/{clusters/3.yaml msgr-failures/few.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    153s
>> 
>> [13252] rados/multimon/{clusters/3.yaml msgr-failures/few.yaml
>> tasks/mon_recovery.yaml}
>> time:    410s
>> 
>> [13253] rados/multimon/{clusters/3.yaml msgr-failures/many.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    149s
>> 
>> [13322] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/radosbench.yaml}
>> time:    2535s
>> 
>> [13398] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    308s
>> 
>> [13288] rados/singleton/{all/rest-api.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    203s
>> 
>> [13409] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1692s
>> 
>> [13313] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1892s
>> 
>> [13429] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    842s
>> 
>> [13408] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2001s
>> 
>> [13427] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1642s
>> 
>> [13315] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    857s
>> 
>> [13425] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/readwrite.yaml}
>> time:    1208s
>> 
>> [13312] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1905s
>> 
>> [13423] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/rados_api_tests.yaml}
>> time:    862s
>> 
>> [13422] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    256s
>> 
>> [13421] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1793s
>> 
>> [13420] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1926s
>> 
>> [13442] rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_api_tests.yaml
>> validater/lockdep.yaml}
>> time:    877s
>> 
>> [13451] rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_cls_all.yaml
>> validater/valgrind.yaml}
>> time:    396s
>> 
>> [13308] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    241s
>> 
>> [13309] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    744s
>> 
>> [13306] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1787s
>> 
>> [13307] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1836s
>> 
>> [13304] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/radosbench.yaml}
>> time:    2162s
>> 
>> [13305] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml workloads/readwrite.yaml}
>> time:    1282s
>> 
>> [13302] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    261s
>> 
>> [13303] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/rados_api_tests.yaml}
>> time:    718s
>> 
>> [13300] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1910s
>> 
>> [13301] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1772s
>> 
>> [13229] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1458s
>> 
>> [13228] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/rados_5925.yaml}
>> time:    221s
>> 
>> [13384] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2184s
>> 
>> [13385] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1953s
>> 
>> [13382] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2330s
>> 
>> [13383] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1607s
>> 
>> [13380] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    304s
>> 
>> [13381] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1147s
>> 
>> [13221] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1125s
>> 
>> [13220] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/rados_5925.yaml}
>> time:    299s
>> 
>> [13223] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2388s
>> 
>> [13222] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    433s
>> 
>> [13225] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/rados_api_tests.yaml}
>> time:    1084s
>> 
>> [13224] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/rados_5925.yaml}
>> time:    277s
>> 
>> [13388] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/radosbench.yaml}
>> time:    3196s
>> 
>> [13226] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/rados_mon_workunits.yaml}
>> time:    458s
>> 
>> [13392] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    223s
>> 
>> [13438] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1898s
>> 
>> [13439] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1833s
>> 
>> [13430] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2315s
>> 
>> [13431] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1446s
>> 
>> [13432] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1941s
>> 
>> [13433] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1854s
>> 
>> [13434] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    239s
>> 
>> [13435] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    889s
>> 
>> [13436] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2498s
>> 
>> [13437] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1365s
>> 
>> [13339] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    759s
>> 
>> [13338] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    234s
>> 
>> [13369] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/rados_api_tests.yaml}
>> time:    944s
>> 
>> [13333] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    760s
>> 
>> [13332] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    232s
>> 
>> [13331] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1724s
>> 
>> [13330] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1860s
>> 
>> [13337] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1645s
>> 
>> [13336] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1897s
>> 
>> [13335] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1322s
>> 
>> [13334] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/radosbench.yaml}
>> time:    3000s
>> 
>> [13238] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    421s
>> 
>> [13239] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2525s
>> 
>> [13232] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/rados_5925.yaml}
>> time:    254s
>> 
>> [13233] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1515s
>> 
>> [13230] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    481s
>> 
>> [13231] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    3058s
>> 
>> [13236] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/rados_5925.yaml}
>> time:    308s
>> 
>> [13237] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1161s
>> 
>> [13234] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    440s
>> 
>> [13235] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    3006s
>> 
>> [13406] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2481s
>> 
>> [13440] rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/mon_recovery.yaml
>> validater/lockdep.yaml}
>> time:    392s
>> 
>> [13324] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1913s
>> 
>> [13325] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1742s
>> 
>> [13326] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    333s
>> 
>> [13327] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/rados_api_tests.yaml}
>> time:    761s
>> 
>> [13320] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    249s
>> 
>> [13321] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/rados_api_tests.yaml}
>> time:    801s
>> 
>> [13289] rados/singleton/{all/rest-api.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    180s
>> 
>> [13279] rados/singleton/{all/mon-thrasher.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    820s
>> 
>> [13287] rados/singleton/{all/peer.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    309s
>> 
>> [13286] rados/singleton/{all/peer.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    162s
>> 
>> [13285] rados/singleton/{all/osd-recovery.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    258s
>> 
>> [13284] rados/singleton/{all/osd-recovery.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    242s
>> 
>> [13283] rados/singleton/{all/osd-recovery-incomplete.yaml
>> fs/btrfs.yaml msgr-failures/many.yaml}
>> time:    1303s
>> 
>> [13282] rados/singleton/{all/osd-recovery-incomplete.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml}
>> time:    1347s
>> 
>> [13281] rados/singleton/{all/osd-backfill.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    201s
>> 
>> [13280] rados/singleton/{all/osd-backfill.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    263s
>> 
>> [13207] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2241s
>> 
>> [13206] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    508s
>> 
>> [13205] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1202s
>> 
>> [13204] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/force-sync-many.yaml
>> workloads/rados_5925.yaml}
>> time:    304s
>> 
>> [13203] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2039s
>> 
>> [13202] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    675s
>> 
>> [13201] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1177s
>> 
>> [13200] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/rados_5925.yaml}
>> time:    278s
>> 
>> [13445] rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_cls_all.yaml
>> validater/valgrind.yaml}
>> time:    440s
>> 
>> [13444] rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_cls_all.yaml
>> validater/lockdep.yaml}
>> time:    348s
>> 
>> [13447] rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/mon_recovery.yaml
>> validater/valgrind.yaml}
>> time:    525s
>> 
>> [13446] rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/mon_recovery.yaml
>> validater/lockdep.yaml}
>> time:    454s
>> 
>> [13441] rados/verify/{1thrash/default.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/mon_recovery.yaml
>> validater/valgrind.yaml}
>> time:    549s
>> 
>> [13241] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1274s
>> 
>> [13209] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1526s
>> 
>> [13208] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/rados_5925.yaml}
>> time:    241s
>> 
>> [13411] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/pggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    802s
>> 
>> [13240] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync.yaml
>> workloads/rados_5925.yaml}
>> time:    318s
>> 
>> [13371] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/readwrite.yaml}
>> time:    1508s
>> 
>> [13428] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/morepggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    257s
>> 
>> [13401] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml workloads/readwrite.yaml}
>> time:    1265s
>> 
>> [13450] rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_cls_all.yaml
>> validater/lockdep.yaml}
>> time:    272s
>> 
>> [13298] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/radosbench.yaml}
>> time:    3243s
>> 
>> [13299] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/readwrite.yaml}
>> time:    1422s
>> 
>> [13353] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml workloads/readwrite.yaml}
>> time:    1460s
>> 
>> [13352] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/radosbench.yaml}
>> time:    2159s
>> 
>> [13355] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1724s
>> 
>> [13354] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1957s
>> 
>> [13357] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/rados_api_tests.yaml}
>> time:    859s
>> 
>> [13356] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/morepggrow.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    244s
>> 
>> [13290] rados/singleton/{all/thrash-rados.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    791s
>> 
>> [13291] rados/singleton/{all/thrash-rados.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    821s
>> 
>> [13292] rados/singleton-nomsgr/{all/filejournal.yaml}
>> time:    181s
>> 
>> [13293]
>> rados/singleton-nomsgr/{all/filestore-idempotent-aio-journal.yaml}
>> time:    7518s
>> 
>> [13294] rados/singleton-nomsgr/{all/filestore-idempotent.yaml}
>> time:    8612s
>> 
>> [13249] rados/multimon/{clusters/21.yaml msgr-failures/many.yaml
>> tasks/mon_recovery.yaml}
>> time:    881s
>> 
>> [13296] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    253s
>> 
>> [13297] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/rados_api_tests.yaml}
>> time:    816s
>> 
>> [13210] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    391s
>> 
>> [13211] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    3040s
>> 
>> [13212] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/rados_5925.yaml}
>> time:    225s
>> 
>> [13213] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1384s
>> 
>> [13214] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    418s
>> 
>> [13215] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2987s
>> 
>> [13216] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/rados_5925.yaml}
>> time:    294s
>> 
>> [13217] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1265s
>> 
>> [13218] rados/monthrash/{ceph/ceph.yaml clusters/9-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    405s
>> 
>> [13192] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/rados_5925.yaml}
>> time:    462s
>> 
>> [13191] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2665s
>> 
>> [13190] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    382s
>> 
>> [13197] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1144s
>> 
>> [13196] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/sync-many.yaml
>> workloads/rados_5925.yaml}
>> time:    547s
>> 
>> [13195] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2714s
>> 
>> [13194] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/one.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    399s
>> 
>> [13391] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1956s
>> 
>> [13448] rados/verify/{1thrash/none.yaml clusters/fixed-2.yaml
>> fs/btrfs.yaml msgr-failures/few.yaml tasks/rados_api_tests.yaml
>> validater/lockdep.yaml}
>> time:    720s
>> 
>> [13426] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1833s
>> 
>> [13346] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/radosbench.yaml}
>> time:    3037s
>> 
>> [13347] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/readwrite.yaml}
>> time:    1379s
>> 
>> [13400] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/radosbench.yaml}
>> time:    2830s
>> 
>> [13394] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/radosbench.yaml}
>> time:    3258s
>> 
>> [13418] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/radosbench.yaml}
>> time:    2363s
>> 
>> [13344] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    261s
>> 
>> [13345] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/rados_api_tests.yaml}
>> time:    927s
>> 
>> [13390] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2411s
>> 
>> [13184] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/rados_5925.yaml}
>> time:    271s
>> 
>> [13185] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/rados_api_tests.yaml}
>> time:    1361s
>> 
>> [13186] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml
>> thrashers/force-sync-many.yaml workloads/rados_mon_workunits.yaml}
>> time:    389s
>> 
>> [13341] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/readwrite.yaml}
>> time:    1393s
>> 
>> [13180] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/rados_5925.yaml}
>> time:    505s
>> 
>> [13181] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1314s
>> 
>> [13182] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    479s
>> 
>> [13183] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2152s
>> 
>> [13348] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1911s
>> 
>> [13349] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/default.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    2113s
>> 
>> [13188] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/rados_5925.yaml}
>> time:    210s
>> 
>> [13189] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/mon-delay.yaml thrashers/many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1403s
>> 
>> [13276] rados/singleton/{all/mon-config-keys.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    466s
>> 
>> [13342] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1833s
>> 
>> [13424] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/radosbench.yaml}
>> time:    2304s
>> 
>> [13277] rados/singleton/{all/mon-config-keys.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    451s
>> 
>> [13265] rados/multimon/{clusters/9.yaml msgr-failures/many.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    176s
>> 
>> [13264] rados/multimon/{clusters/9.yaml msgr-failures/few.yaml
>> tasks/mon_recovery.yaml}
>> time:    545s
>> 
>> [13267] rados/multimon/{clusters/9.yaml msgr-failures/many.yaml
>> tasks/mon_recovery.yaml}
>> time:    515s
>> 
>> [13266] rados/multimon/{clusters/9.yaml msgr-failures/many.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    173s
>> 
>> [13261] rados/multimon/{clusters/6.yaml msgr-failures/many.yaml
>> tasks/mon_recovery.yaml}
>> time:    466s
>> 
>> [13260] rados/multimon/{clusters/6.yaml msgr-failures/many.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    167s
>> 
>> [13263] rados/multimon/{clusters/9.yaml msgr-failures/few.yaml
>> tasks/mon_clock_with_skews.yaml}
>> time:    177s
>> 
>> [13262] rados/multimon/{clusters/9.yaml msgr-failures/few.yaml
>> tasks/mon_clock_no_skews.yaml}
>> time:    191s
>> 
>> [13269] rados/singleton/{all/admin-socket.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    117s
>> 
>> [13268] rados/singleton/{all/admin-socket.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    117s
>> 
>> [13343] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1701s
>> 
>> [13379] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1767s
>> 
>> [13378] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    1967s
>> 
>> [13377] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/readwrite.yaml}
>> time:    1301s
>> 
>> [13376] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/radosbench.yaml}
>> time:    3180s
>> 
>> [13375] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/rados_api_tests.yaml}
>> time:    905s
>> 
>> [13374] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    318s
>> 
>> [13373] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/snaps-many-objects.yaml}
>> time:    1913s
>> 
>> [13372] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2092s
>> 
>> [13340] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/pggrow.yaml
>> workloads/radosbench.yaml}
>> time:    2672s
>> 
>> [13370] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/osd-delay.yaml thrashers/default.yaml
>> workloads/radosbench.yaml}
>> time:    2708s
>> 
>> [13351] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/rados_api_tests.yaml}
>> time:    921s
>> 
>> [13399] rados/thrash/{clusters/fixed-2.yaml fs/xfs.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/rados_api_tests.yaml}
>> time:    753s
>> 
>> [13328] rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml
>> msgr-failures/osd-delay.yaml thrashers/mapgap.yaml
>> workloads/radosbench.yaml}
>> time:    3182s
>> 
>> [13350] rados/thrash/{clusters/fixed-2.yaml fs/ext4.yaml
>> msgr-failures/few.yaml thrashers/mapgap.yaml
>> workloads/admin_socket_objecter_requests.yaml}
>> time:    275s
>> 
>> [13179] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2196s
>> 
>> [13178] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    413s
>> 
>> [13274] rados/singleton/{all/lost-unfound.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    186s
>> 
>> [13275] rados/singleton/{all/lost-unfound.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    215s
>> 
>> [13272] rados/singleton/{all/dump-stuck.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    222s
>> 
>> [13273] rados/singleton/{all/dump-stuck.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    240s
>> 
>> [13270] rados/singleton/{all/cephtool.yaml fs/btrfs.yaml
>> msgr-failures/few.yaml}
>> time:    347s
>> 
>> [13271] rados/singleton/{all/cephtool.yaml fs/btrfs.yaml
>> msgr-failures/many.yaml}
>> time:    367s
>> 
>> [13171] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2857s
>> 
>> [13170] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/many.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    724s
>> 
>> [13173] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1300s
>> 
>> [13172] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/rados_5925.yaml}
>> time:    216s
>> 
>> [13175] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/snaps-few-objects.yaml}
>> time:    2867s
>> 
>> [13174] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/one.yaml
>> workloads/rados_mon_workunits.yaml}
>> time:    699s
>> 
>> [13177] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/rados_api_tests.yaml}
>> time:    1168s
>> 
>> [13176] rados/monthrash/{ceph/ceph.yaml clusters/3-mons.yaml
>> fs/xfs.yaml msgr-failures/few.yaml thrashers/sync-many.yaml
>> workloads/rados_5925.yaml}
>> time:    527s
>> 
>> 
>> 

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux