Hi Sage, Samuel & All,
I upgraded to GAINT, but still appearing that errors |: I'm trying on deleting related objects/volumes, but very hard to verify missing objects :(. Guide me to resolve it, please! (I send attached detail log). 2014-11-03 11:37:57.730820 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.856578 7f28fc013700 -1 *** Caught signal (Segmentation fault) ** in thread 7f28fc013700 ceph version 0.87-6-gdba7def (dba7defc623474ad17263c9fccfec60fe7a439f0) 1: /usr/bin/ceph-osd() [0x9b6725] 2: (()+0xfcb0) [0x7f291fc2acb0] 3: (ReplicatedPG::trim_object(hobject_t const&)+0x395) [0x811b55] 4: (ReplicatedPG::TrimmingObjects::react(ReplicatedPG::SnapTrim const&)+0x43e) [0x82b9be] 5: (boost::statechart::simple_state<ReplicatedPG::TrimmingObjects, ReplicatedPG::SnapTrimmer, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0xc0) [0x870ce0] 6: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_exception_translator>::process_queued_events()+0xfb) [0x85618b] 7: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x1e) [0x85633e] 8: (ReplicatedPG::snap_trimmer()+0x4f8) [0x7d5ef8] 9: (OSD::SnapTrimWQ::_process(PG*)+0x14) [0x673ab4] 10: (ThreadPool::worker(ThreadPool::WorkThread*)+0x48e) [0xa8fade] 11: (ThreadPool::WorkThread::entry()+0x10) [0xa92870] 12: (()+0x7e9a) [0x7f291fc22e9a] 13: (clone()+0x6d) [0x7f291e5ed31d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. -9993> 2014-11-03 11:37:47.689335 7f28fc814700 1 -- 172.30.5.2:6803/7606 --> 172.30.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x26c59000 con 0x22fbc420 .... -2> 2014-11-03 11:37:57.853585 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.9e4( v 105946'113392 lc 105946'113391 (103622'109598,105946'113392] local-les=1 05948 n=88 ec=25000 les/c 105948/105943 105947/105947/105947) [21,112,33] r=0 lpr=105947 pi=105933-105946/4 crt=105946'113392 lcod 0'0 mlcod 0'0 active+recovery _wait+degraded m=1 snaptrimq=[303~3,307~1]] enter Started/Primary/Active/Recovering -1> 2014-11-03 11:37:57.853735 7f28fc814700 1 -- 172.30.5.2:6803/7606 --> 172.30.5.9:6806/24552 -- MOSDPGPull(24.9e4 105950 [PullOp(5abb99e4/rbd_data.5dd32 f2ae8944a.0000000000000165/head//24, recovery_info: ObjectRecoveryInfo(5abb99e4/rbd_data.5dd32f2ae8944a.0000000000000165/head//24@105946'113392, copy_subset: [0 ~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_c omplete:false))]) v2 -- ?+0 0x229e7e00 con 0x22fb7000 0> 2014-11-03 11:37:57.856578 7f28fc013700 -1 *** Caught signal (Segmentation fault) ** Thanks! -- Tuan HaNoi-VietNam On 11/01/2014 09:21 AM, Ta Ba Tuan wrote: Hi Samuel and Sage, |
2014-11-03 11:37:13.637565 7f29206ff780 0 ceph version 0.87-6-gdba7def (dba7defc623474ad17263c9fccfec60fe7a439f0), process ceph-osd, pid 7606 2014-11-03 11:37:13.656679 7f29206ff780 0 filestore(/var/lib/ceph/osd/cloud-21) backend xfs (magic 0x58465342) 2014-11-03 11:37:13.656710 7f29206ff780 1 filestore(/var/lib/ceph/osd/cloud-21) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-11-03 11:37:13.663531 7f29206ff780 0 genericfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_features: FIEMAP ioctl is supported and appears to work 2014-11-03 11:37:13.663574 7f29206ff780 0 genericfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-11-03 11:37:13.665470 7f29206ff780 0 genericfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_features: syncfs(2) syscall fully supported (by glibc and kernel) 2014-11-03 11:37:13.665705 7f29206ff780 0 xfsfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_feature: extsize is disabled by conf 2014-11-03 11:37:13.814074 7f29206ff780 0 filestore(/var/lib/ceph/osd/cloud-21) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-11-03 11:37:13.840993 7f29206ff780 1 journal _open /dev/sdi1 fd 20: 11998855168 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-11-03 11:37:14.324143 7f29206ff780 1 journal _open /dev/sdi1 fd 20: 11998855168 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-11-03 11:37:14.643739 7f29206ff780 1 journal close /dev/sdi1 2014-11-03 11:37:14.677411 7f29206ff780 0 filestore(/var/lib/ceph/osd/cloud-21) backend xfs (magic 0x58465342) 2014-11-03 11:37:14.679964 7f29206ff780 0 genericfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_features: FIEMAP ioctl is supported and appears to work 2014-11-03 11:37:14.679992 7f29206ff780 0 genericfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-11-03 11:37:14.681943 7f29206ff780 0 genericfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_features: syncfs(2) syscall fully supported (by glibc and kernel) 2014-11-03 11:37:14.682102 7f29206ff780 0 xfsfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_feature: extsize is disabled by conf 2014-11-03 11:37:14.695475 7f29206ff780 0 filestore(/var/lib/ceph/osd/cloud-21) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-11-03 11:37:14.701086 7f29206ff780 1 journal _open /dev/sdi1 fd 20: 11998855168 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-11-03 11:37:14.758904 7f29206ff780 1 journal _open /dev/sdi1 fd 20: 11998855168 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-11-03 11:37:14.863074 7f29206ff780 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello 2014-11-03 11:37:14.890527 7f29206ff780 0 osd.21 105941 crush map has features 1107558400, adjusting msgr requires for clients 2014-11-03 11:37:14.890554 7f29206ff780 0 osd.21 105941 crush map has features 1107558400 was 8705, adjusting msgr requires for mons 2014-11-03 11:37:14.890561 7f29206ff780 0 osd.21 105941 crush map has features 1107558400, adjusting msgr requires for osds 2014-11-03 11:37:14.890589 7f29206ff780 0 osd.21 105941 load_pgs 2014-11-03 11:37:22.444522 7f29206ff780 0 osd.21 105941 load_pgs opened 263 pgs 2014-11-03 11:37:22.452089 7f29206ff780 -1 osd.21 105941 set_disk_tp_priority(22) Invalid argument: osd_disk_thread_ioprio_class is but only the following values are allowed: idle, be or rt 2014-11-03 11:37:22.466686 7f290c834700 0 osd.21 105941 ignoring osdmap until we have initialized 2014-11-03 11:37:22.466800 7f290c834700 0 osd.21 105941 ignoring osdmap until we have initialized 2014-11-03 11:37:22.504203 7f29206ff780 0 osd.21 105941 done with init, starting boot process 2014-11-03 11:37:24.020234 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.021131 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.027579 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.062079 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.082891 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.191453 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.201100 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.218066 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.232587 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.387445 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.399771 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.400978 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.410395 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.551231 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.562410 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.656029 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:24.668334 7f28fb812700 0 osd.21 105947 do_command r=0 2014-11-03 11:37:25.073467 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:25.074153 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:25.097260 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:25.099535 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:25.248894 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:25.259587 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:25.301837 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:25.334632 7f28fb812700 0 osd.21 105948 do_command r=0 2014-11-03 11:37:27.696673 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:27.750294 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:27.863269 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:27.876308 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:28.971667 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:28.982410 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.108889 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.117920 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.145567 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.157324 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.358309 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.368649 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.492932 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.511098 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.748482 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.759886 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.941482 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:29.952893 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.571065 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.575563 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.582026 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.588705 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.657177 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.664878 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.678028 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.688462 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.753807 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.764586 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.975228 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:30.991399 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:31.189885 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:31.203921 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.160808 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.177538 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.493702 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.506313 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.664039 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.681594 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.876135 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.889356 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.954018 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:32.963454 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.036740 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.049165 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.067356 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.079046 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.261434 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.270783 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.426079 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:33.446346 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:34.952833 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:34.977559 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.180571 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.190028 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.192442 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.202026 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.359113 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.370741 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.474420 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.480529 7f28fc013700 0 log_channel(default) log [INF] : 26.568 deep-scrub ok 2014-11-03 11:37:35.488743 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.516605 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.528271 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.596943 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.612053 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.667958 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.684975 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.837306 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:35.855332 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.015629 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.032409 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.361626 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.381756 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.652465 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.666099 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.677406 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.695300 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.857256 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:36.868757 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.069777 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.081535 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.233006 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.241480 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.478662 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.494984 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.673687 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.692126 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.835335 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.846950 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.860605 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.876935 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.985406 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:37.995839 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.007267 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.017112 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.032563 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.045000 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.204248 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.221083 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.383674 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.395431 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.697956 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.712643 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.898975 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:38.913518 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.175055 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.191628 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.365165 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.378413 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.385428 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.395654 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.672562 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:39.688895 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.008468 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.022375 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.175941 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.191293 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.237078 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.246362 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.326073 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.337125 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.353458 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.366170 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.496831 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.507070 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.745189 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.757837 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.903159 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.918106 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.922380 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:40.932253 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.067750 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.079190 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.375464 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.388279 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.526136 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.537351 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.671925 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.690669 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.825874 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.840397 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.978696 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:41.994764 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.449136 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.458471 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.611133 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.618514 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.632783 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.643709 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.690177 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.703218 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.720458 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.727628 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.793534 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:42.806614 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:43.220317 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:43.274007 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:43.690318 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:43.701283 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:43.850361 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:43.866040 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.036167 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.043932 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.116120 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.126249 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.265439 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.275616 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.905802 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.916531 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.961088 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:44.970937 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.056930 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.068721 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.079088 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.089613 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.211203 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.224096 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.460239 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.470623 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.608263 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:45.618978 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.004102 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.021855 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.168220 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.180625 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.312895 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.324370 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.474058 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.487342 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.644664 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.660641 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.808121 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:46.818785 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.174564 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.187346 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.322455 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.331113 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.332268 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.344828 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.441172 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.509073 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.513789 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.520110 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.667421 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.680402 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.814859 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.829752 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.973827 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:47.987246 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.144710 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.157438 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.311719 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.325287 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.464990 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.465694 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.478653 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.479727 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.622856 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.639235 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.842248 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:48.855637 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:49.745978 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:49.760383 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:49.840804 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:49.851379 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:49.875106 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:49.886366 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:51.184610 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:51.198017 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:51.996804 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.009086 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.121071 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.130589 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.163484 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.175252 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.238427 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.248520 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.328388 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.338358 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.502375 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.573820 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.825583 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.832959 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.833915 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.845168 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.981050 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:52.990792 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.117776 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.129512 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.270307 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.280948 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.432689 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.445831 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.572148 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.585508 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.983765 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:53.998823 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.125124 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.135455 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.171496 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.180003 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.275437 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.286341 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.461854 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.471835 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.535040 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.542780 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.605534 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.617876 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.649008 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.659767 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.762080 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.775629 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.928387 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:54.939780 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.201568 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.211699 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.360186 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.368743 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.500507 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.508473 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.565173 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.581056 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.711988 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.730451 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.880974 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:55.895068 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.040074 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.050934 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.350013 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.366344 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.518384 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.535928 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.778534 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.792558 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.895951 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.909810 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.922082 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:56.933294 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.004371 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.015040 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.016021 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.026596 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.082463 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.099960 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.244229 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.258871 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.418928 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.432773 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.573473 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.584631 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.720579 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.730820 7f28fb812700 0 osd.21 105950 do_command r=0 2014-11-03 11:37:57.856578 7f28fc013700 -1 *** Caught signal (Segmentation fault) ** -10000> 2014-11-03 11:37:47.688638 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x4c792c0 session 0x32be000 -9999> 2014-11-03 11:37:47.688624 7f291450b700 5 -- op tracker -- seq: 2747, time: 2014-11-03 11:37:47.688624, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254333 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953123 snapset=0=[]:[] snapc=0=[]) -9998> 2014-11-03 11:37:47.688670 7f291450b700 5 -- op tracker -- seq: 2748, time: 2014-11-03 11:37:47.688669, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197362 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195954 snapset=0=[]:[] snapc=0=[]) -9997> 2014-11-03 11:37:47.688729 7f2911d06700 5 -- op tracker -- seq: 2747, time: 2014-11-03 11:37:47.688728, event: commit_sent, op: osd_sub_op(client.8703713.0:32254333 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953123 snapset=0=[]:[] snapc=0=[]) -9996> 2014-11-03 11:37:47.688764 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254333 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23c1d8c0 con 0x22fefdc0 -9995> 2014-11-03 11:37:47.688832 7f2911d06700 5 -- op tracker -- seq: 2748, time: 2014-11-03 11:37:47.688832, event: commit_sent, op: osd_sub_op(client.7867942.0:137197362 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195954 snapset=0=[]:[] snapc=0=[]) -9994> 2014-11-03 11:37:47.688881 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867942.0:137197362 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28b17340 con 0x22fbc840 -9993> 2014-11-03 11:37:47.689335 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x26c59000 con 0x22fbc420 -9992> 2014-11-03 11:37:47.689591 7f2912507700 5 -- op tracker -- seq: 2747, time: 2014-11-03 11:37:47.689591, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254333 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953123 snapset=0=[]:[] snapc=0=[]) -9991> 2014-11-03 11:37:47.689642 7f2912507700 5 -- op tracker -- seq: 2747, time: 2014-11-03 11:37:47.689642, event: done, op: osd_sub_op(client.8703713.0:32254333 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953123 snapset=0=[]:[] snapc=0=[]) -9990> 2014-11-03 11:37:47.689687 7f2912507700 5 -- op tracker -- seq: 2748, time: 2014-11-03 11:37:47.689687, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197362 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195954 snapset=0=[]:[] snapc=0=[]) -9989> 2014-11-03 11:37:47.689713 7f2912507700 5 -- op tracker -- seq: 2748, time: 2014-11-03 11:37:47.689713, event: done, op: osd_sub_op(client.7867942.0:137197362 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195954 snapset=0=[]:[] snapc=0=[]) -9988> 2014-11-03 11:37:47.714319 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 42 ==== osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (911917462 0 3511520646) 0x2ce3b000 con 0x22fbc000 -9987> 2014-11-03 11:37:47.714357 7f28f71fd700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714184, event: header_read, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9986> 2014-11-03 11:37:47.714373 7f28f71fd700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714196, event: throttled, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9985> 2014-11-03 11:37:47.714382 7f28f71fd700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714304, event: all_read, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9984> 2014-11-03 11:37:47.714393 7f28f71fd700 5 -- op tracker -- seq: 2749, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9983> 2014-11-03 11:37:47.714480 7f290001b700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714480, event: reached_pg, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9982> 2014-11-03 11:37:47.714525 7f290001b700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714525, event: started, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9981> 2014-11-03 11:37:47.714628 7f290001b700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714628, event: started, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9980> 2014-11-03 11:37:47.714676 7f290001b700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714675, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9979> 2014-11-03 11:37:47.714756 7f2914d0c700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.714756, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9978> 2014-11-03 11:37:47.738804 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 103 ==== osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3018737390 0 3511520646) 0x27072400 con 0x22fb76e0 -9977> 2014-11-03 11:37:47.738867 7f28edd69700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.738642, event: header_read, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9976> 2014-11-03 11:37:47.738891 7f28edd69700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.738645, event: throttled, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9975> 2014-11-03 11:37:47.738913 7f28edd69700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.738786, event: all_read, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9974> 2014-11-03 11:37:47.738930 7f28edd69700 5 -- op tracker -- seq: 2750, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9973> 2014-11-03 11:37:47.739002 7f290101d700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.739002, event: reached_pg, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9972> 2014-11-03 11:37:47.739045 7f290101d700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.739045, event: started, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9971> 2014-11-03 11:37:47.739217 7f290101d700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.739217, event: started, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9970> 2014-11-03 11:37:47.739249 7f290101d700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.739249, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9969> 2014-11-03 11:37:47.739458 7f291450b700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.739457, event: journaled_completion_queued, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9968> 2014-11-03 11:37:47.739542 7f2914d0c700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.739542, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9967> 2014-11-03 11:37:47.739565 7f2911d06700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.739565, event: commit_sent, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9966> 2014-11-03 11:37:47.739601 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] ondisk, result = 0) v2 -- ?+0 0x191b3b00 con 0x22fbc000 -9965> 2014-11-03 11:37:47.739875 7f2912507700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.739875, event: sub_op_applied, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9964> 2014-11-03 11:37:47.739930 7f2912507700 5 -- op tracker -- seq: 2749, time: 2014-11-03 11:37:47.739930, event: done, op: osd_sub_op(client.7868762.0:225662 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100459 snapset=0=[]:[] snapc=0=[]) -9963> 2014-11-03 11:37:47.740166 7f291450b700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.740166, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9962> 2014-11-03 11:37:47.740207 7f2911d06700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.740207, event: commit_sent, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9961> 2014-11-03 11:37:47.740249 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0x191b3840 con 0x22fb76e0 -9960> 2014-11-03 11:37:47.740551 7f2912507700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.740551, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9959> 2014-11-03 11:37:47.740621 7f2912507700 5 -- op tracker -- seq: 2750, time: 2014-11-03 11:37:47.740621, event: done, op: osd_sub_op(client.12404444.0:39562 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374236 snapset=0=[]:[] snapc=0=[]) -9958> 2014-11-03 11:37:47.743719 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 43 ==== osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (146939116 0 3511520646) 0x26942c00 con 0x22fbc000 -9957> 2014-11-03 11:37:47.743753 7f28f71fd700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.743586, event: header_read, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9956> 2014-11-03 11:37:47.743778 7f28f71fd700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.743587, event: throttled, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9955> 2014-11-03 11:37:47.743795 7f28f71fd700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.743706, event: all_read, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9954> 2014-11-03 11:37:47.743815 7f28f71fd700 5 -- op tracker -- seq: 2751, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9953> 2014-11-03 11:37:47.743870 7f28fd816700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.743869, event: reached_pg, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9952> 2014-11-03 11:37:47.743896 7f28fd816700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.743896, event: started, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9951> 2014-11-03 11:37:47.743979 7f28fd816700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.743979, event: started, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9950> 2014-11-03 11:37:47.744007 7f28fd816700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.744007, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9949> 2014-11-03 11:37:47.744061 7f2914d0c700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.744060, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9948> 2014-11-03 11:37:47.745378 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 76 ==== osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1791056127 0 3511520646) 0x25ed6400 con 0x22ff3b00 -9947> 2014-11-03 11:37:47.745411 7f28f0d99700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.745269, event: header_read, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9946> 2014-11-03 11:37:47.745429 7f28f0d99700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.745272, event: throttled, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9945> 2014-11-03 11:37:47.745439 7f28f0d99700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.745363, event: all_read, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9944> 2014-11-03 11:37:47.745451 7f28f0d99700 5 -- op tracker -- seq: 2752, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9943> 2014-11-03 11:37:47.745504 7f290001b700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.745504, event: reached_pg, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9942> 2014-11-03 11:37:47.745534 7f290001b700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.745534, event: started, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9941> 2014-11-03 11:37:47.745618 7f290001b700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.745617, event: started, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9940> 2014-11-03 11:37:47.745644 7f290001b700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.745644, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9939> 2014-11-03 11:37:47.772688 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 104 ==== osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1046923561 0 3511520646) 0x23c3ec00 con 0x22fb76e0 -9938> 2014-11-03 11:37:47.772733 7f28edd69700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.772560, event: header_read, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9937> 2014-11-03 11:37:47.772776 7f28edd69700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.772561, event: throttled, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9936> 2014-11-03 11:37:47.772795 7f28edd69700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.772676, event: all_read, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9935> 2014-11-03 11:37:47.772814 7f28edd69700 5 -- op tracker -- seq: 2753, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9934> 2014-11-03 11:37:47.772927 7f28fe818700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.772927, event: reached_pg, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9933> 2014-11-03 11:37:47.772978 7f28fe818700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.772978, event: started, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9932> 2014-11-03 11:37:47.773127 7f28fe818700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.773127, event: started, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9931> 2014-11-03 11:37:47.773183 7f28fe818700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.773183, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9930> 2014-11-03 11:37:47.773439 7f28f43cf700 1 -- 192.168.5.2:6803/7606 <== osd.79 192.168.5.6:6851/12953 54 ==== osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (2300172249 0 3511520646) 0x191b6800 con 0x22fef6e0 -9929> 2014-11-03 11:37:47.773502 7f28f43cf700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.773247, event: header_read, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9928> 2014-11-03 11:37:47.773551 7f28f43cf700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.773251, event: throttled, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9927> 2014-11-03 11:37:47.773575 7f28f43cf700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.773417, event: all_read, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9926> 2014-11-03 11:37:47.773614 7f28f43cf700 5 -- op tracker -- seq: 2754, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9925> 2014-11-03 11:37:47.773705 7f290181e700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.773704, event: reached_pg, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9924> 2014-11-03 11:37:47.773769 7f290181e700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.773768, event: started, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9923> 2014-11-03 11:37:47.773953 7f290181e700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.773953, event: started, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9922> 2014-11-03 11:37:47.773993 7f290181e700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.773993, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9921> 2014-11-03 11:37:47.786806 7f291450b700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.786806, event: journaled_completion_queued, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9920> 2014-11-03 11:37:47.786856 7f2911d06700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.786855, event: commit_sent, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9919> 2014-11-03 11:37:47.786918 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294adb00 con 0x22fbc000 -9918> 2014-11-03 11:37:47.786927 7f2914d0c700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.786927, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9917> 2014-11-03 11:37:47.786985 7f2914d0c700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.786985, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9916> 2014-11-03 11:37:47.787040 7f2914d0c700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.787040, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9915> 2014-11-03 11:37:47.787191 7f2912507700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.787191, event: sub_op_applied, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9914> 2014-11-03 11:37:47.787271 7f2912507700 5 -- op tracker -- seq: 2751, time: 2014-11-03 11:37:47.787271, event: done, op: osd_sub_op(client.7868762.0:225663 6.4f0 4b5f44f0/rbd_data.662b8b2ae8944a.0000000000000229/head//6 [] v 105950'13100460 snapset=0=[]:[] snapc=0=[]) -9913> 2014-11-03 11:37:47.790183 7f291450b700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.790183, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9912> 2014-11-03 11:37:47.790232 7f291450b700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.790232, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9911> 2014-11-03 11:37:47.790243 7f2911d06700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.790243, event: commit_sent, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9910> 2014-11-03 11:37:47.790260 7f291450b700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.790260, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9909> 2014-11-03 11:37:47.790291 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x25591580 con 0x22ff3b00 -9908> 2014-11-03 11:37:47.790328 7f2911d06700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.790327, event: commit_sent, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9907> 2014-11-03 11:37:47.790351 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0x23c1c840 con 0x22fb76e0 -9906> 2014-11-03 11:37:47.790393 7f2911d06700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.790393, event: commit_sent, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9905> 2014-11-03 11:37:47.790413 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6851/12953 -- osd_sub_op_reply(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1c580 con 0x22fef6e0 -9904> 2014-11-03 11:37:47.790802 7f2912507700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.790802, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9903> 2014-11-03 11:37:47.790841 7f2912507700 5 -- op tracker -- seq: 2752, time: 2014-11-03 11:37:47.790841, event: done, op: osd_sub_op(client.12146205.0:439291 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222061 snapset=0=[]:[] snapc=0=[]) -9902> 2014-11-03 11:37:47.790878 7f2912507700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.790878, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9901> 2014-11-03 11:37:47.790912 7f2912507700 5 -- op tracker -- seq: 2753, time: 2014-11-03 11:37:47.790912, event: done, op: osd_sub_op(client.12404444.0:39568 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374237 snapset=0=[]:[] snapc=0=[]) -9900> 2014-11-03 11:37:47.790936 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 82 ==== MOSDPGPush(6.58e 105950 [PushOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, version: 105938'11622009, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1698164618 0 0) 0x26c59000 con 0x22fbc420 -9899> 2014-11-03 11:37:47.791000 7f28f72fe700 5 -- op tracker -- seq: 2755, time: 2014-11-03 11:37:47.781292, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, version: 105938'11622009, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9898> 2014-11-03 11:37:47.791033 7f28f72fe700 5 -- op tracker -- seq: 2755, time: 2014-11-03 11:37:47.781294, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, version: 105938'11622009, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9897> 2014-11-03 11:37:47.791075 7f28f72fe700 5 -- op tracker -- seq: 2755, time: 2014-11-03 11:37:47.790918, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, version: 105938'11622009, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9896> 2014-11-03 11:37:47.791094 7f2912507700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.791094, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9895> 2014-11-03 11:37:47.791103 7f28f72fe700 5 -- op tracker -- seq: 2755, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, version: 105938'11622009, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9894> 2014-11-03 11:37:47.791118 7f2912507700 5 -- op tracker -- seq: 2754, time: 2014-11-03 11:37:47.791118, event: done, op: osd_sub_op(client.7867942.0:137197627 6.83 e5c09083/rbd_data.43c40916a10.0000000000000745/head//6 [] v 105950'11342778 snapset=0=[]:[] snapc=0=[]) -9893> 2014-11-03 11:37:47.791161 7f290101d700 5 -- op tracker -- seq: 2755, time: 2014-11-03 11:37:47.791161, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, version: 105938'11622009, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9892> 2014-11-03 11:37:47.791404 7f290101d700 5 -- op tracker -- seq: 2755, time: 2014-11-03 11:37:47.791403, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6, version: 105938'11622009, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87f82d8e/rbd_data.45e62779c99cf1.00000000000022b5/head//6@105938'11622009, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9891> 2014-11-03 11:37:47.799086 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.60 192.168.5.5:0/5983 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:47.797477) v2 ==== 47+0+0 (2512343275 0 0) 0x2333fa00 con 0x23ea4b00 -9890> 2014-11-03 11:37:47.799140 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:47.797477) v2 -- ?+0 0x2a347200 con 0x23ea4b00 -9889> 2014-11-03 11:37:47.799197 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.60 192.168.5.5:0/5983 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:47.797477) v2 ==== 47+0+0 (2512343275 0 0) 0x226b5200 con 0x2445c9a0 -9888> 2014-11-03 11:37:47.799233 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:47.797477) v2 -- ?+0 0x23354c00 con 0x2445c9a0 -9887> 2014-11-03 11:37:47.803925 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x29fcde00 con 0x231abb00 -9886> 2014-11-03 11:37:47.813361 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x27803840 sd=62 :6802 s=0 pgs=0 cs=0 l=0 c=0x4c799a0).accept sd=62 192.168.5.2:42800/0 -9885> 2014-11-03 11:37:47.814032 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450632 192.168.5.2:0/1023924 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2369d800 con 0x4c799a0 -9884> 2014-11-03 11:37:47.814859 7f28fb812700 0 osd.21 105950 do_command r=0 -9883> 2014-11-03 11:37:47.814892 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1023924 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xccda380 con 0x4c799a0 -9882> 2014-11-03 11:37:47.828846 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450632 192.168.5.2:0/1023924 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x187f4000 con 0x4c799a0 -9881> 2014-11-03 11:37:47.829752 7f28fb812700 0 osd.21 105950 do_command r=0 -9880> 2014-11-03 11:37:47.829771 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1023924 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x7835540 con 0x4c799a0 -9879> 2014-11-03 11:37:47.834913 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.28 192.168.5.3:0/13976 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:47.834288) v2 ==== 47+0+0 (123079307 0 0) 0x2d361c00 con 0x23b1cb00 -9878> 2014-11-03 11:37:47.834967 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/13976 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:47.834288) v2 -- ?+0 0x226b5200 con 0x23b1cb00 -9877> 2014-11-03 11:37:47.835025 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.28 192.168.5.3:0/13976 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:47.834288) v2 ==== 47+0+0 (123079307 0 0) 0x253e5e00 con 0x242c0c60 -9876> 2014-11-03 11:37:47.835084 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/13976 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:47.834288) v2 -- ?+0 0x2333fa00 con 0x242c0c60 -9875> 2014-11-03 11:37:47.836743 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1023924 pipe(0x27803840 sd=62 :6802 s=2 pgs=2 cs=1 l=1 c=0x4c799a0).reader couldn't read tag, (0) Success -9874> 2014-11-03 11:37:47.836818 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1023924 pipe(0x27803840 sd=62 :6802 s=2 pgs=2 cs=1 l=1 c=0x4c799a0).fault (0) Success -9873> 2014-11-03 11:37:47.836953 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x4c799a0 session 0x32be000 -9872> 2014-11-03 11:37:47.855980 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 60 ==== osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (963426550 0 3511520646) 0x26151c00 con 0x22fbc840 -9871> 2014-11-03 11:37:47.856036 7f28f68f4700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.855775, event: header_read, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9870> 2014-11-03 11:37:47.856056 7f28f68f4700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.855778, event: throttled, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9869> 2014-11-03 11:37:47.856069 7f28f68f4700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.855925, event: all_read, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9868> 2014-11-03 11:37:47.856081 7f28f68f4700 5 -- op tracker -- seq: 2756, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9867> 2014-11-03 11:37:47.856171 7f28fe017700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.856171, event: reached_pg, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9866> 2014-11-03 11:37:47.856208 7f28fe017700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.856208, event: started, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9865> 2014-11-03 11:37:47.856486 7f28fe017700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.856486, event: started, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9864> 2014-11-03 11:37:47.856590 7f28fe017700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.856589, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9863> 2014-11-03 11:37:47.856713 7f2914d0c700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:47.856712, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9862> 2014-11-03 11:37:47.860592 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 61 ==== osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1520295176 0 3511520646) 0x25eb9400 con 0x22fbc840 -9861> 2014-11-03 11:37:47.860628 7f28f68f4700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:47.860501, event: header_read, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9860> 2014-11-03 11:37:47.860655 7f28f68f4700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:47.860502, event: throttled, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9859> 2014-11-03 11:37:47.860675 7f28f68f4700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:47.860580, event: all_read, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9858> 2014-11-03 11:37:47.860693 7f28f68f4700 5 -- op tracker -- seq: 2757, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9857> 2014-11-03 11:37:47.860763 7f290081c700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:47.860763, event: reached_pg, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9856> 2014-11-03 11:37:47.860790 7f290081c700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:47.860790, event: started, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9855> 2014-11-03 11:37:47.860922 7f290081c700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:47.860922, event: started, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9854> 2014-11-03 11:37:47.860961 7f290081c700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:47.860961, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9853> 2014-11-03 11:37:47.897179 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.76 192.168.5.6:0/22493 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:47.896654) v2 ==== 47+0+0 (344105731 0 0) 0x8025a00 con 0x23b1c160 -9852> 2014-11-03 11:37:47.897254 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:47.896654) v2 -- ?+0 0x2d361c00 con 0x23b1c160 -9851> 2014-11-03 11:37:47.897520 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.76 192.168.5.6:0/22493 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:47.896654) v2 ==== 47+0+0 (344105731 0 0) 0x852f000 con 0x24229dc0 -9850> 2014-11-03 11:37:47.897564 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:47.896654) v2 -- ?+0 0x253e5e00 con 0x24229dc0 -9849> 2014-11-03 11:37:47.914790 7f28e9326700 1 -- 192.168.5.2:6802/7606 <== client.9598152 192.168.9.2:0/1027858 9 ==== osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) v4 ==== 229+0+20480 (2512679909 0 767690401) 0x23108c80 con 0x2311c9a0 -9848> 2014-11-03 11:37:47.914832 7f28e9326700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.914618, event: header_read, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9847> 2014-11-03 11:37:47.914848 7f28e9326700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.914622, event: throttled, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9846> 2014-11-03 11:37:47.914857 7f28e9326700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.914776, event: all_read, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9845> 2014-11-03 11:37:47.914865 7f28e9326700 5 -- op tracker -- seq: 2758, time: 0.000000, event: dispatched, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9844> 2014-11-03 11:37:47.914987 7f28fe818700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.914986, event: reached_pg, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9843> 2014-11-03 11:37:47.915153 7f28fe818700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.915153, event: started, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9842> 2014-11-03 11:37:47.915194 7f28fe818700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.915194, event: started, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9841> 2014-11-03 11:37:47.915270 7f28fe818700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.915270, event: waiting for subops from 45,107, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9840> 2014-11-03 11:37:47.915330 7f28fe818700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131093 snapset=0=[]:[] snapc=0=[]) v11 -- ?+21309 0x2dcb1000 con 0x22ff3420 -9839> 2014-11-03 11:37:47.915416 7f28fe818700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6833/17474 -- osd_sub_op(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131093 snapset=0=[]:[] snapc=0=[]) v11 -- ?+21309 0x2dcb1c00 con 0x22fe0dc0 -9838> 2014-11-03 11:37:47.915565 7f28fe818700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.915565, event: commit_queued_for_journal_write, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9837> 2014-11-03 11:37:47.916471 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 141 ==== osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2743345564 0 3511520646) 0x263ccc00 con 0x22fefdc0 -9836> 2014-11-03 11:37:47.916498 7f28f36c2700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:47.916292, event: header_read, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9835> 2014-11-03 11:37:47.916510 7f28f36c2700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:47.916294, event: throttled, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9834> 2014-11-03 11:37:47.916519 7f28f36c2700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:47.916458, event: all_read, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9833> 2014-11-03 11:37:47.916527 7f28f36c2700 5 -- op tracker -- seq: 2759, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9832> 2014-11-03 11:37:47.916581 7f290101d700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:47.916580, event: reached_pg, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9831> 2014-11-03 11:37:47.916610 7f290101d700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:47.916610, event: started, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9830> 2014-11-03 11:37:47.916701 7f290101d700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:47.916701, event: started, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9829> 2014-11-03 11:37:47.916727 7f290101d700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:47.916727, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9828> 2014-11-03 11:37:47.917146 7f28f52de700 1 -- 192.168.5.2:6803/7606 <== osd.107 192.168.5.8:6833/17474 32 ==== osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (220985277 0 0) 0x28a7c000 con 0x22fe0dc0 -9827> 2014-11-03 11:37:47.917174 7f28f52de700 5 -- op tracker -- seq: 2760, time: 2014-11-03 11:37:47.917079, event: header_read, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9826> 2014-11-03 11:37:47.917232 7f28f52de700 5 -- op tracker -- seq: 2760, time: 2014-11-03 11:37:47.917081, event: throttled, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9825> 2014-11-03 11:37:47.917249 7f28f52de700 5 -- op tracker -- seq: 2760, time: 2014-11-03 11:37:47.917134, event: all_read, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9824> 2014-11-03 11:37:47.917257 7f28f52de700 5 -- op tracker -- seq: 2760, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9823> 2014-11-03 11:37:47.917319 7f28fe818700 5 -- op tracker -- seq: 2760, time: 2014-11-03 11:37:47.917318, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9822> 2014-11-03 11:37:47.917341 7f28fe818700 5 -- op tracker -- seq: 2760, time: 2014-11-03 11:37:47.917341, event: started, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9821> 2014-11-03 11:37:47.917355 7f28fe818700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.917355, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9820> 2014-11-03 11:37:47.917375 7f28fe818700 5 -- op tracker -- seq: 2760, time: 2014-11-03 11:37:47.917375, event: done, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9819> 2014-11-03 11:37:47.917680 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 30 ==== osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (1784036322 0 0) 0x29ce3b80 con 0x22ff3420 -9818> 2014-11-03 11:37:47.917728 7f28f0b97700 5 -- op tracker -- seq: 2761, time: 2014-11-03 11:37:47.917574, event: header_read, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9817> 2014-11-03 11:37:47.917756 7f28f0b97700 5 -- op tracker -- seq: 2761, time: 2014-11-03 11:37:47.917578, event: throttled, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9816> 2014-11-03 11:37:47.917777 7f28f0b97700 5 -- op tracker -- seq: 2761, time: 2014-11-03 11:37:47.917663, event: all_read, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9815> 2014-11-03 11:37:47.917793 7f28f0b97700 5 -- op tracker -- seq: 2761, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9814> 2014-11-03 11:37:47.917902 7f290101d700 5 -- op tracker -- seq: 2761, time: 2014-11-03 11:37:47.917902, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9813> 2014-11-03 11:37:47.917932 7f290101d700 5 -- op tracker -- seq: 2761, time: 2014-11-03 11:37:47.917932, event: started, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9812> 2014-11-03 11:37:47.917948 7f290101d700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:47.917948, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9811> 2014-11-03 11:37:47.917967 7f290101d700 5 -- op tracker -- seq: 2761, time: 2014-11-03 11:37:47.917967, event: done, op: osd_sub_op_reply(client.9598152.0:459473 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -9810> 2014-11-03 11:37:47.939122 7f28f4ad6700 1 -- 192.168.5.2:6803/7606 <== osd.89 192.168.5.7:6812/24271 41 ==== osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1202+0+0 (2770770704 0 0) 0x2d576400 con 0x22fe06e0 -9809> 2014-11-03 11:37:47.939192 7f28f4ad6700 5 -- op tracker -- seq: 2762, time: 2014-11-03 11:37:47.938928, event: header_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9808> 2014-11-03 11:37:47.939224 7f28f4ad6700 5 -- op tracker -- seq: 2762, time: 2014-11-03 11:37:47.938933, event: throttled, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9807> 2014-11-03 11:37:47.939276 7f28f4ad6700 5 -- op tracker -- seq: 2762, time: 2014-11-03 11:37:47.939095, event: all_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9806> 2014-11-03 11:37:47.939309 7f28f4ad6700 5 -- op tracker -- seq: 2762, time: 0.000000, event: dispatched, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9805> 2014-11-03 11:37:47.939419 7f28fd816700 5 -- op tracker -- seq: 2762, time: 2014-11-03 11:37:47.939418, event: reached_pg, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9804> 2014-11-03 11:37:47.939478 7f28fd816700 5 -- op tracker -- seq: 2762, time: 2014-11-03 11:37:47.939478, event: started, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9803> 2014-11-03 11:37:47.939502 7f28fd816700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6812/24271 -- osd_sub_op_reply(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x279fc600 con 0x22fe06e0 -9802> 2014-11-03 11:37:47.939556 7f28fd816700 5 -- op tracker -- seq: 2762, time: 2014-11-03 11:37:47.939556, event: done, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9801> 2014-11-03 11:37:47.940721 7f28f4ad6700 1 -- 192.168.5.2:6803/7606 <== osd.89 192.168.5.7:6812/24271 42 ==== osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1202+0+0 (2566622843 0 0) 0x2d574c00 con 0x22fe06e0 -9800> 2014-11-03 11:37:47.940755 7f28f4ad6700 5 -- op tracker -- seq: 2763, time: 2014-11-03 11:37:47.940649, event: header_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9799> 2014-11-03 11:37:47.940780 7f28f4ad6700 5 -- op tracker -- seq: 2763, time: 2014-11-03 11:37:47.940651, event: throttled, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9798> 2014-11-03 11:37:47.940791 7f28f4ad6700 5 -- op tracker -- seq: 2763, time: 2014-11-03 11:37:47.940711, event: all_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9797> 2014-11-03 11:37:47.940802 7f28f4ad6700 5 -- op tracker -- seq: 2763, time: 0.000000, event: dispatched, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9796> 2014-11-03 11:37:47.940897 7f290001b700 5 -- op tracker -- seq: 2763, time: 2014-11-03 11:37:47.940897, event: reached_pg, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9795> 2014-11-03 11:37:47.940979 7f290001b700 5 -- op tracker -- seq: 2763, time: 2014-11-03 11:37:47.940979, event: started, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9794> 2014-11-03 11:37:47.941013 7f290001b700 5 -- op tracker -- seq: 2763, time: 2014-11-03 11:37:47.941013, event: done, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -9793> 2014-11-03 11:37:47.946237 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 24 ==== MOSDPGPush(6.58e 105950 [PushOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, version: 105945'11622047, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1654160972 0 0) 0x29fcde00 con 0x231abb00 -9792> 2014-11-03 11:37:47.946288 7f28eb945700 5 -- op tracker -- seq: 2764, time: 2014-11-03 11:37:47.934759, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, version: 105945'11622047, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9791> 2014-11-03 11:37:47.946347 7f28eb945700 5 -- op tracker -- seq: 2764, time: 2014-11-03 11:37:47.934761, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, version: 105945'11622047, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9790> 2014-11-03 11:37:47.946393 7f28eb945700 5 -- op tracker -- seq: 2764, time: 2014-11-03 11:37:47.946222, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, version: 105945'11622047, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9789> 2014-11-03 11:37:47.946438 7f28eb945700 5 -- op tracker -- seq: 2764, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, version: 105945'11622047, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9788> 2014-11-03 11:37:47.946591 7f28fe818700 5 -- op tracker -- seq: 2764, time: 2014-11-03 11:37:47.946591, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, version: 105945'11622047, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9787> 2014-11-03 11:37:47.946866 7f28fe818700 5 -- op tracker -- seq: 2764, time: 2014-11-03 11:37:47.946866, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6, version: 105945'11622047, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(52f06d8e/rbd_data.8520512ae8944a.00000000000000af/head//6@105945'11622047, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9786> 2014-11-03 11:37:47.972061 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x27804340 sd=62 :6802 s=0 pgs=0 cs=0 l=0 c=0x4c79580).accept sd=62 192.168.5.2:42806/0 -9785> 2014-11-03 11:37:47.972981 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450778 192.168.5.2:0/1025207 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x252d7800 con 0x4c79580 -9784> 2014-11-03 11:37:47.973827 7f28fb812700 0 osd.21 105950 do_command r=0 -9783> 2014-11-03 11:37:47.973870 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025207 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f380 con 0x4c79580 -9782> 2014-11-03 11:37:47.986261 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450778 192.168.5.2:0/1025207 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x852f600 con 0x4c79580 -9781> 2014-11-03 11:37:47.987246 7f28fb812700 0 osd.21 105950 do_command r=0 -9780> 2014-11-03 11:37:47.987266 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025207 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x936ae00 con 0x4c79580 -9779> 2014-11-03 11:37:47.995196 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025207 pipe(0x27804340 sd=62 :6802 s=2 pgs=2 cs=1 l=1 c=0x4c79580).reader couldn't read tag, (0) Success -9778> 2014-11-03 11:37:47.995281 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025207 pipe(0x27804340 sd=62 :6802 s=2 pgs=2 cs=1 l=1 c=0x4c79580).fault (0) Success -9777> 2014-11-03 11:37:47.995431 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x4c79580 session 0x32be000 -9776> 2014-11-03 11:37:48.049218 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.77 192.168.5.6:0/32590 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.048115) v2 ==== 47+0+0 (1685268463 0 0) 0x23408a00 con 0x2484a420 -9775> 2014-11-03 11:37:48.049273 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/32590 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.048115) v2 -- ?+0 0x8025a00 con 0x2484a420 -9774> 2014-11-03 11:37:48.049266 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.77 192.168.5.6:0/32590 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.048115) v2 ==== 47+0+0 (1685268463 0 0) 0x1a327200 con 0x24205840 -9773> 2014-11-03 11:37:48.049323 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/32590 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.048115) v2 -- ?+0 0x852f000 con 0x24205840 -9772> 2014-11-03 11:37:48.078490 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 65 ==== osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1601794557 0 3511520646) 0x29e1f400 con 0x2308b2c0 -9771> 2014-11-03 11:37:48.078551 7f28eed79700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.078261, event: header_read, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9770> 2014-11-03 11:37:48.078590 7f28eed79700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.078265, event: throttled, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9769> 2014-11-03 11:37:48.078619 7f28eed79700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.078437, event: all_read, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9768> 2014-11-03 11:37:48.078633 7f28eed79700 5 -- op tracker -- seq: 2765, time: 0.000000, event: dispatched, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9767> 2014-11-03 11:37:48.078736 7f290201f700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.078736, event: reached_pg, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9766> 2014-11-03 11:37:48.078784 7f290201f700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.078784, event: started, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9765> 2014-11-03 11:37:48.078908 7f290201f700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.078908, event: started, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9764> 2014-11-03 11:37:48.078955 7f290201f700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.078955, event: commit_queued_for_journal_write, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9763> 2014-11-03 11:37:48.085100 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.35 192.168.5.3:0/6455 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.084330) v2 ==== 47+0+0 (1218758805 0 0) 0x1106c200 con 0x2484a6e0 -9762> 2014-11-03 11:37:48.085170 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.084330) v2 -- ?+0 0x23408a00 con 0x2484a6e0 -9761> 2014-11-03 11:37:48.085237 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.35 192.168.5.3:0/6455 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.084330) v2 ==== 47+0+0 (1218758805 0 0) 0x6352c00 con 0x24321160 -9760> 2014-11-03 11:37:48.085287 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.084330) v2 -- ?+0 0x1a327200 con 0x24321160 -9759> 2014-11-03 11:37:48.094203 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 69 ==== osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3587756711 0 3511520646) 0x24bec800 con 0x2308b840 -9758> 2014-11-03 11:37:48.094245 7f28ef985700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.094070, event: header_read, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9757> 2014-11-03 11:37:48.094259 7f28ef985700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.094073, event: throttled, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9756> 2014-11-03 11:37:48.094268 7f28ef985700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.094187, event: all_read, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9755> 2014-11-03 11:37:48.094276 7f28ef985700 5 -- op tracker -- seq: 2766, time: 0.000000, event: dispatched, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9754> 2014-11-03 11:37:48.094347 7f28fe017700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.094347, event: reached_pg, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9753> 2014-11-03 11:37:48.094385 7f28fe017700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.094385, event: started, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9752> 2014-11-03 11:37:48.094522 7f28fe017700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.094522, event: started, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9751> 2014-11-03 11:37:48.094576 7f28fe017700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.094576, event: commit_queued_for_journal_write, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9750> 2014-11-03 11:37:48.101693 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.85 192.168.5.7:0/27520 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.099863) v2 ==== 47+0+0 (318091474 0 0) 0x234dcc00 con 0x233cb6e0 -9749> 2014-11-03 11:37:48.101710 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.85 192.168.5.7:0/27520 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.099863) v2 ==== 47+0+0 (318091474 0 0) 0x22a08a00 con 0x233cb2c0 -9748> 2014-11-03 11:37:48.101738 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/27520 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.099863) v2 -- ?+0 0x6352c00 con 0x233cb6e0 -9747> 2014-11-03 11:37:48.101798 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/27520 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.099863) v2 -- ?+0 0x1106c200 con 0x233cb2c0 -9746> 2014-11-03 11:37:48.113695 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.81 192.168.5.6:0/27170 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.112324) v2 ==== 47+0+0 (3318641525 0 0) 0x26bdb400 con 0x24229840 -9745> 2014-11-03 11:37:48.113715 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.81 192.168.5.6:0/27170 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.112324) v2 ==== 47+0+0 (3318641525 0 0) 0x2470a800 con 0x24925b00 -9744> 2014-11-03 11:37:48.113754 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.112324) v2 -- ?+0 0x234dcc00 con 0x24229840 -9743> 2014-11-03 11:37:48.113940 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.112324) v2 -- ?+0 0x22a08a00 con 0x24925b00 -9742> 2014-11-03 11:37:48.123144 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.104 192.168.5.8:0/2576 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.122068) v2 ==== 47+0+0 (2821737035 0 0) 0x2796e400 con 0x239f8000 -9741> 2014-11-03 11:37:48.123137 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.104 192.168.5.8:0/2576 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.122068) v2 ==== 47+0+0 (2821737035 0 0) 0x2369da00 con 0x239f8dc0 -9740> 2014-11-03 11:37:48.123187 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.122068) v2 -- ?+0 0x2470a800 con 0x239f8000 -9739> 2014-11-03 11:37:48.123246 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.122068) v2 -- ?+0 0x26bdb400 con 0x239f8dc0 -9738> 2014-11-03 11:37:48.132132 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.64 192.168.5.5:0/24829 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.130764) v2 ==== 47+0+0 (1445566057 0 0) 0xa5d2e00 con 0x23b1c2c0 -9737> 2014-11-03 11:37:48.132175 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/24829 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.130764) v2 -- ?+0 0x2796e400 con 0x23b1c2c0 -9736> 2014-11-03 11:37:48.132199 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.64 192.168.5.5:0/24829 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.130764) v2 ==== 47+0+0 (1445566057 0 0) 0x31a6e00 con 0x161ffdc0 -9735> 2014-11-03 11:37:48.132257 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/24829 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.130764) v2 -- ?+0 0x2369da00 con 0x161ffdc0 -9734> 2014-11-03 11:37:48.143171 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x278032c0 sd=205 :6802 s=0 pgs=0 cs=0 l=0 c=0x4c79420).accept sd=205 192.168.5.2:42812/0 -9733> 2014-11-03 11:37:48.143943 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450638 192.168.5.2:0/1025488 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x242e2e00 con 0x4c79420 -9732> 2014-11-03 11:37:48.144710 7f28fb812700 0 osd.21 105950 do_command r=0 -9731> 2014-11-03 11:37:48.144752 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025488 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968fe00 con 0x4c79420 -9730> 2014-11-03 11:37:48.156423 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450638 192.168.5.2:0/1025488 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2b1ff000 con 0x4c79420 -9729> 2014-11-03 11:37:48.157438 7f28fb812700 0 osd.21 105950 do_command r=0 -9728> 2014-11-03 11:37:48.157463 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025488 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x936ae00 con 0x4c79420 -9727> 2014-11-03 11:37:48.164854 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025488 pipe(0x278032c0 sd=205 :6802 s=2 pgs=2 cs=1 l=1 c=0x4c79420).reader couldn't read tag, (0) Success -9726> 2014-11-03 11:37:48.164925 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025488 pipe(0x278032c0 sd=205 :6802 s=2 pgs=2 cs=1 l=1 c=0x4c79420).fault (0) Success -9725> 2014-11-03 11:37:48.165066 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x4c79420 session 0x32be000 -9724> 2014-11-03 11:37:48.167497 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.111 192.168.5.8:0/32649 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.166034) v2 ==== 47+0+0 (1572291782 0 0) 0x23ba6800 con 0x24229c60 -9723> 2014-11-03 11:37:48.167511 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.111 192.168.5.8:0/32649 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.166034) v2 ==== 47+0+0 (1572291782 0 0) 0x243ff800 con 0x23b1c000 -9722> 2014-11-03 11:37:48.167590 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.166034) v2 -- ?+0 0x31a6e00 con 0x24229c60 -9721> 2014-11-03 11:37:48.167699 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.166034) v2 -- ?+0 0xa5d2e00 con 0x23b1c000 -9720> 2014-11-03 11:37:48.170675 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.57 192.168.5.5:0/16411 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.168953) v2 ==== 47+0+0 (4268805652 0 0) 0x896d200 con 0x242c0160 -9719> 2014-11-03 11:37:48.170710 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.168953) v2 -- ?+0 0x243ff800 con 0x242c0160 -9718> 2014-11-03 11:37:48.170755 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.57 192.168.5.5:0/16411 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.168953) v2 ==== 47+0+0 (4268805652 0 0) 0x896d400 con 0x2469e420 -9717> 2014-11-03 11:37:48.170830 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.168953) v2 -- ?+0 0x23ba6800 con 0x2469e420 -9716> 2014-11-03 11:37:48.183075 7f291450b700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:48.183075, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9715> 2014-11-03 11:37:48.183209 7f2911d06700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:48.183208, event: commit_sent, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9714> 2014-11-03 11:37:48.183241 7f2914d0c700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:48.183241, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9713> 2014-11-03 11:37:48.183286 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1d600 con 0x22fbc840 -9712> 2014-11-03 11:37:48.183312 7f2914d0c700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:48.183312, event: write_thread_in_journal_buffer, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9711> 2014-11-03 11:37:48.183370 7f2914d0c700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:48.183370, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9710> 2014-11-03 11:37:48.183704 7f2912507700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:48.183703, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9709> 2014-11-03 11:37:48.183785 7f2912507700 5 -- op tracker -- seq: 2756, time: 2014-11-03 11:37:48.183785, event: done, op: osd_sub_op(client.7867942.0:137197363 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195955 snapset=0=[]:[] snapc=0=[]) -9708> 2014-11-03 11:37:48.184746 7f2914d0c700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.184745, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9707> 2014-11-03 11:37:48.184806 7f2914d0c700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.184806, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9706> 2014-11-03 11:37:48.220301 7f2918d14700 5 osd.21 105950 tick -9705> 2014-11-03 11:37:48.223514 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 77 ==== osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3276836613 0 3511520646) 0x295a2000 con 0x22ff3b00 -9704> 2014-11-03 11:37:48.223563 7f28f0d99700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.223357, event: header_read, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9703> 2014-11-03 11:37:48.223599 7f28f0d99700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.223360, event: throttled, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9702> 2014-11-03 11:37:48.223622 7f28f0d99700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.223492, event: all_read, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9701> 2014-11-03 11:37:48.223651 7f28f0d99700 5 -- op tracker -- seq: 2767, time: 0.000000, event: dispatched, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9700> 2014-11-03 11:37:48.223729 7f28fd816700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.223729, event: reached_pg, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9699> 2014-11-03 11:37:48.223784 7f28fd816700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.223784, event: started, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9698> 2014-11-03 11:37:48.223897 7f291450b700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:48.223896, event: journaled_completion_queued, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9697> 2014-11-03 11:37:48.223935 7f291450b700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:48.223935, event: journaled_completion_queued, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9696> 2014-11-03 11:37:48.223945 7f28fd816700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.223945, event: started, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9695> 2014-11-03 11:37:48.223971 7f291450b700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:48.223971, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9694> 2014-11-03 11:37:48.223990 7f28fd816700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.223990, event: commit_queued_for_journal_write, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9693> 2014-11-03 11:37:48.224007 7f291450b700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.224007, event: journaled_completion_queued, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9692> 2014-11-03 11:37:48.224004 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 62 ==== osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (245275213 0 3511520646) 0x2cf80800 con 0x22fbc840 -9691> 2014-11-03 11:37:48.224036 7f28f68f4700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.223902, event: header_read, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9690> 2014-11-03 11:37:48.224025 7f2911d06700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:48.224025, event: commit_sent, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9689> 2014-11-03 11:37:48.224073 7f28f68f4700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.223904, event: throttled, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9688> 2014-11-03 11:37:48.224090 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1d340 con 0x22fbc840 -9687> 2014-11-03 11:37:48.224095 7f28f68f4700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.223991, event: all_read, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9686> 2014-11-03 11:37:48.224088 7f291450b700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.224088, event: journaled_completion_queued, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9685> 2014-11-03 11:37:48.224139 7f2911d06700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:48.224139, event: op_commit, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9684> 2014-11-03 11:37:48.224139 7f28f68f4700 5 -- op tracker -- seq: 2768, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9683> 2014-11-03 11:37:48.224175 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.2:0/1027858 -- osd_op_reply(459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] v105950'131093 uv131093 ondisk = 0) v6 -- ?+0 0x23c1c000 con 0x2311c9a0 -9682> 2014-11-03 11:37:48.224202 7f2911d06700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:48.224202, event: commit_sent, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9681> 2014-11-03 11:37:48.224210 7f2914d0c700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.224210, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9680> 2014-11-03 11:37:48.224228 7f2911d06700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:48.224228, event: commit_sent, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9679> 2014-11-03 11:37:48.224214 7f290081c700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.224214, event: reached_pg, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9678> 2014-11-03 11:37:48.224270 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23c1cb00 con 0x22fefdc0 -9677> 2014-11-03 11:37:48.224276 7f290081c700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.224276, event: started, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9676> 2014-11-03 11:37:48.224326 7f2911d06700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.224326, event: commit_sent, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9675> 2014-11-03 11:37:48.224363 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1cdc0 con 0x2308b2c0 -9674> 2014-11-03 11:37:48.224392 7f2911d06700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.224392, event: commit_sent, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9673> 2014-11-03 11:37:48.224406 7f290081c700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.224406, event: started, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9672> 2014-11-03 11:37:48.224413 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1d080 con 0x2308b840 -9671> 2014-11-03 11:37:48.224442 7f290081c700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.224442, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9670> 2014-11-03 11:37:48.224565 7f2912507700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:48.224565, event: sub_op_applied, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9669> 2014-11-03 11:37:48.224637 7f2912507700 5 -- op tracker -- seq: 2757, time: 2014-11-03 11:37:48.224637, event: done, op: osd_sub_op(client.7869425.0:381938 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195956 snapset=0=[]:[] snapc=0=[]) -9668> 2014-11-03 11:37:48.224957 7f2912507700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:48.224957, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9667> 2014-11-03 11:37:48.225005 7f2912507700 5 -- op tracker -- seq: 2759, time: 2014-11-03 11:37:48.225004, event: done, op: osd_sub_op(client.8703713.0:32254334 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953124 snapset=0=[]:[] snapc=0=[]) -9666> 2014-11-03 11:37:48.225120 7f2912507700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:48.225119, event: op_applied, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9665> 2014-11-03 11:37:48.225233 7f2912507700 5 -- op tracker -- seq: 2758, time: 2014-11-03 11:37:48.225233, event: done, op: osd_op(client.9598152.0:459473 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3063808~20480] 24.8f258d3e ack+ondisk+write e105950) -9664> 2014-11-03 11:37:48.225697 7f2912507700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.225697, event: sub_op_applied, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9663> 2014-11-03 11:37:48.225743 7f2912507700 5 -- op tracker -- seq: 2765, time: 2014-11-03 11:37:48.225743, event: done, op: osd_sub_op(client.9052442.0:1100485 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870429 snapset=0=[]:[] snapc=0=[]) -9662> 2014-11-03 11:37:48.225870 7f2912507700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.225870, event: sub_op_applied, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9661> 2014-11-03 11:37:48.225901 7f2912507700 5 -- op tracker -- seq: 2766, time: 2014-11-03 11:37:48.225901, event: done, op: osd_sub_op(client.8133525.0:451258 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000600 snapset=0=[]:[] snapc=0=[]) -9660> 2014-11-03 11:37:48.226894 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 142 ==== osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1110318794 0 3511520646) 0xd4a7400 con 0x22fefdc0 -9659> 2014-11-03 11:37:48.226926 7f28f36c2700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.226759, event: header_read, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9658> 2014-11-03 11:37:48.226945 7f28f36c2700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.226760, event: throttled, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9657> 2014-11-03 11:37:48.226957 7f28f36c2700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.226884, event: all_read, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9656> 2014-11-03 11:37:48.226999 7f28f36c2700 5 -- op tracker -- seq: 2769, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9655> 2014-11-03 11:37:48.227066 7f290101d700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.227066, event: reached_pg, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9654> 2014-11-03 11:37:48.227111 7f290101d700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.227111, event: started, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9653> 2014-11-03 11:37:48.227264 7f290101d700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.227264, event: started, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9652> 2014-11-03 11:37:48.227318 7f290101d700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.227318, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9651> 2014-11-03 11:37:48.232718 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x26bdbc00 con 0x231abb00 -9650> 2014-11-03 11:37:48.242024 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.39 192.168.5.3:0/23401 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.241352) v2 ==== 47+0+0 (3640996191 0 0) 0x243ff200 con 0x31a9000 -9649> 2014-11-03 11:37:48.242083 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/23401 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.241352) v2 -- ?+0 0x896d200 con 0x31a9000 -9648> 2014-11-03 11:37:48.242384 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.39 192.168.5.3:0/23401 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.241352) v2 ==== 47+0+0 (3640996191 0 0) 0x2a8d7600 con 0x2389f000 -9647> 2014-11-03 11:37:48.242447 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/23401 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.241352) v2 -- ?+0 0x896d400 con 0x2389f000 -9646> 2014-11-03 11:37:48.259337 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 63 ==== osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3441081698 0 3511520646) 0x2cf7f000 con 0x22fbc840 -9645> 2014-11-03 11:37:48.259415 7f28f68f4700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.259181, event: header_read, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9644> 2014-11-03 11:37:48.259437 7f28f68f4700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.259184, event: throttled, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9643> 2014-11-03 11:37:48.259450 7f28f68f4700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.259318, event: all_read, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9642> 2014-11-03 11:37:48.259477 7f28f68f4700 5 -- op tracker -- seq: 2770, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9641> 2014-11-03 11:37:48.259547 7f28fe017700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.259547, event: reached_pg, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9640> 2014-11-03 11:37:48.259587 7f28fe017700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.259587, event: started, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9639> 2014-11-03 11:37:48.259753 7f28fe017700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.259753, event: started, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9638> 2014-11-03 11:37:48.259795 7f28fe017700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.259795, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9637> 2014-11-03 11:37:48.286513 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 37 ==== osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (500846205 0 3511520646) 0x2d574000 con 0x231ab580 -9636> 2014-11-03 11:37:48.286600 7f28ea430700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.286378, event: header_read, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9635> 2014-11-03 11:37:48.286628 7f28ea430700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.286380, event: throttled, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9634> 2014-11-03 11:37:48.286637 7f28ea430700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.286499, event: all_read, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9633> 2014-11-03 11:37:48.286645 7f28ea430700 5 -- op tracker -- seq: 2771, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9632> 2014-11-03 11:37:48.286773 7f290181e700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.286773, event: reached_pg, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9631> 2014-11-03 11:37:48.286836 7f290181e700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.286835, event: started, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9630> 2014-11-03 11:37:48.286965 7f290181e700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.286965, event: started, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9629> 2014-11-03 11:37:48.286998 7f290181e700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.286998, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9628> 2014-11-03 11:37:48.290202 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.95 192.168.5.7:0/14919 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.288022) v2 ==== 47+0+0 (3213613292 0 0) 0x6425a00 con 0x2484a580 -9627> 2014-11-03 11:37:48.290215 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.95 192.168.5.7:0/14919 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.288022) v2 ==== 47+0+0 (3213613292 0 0) 0x80d2000 con 0x24321000 -9626> 2014-11-03 11:37:48.290250 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/14919 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.288022) v2 -- ?+0 0x243ff200 con 0x2484a580 -9625> 2014-11-03 11:37:48.290305 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/14919 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.288022) v2 -- ?+0 0x2a8d7600 con 0x24321000 -9624> 2014-11-03 11:37:48.290624 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 64 ==== osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1211135898 0 3511520646) 0x2ce3c800 con 0x22fbc840 -9623> 2014-11-03 11:37:48.290658 7f28f68f4700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.290487, event: header_read, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9622> 2014-11-03 11:37:48.290688 7f28f68f4700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.290489, event: throttled, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9621> 2014-11-03 11:37:48.290704 7f28f68f4700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.290612, event: all_read, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9620> 2014-11-03 11:37:48.290725 7f28f68f4700 5 -- op tracker -- seq: 2772, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9619> 2014-11-03 11:37:48.290799 7f290081c700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.290799, event: reached_pg, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9618> 2014-11-03 11:37:48.290829 7f290081c700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.290829, event: started, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9617> 2014-11-03 11:37:48.290914 7f290081c700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.290914, event: started, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9616> 2014-11-03 11:37:48.290942 7f290081c700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.290942, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9615> 2014-11-03 11:37:48.297459 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.8371799 192.168.1.98:0/1032082 6 ==== ping magic: 0 v1 ==== 0+0+0 (0 0 0) 0x23637c40 con 0x23158c60 -9614> 2014-11-03 11:37:48.309877 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x191b3b00 sd=235 :6802 s=0 pgs=0 cs=0 l=0 c=0x24e1ddc0).accept sd=235 192.168.5.2:42815/0 -9613> 2014-11-03 11:37:48.310933 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450644 192.168.5.2:0/1025758 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x80d2400 con 0x24e1ddc0 -9612> 2014-11-03 11:37:48.311719 7f28fb812700 0 osd.21 105950 do_command r=0 -9611> 2014-11-03 11:37:48.311769 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025758 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xccdac40 con 0x24e1ddc0 -9610> 2014-11-03 11:37:48.316592 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 78 ==== osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (879217548 0 3511520646) 0x25948c00 con 0x22ff3b00 -9609> 2014-11-03 11:37:48.316653 7f28f0d99700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.316414, event: header_read, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9608> 2014-11-03 11:37:48.316672 7f28f0d99700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.316416, event: throttled, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9607> 2014-11-03 11:37:48.316691 7f28f0d99700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.316567, event: all_read, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9606> 2014-11-03 11:37:48.316703 7f28f0d99700 5 -- op tracker -- seq: 2773, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9605> 2014-11-03 11:37:48.316763 7f290001b700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.316763, event: reached_pg, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9604> 2014-11-03 11:37:48.316800 7f290001b700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.316800, event: started, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9603> 2014-11-03 11:37:48.316903 7f290001b700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.316903, event: started, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9602> 2014-11-03 11:37:48.316941 7f290001b700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.316941, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9601> 2014-11-03 11:37:48.324011 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450644 192.168.5.2:0/1025758 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x80d2c00 con 0x24e1ddc0 -9600> 2014-11-03 11:37:48.325287 7f28fb812700 0 osd.21 105950 do_command r=0 -9599> 2014-11-03 11:37:48.325314 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025758 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0xbfa8000 con 0x24e1ddc0 -9598> 2014-11-03 11:37:48.327244 7f28ede6a700 1 -- 192.168.5.2:6803/7606 <== osd.57 192.168.5.5:6807/16411 34 ==== MOSDPGPush(6.167 105950 [PushOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6, version: 105946'5727205, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6@105946'5727205, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (2440343753 0 0) 0x218c4c00 con 0x230c29a0 -9597> 2014-11-03 11:37:48.327287 7f28ede6a700 5 -- op tracker -- seq: 2774, time: 2014-11-03 11:37:48.311503, event: header_read, op: MOSDPGPush(6.167 105950 [PushOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6, version: 105946'5727205, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6@105946'5727205, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9596> 2014-11-03 11:37:48.327325 7f28ede6a700 5 -- op tracker -- seq: 2774, time: 2014-11-03 11:37:48.311505, event: throttled, op: MOSDPGPush(6.167 105950 [PushOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6, version: 105946'5727205, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6@105946'5727205, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9595> 2014-11-03 11:37:48.327354 7f28ede6a700 5 -- op tracker -- seq: 2774, time: 2014-11-03 11:37:48.327230, event: all_read, op: MOSDPGPush(6.167 105950 [PushOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6, version: 105946'5727205, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6@105946'5727205, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9594> 2014-11-03 11:37:48.327381 7f28ede6a700 5 -- op tracker -- seq: 2774, time: 0.000000, event: dispatched, op: MOSDPGPush(6.167 105950 [PushOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6, version: 105946'5727205, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6@105946'5727205, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9593> 2014-11-03 11:37:48.327525 7f28fd816700 5 -- op tracker -- seq: 2774, time: 2014-11-03 11:37:48.327525, event: reached_pg, op: MOSDPGPush(6.167 105950 [PushOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6, version: 105946'5727205, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6@105946'5727205, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9592> 2014-11-03 11:37:48.327724 7f28fd816700 5 -- op tracker -- seq: 2774, time: 2014-11-03 11:37:48.327724, event: done, op: MOSDPGPush(6.167 105950 [PushOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6, version: 105946'5727205, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6@105946'5727205, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9591> 2014-11-03 11:37:48.331755 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025758 pipe(0x191b3b00 sd=235 :6802 s=2 pgs=2 cs=1 l=1 c=0x24e1ddc0).reader couldn't read tag, (0) Success -9590> 2014-11-03 11:37:48.331800 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025758 pipe(0x191b3b00 sd=235 :6802 s=2 pgs=2 cs=1 l=1 c=0x24e1ddc0).fault (0) Success -9589> 2014-11-03 11:37:48.331891 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x24e1ddc0 session 0x23108c80 -9588> 2014-11-03 11:37:48.335652 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 143 ==== osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (206786685 0 3511520646) 0x2de0dc00 con 0x22fefdc0 -9587> 2014-11-03 11:37:48.335692 7f28f36c2700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.335518, event: header_read, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9586> 2014-11-03 11:37:48.335711 7f28f36c2700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.335520, event: throttled, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9585> 2014-11-03 11:37:48.335720 7f28f36c2700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.335638, event: all_read, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9584> 2014-11-03 11:37:48.335729 7f28f36c2700 5 -- op tracker -- seq: 2775, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9583> 2014-11-03 11:37:48.335828 7f28ff81a700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.335828, event: reached_pg, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9582> 2014-11-03 11:37:48.335863 7f28ff81a700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.335863, event: started, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9581> 2014-11-03 11:37:48.336014 7f28ff81a700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.336014, event: started, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9580> 2014-11-03 11:37:48.336048 7f28ff81a700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.336048, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9579> 2014-11-03 11:37:48.354798 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 82 ==== osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1138756175 0 3511520646) 0x25ed4000 con 0x22fbcb00 -9578> 2014-11-03 11:37:48.354846 7f28f6cf8700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.354652, event: header_read, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9577> 2014-11-03 11:37:48.354862 7f28f6cf8700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.354654, event: throttled, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9576> 2014-11-03 11:37:48.354875 7f28f6cf8700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.354781, event: all_read, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9575> 2014-11-03 11:37:48.354884 7f28f6cf8700 5 -- op tracker -- seq: 2776, time: 0.000000, event: dispatched, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9574> 2014-11-03 11:37:48.354979 7f28fe017700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.354979, event: reached_pg, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9573> 2014-11-03 11:37:48.355019 7f28fe017700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.355019, event: started, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9572> 2014-11-03 11:37:48.355113 7f28fe017700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.355113, event: started, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9571> 2014-11-03 11:37:48.355148 7f28fe017700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.355148, event: commit_queued_for_journal_write, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9570> 2014-11-03 11:37:48.355233 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 83 ==== osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2527999455 0 3511520646) 0x2799a800 con 0x22fbc420 -9569> 2014-11-03 11:37:48.355265 7f28f72fe700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.354990, event: header_read, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9568> 2014-11-03 11:37:48.355283 7f28f72fe700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.354992, event: throttled, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9567> 2014-11-03 11:37:48.355296 7f28f72fe700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.355214, event: all_read, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9566> 2014-11-03 11:37:48.355308 7f28f72fe700 5 -- op tracker -- seq: 2777, time: 0.000000, event: dispatched, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9565> 2014-11-03 11:37:48.355413 7f28fe818700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.355413, event: reached_pg, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9564> 2014-11-03 11:37:48.355467 7f28fe818700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.355467, event: started, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9563> 2014-11-03 11:37:48.355664 7f28fe818700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.355663, event: started, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9562> 2014-11-03 11:37:48.355720 7f28fe818700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.355719, event: commit_queued_for_journal_write, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9561> 2014-11-03 11:37:48.356472 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 84 ==== osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3573274150 0 3511520646) 0x263ccc00 con 0x22fbc420 -9560> 2014-11-03 11:37:48.356504 7f28f72fe700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.356370, event: header_read, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9559> 2014-11-03 11:37:48.356522 7f28f72fe700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.356372, event: throttled, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9558> 2014-11-03 11:37:48.356531 7f28f72fe700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.356456, event: all_read, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9557> 2014-11-03 11:37:48.356544 7f28f72fe700 5 -- op tracker -- seq: 2778, time: 0.000000, event: dispatched, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9556> 2014-11-03 11:37:48.356601 7f290101d700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.356601, event: reached_pg, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9555> 2014-11-03 11:37:48.356657 7f290101d700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.356657, event: started, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9554> 2014-11-03 11:37:48.356750 7f290101d700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.356750, event: started, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9553> 2014-11-03 11:37:48.356782 7f290101d700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.356782, event: commit_queued_for_journal_write, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9552> 2014-11-03 11:37:48.368978 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 25 ==== MOSDPGPush(6.58e 105950 [PushOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, version: 105945'11622055, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (2111084544 0 0) 0x26bdbc00 con 0x231abb00 -9551> 2014-11-03 11:37:48.369055 7f28eb945700 5 -- op tracker -- seq: 2779, time: 2014-11-03 11:37:48.356342, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, version: 105945'11622055, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9550> 2014-11-03 11:37:48.369099 7f28eb945700 5 -- op tracker -- seq: 2779, time: 2014-11-03 11:37:48.356344, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, version: 105945'11622055, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9549> 2014-11-03 11:37:48.369127 7f28eb945700 5 -- op tracker -- seq: 2779, time: 2014-11-03 11:37:48.368957, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, version: 105945'11622055, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9548> 2014-11-03 11:37:48.369154 7f28eb945700 5 -- op tracker -- seq: 2779, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, version: 105945'11622055, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9547> 2014-11-03 11:37:48.369256 7f28fe818700 5 -- op tracker -- seq: 2779, time: 2014-11-03 11:37:48.369256, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, version: 105945'11622055, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9546> 2014-11-03 11:37:48.369578 7f28fe818700 5 -- op tracker -- seq: 2779, time: 2014-11-03 11:37:48.369577, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6, version: 105945'11622055, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a223658e/rbd_data.74ae9c3be03aff.0000000000001980/head//6@105945'11622055, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -9545> 2014-11-03 11:37:48.407379 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 70 ==== osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3045613564 0 3511520646) 0x263cd800 con 0x2308b840 -9544> 2014-11-03 11:37:48.407427 7f28ef985700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.407258, event: header_read, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9543> 2014-11-03 11:37:48.407460 7f28ef985700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.407261, event: throttled, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9542> 2014-11-03 11:37:48.407472 7f28ef985700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.407360, event: all_read, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9541> 2014-11-03 11:37:48.407484 7f28ef985700 5 -- op tracker -- seq: 2780, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9540> 2014-11-03 11:37:48.407569 7f290201f700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.407569, event: reached_pg, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9539> 2014-11-03 11:37:48.407603 7f290201f700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.407603, event: started, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9538> 2014-11-03 11:37:48.407832 7f290201f700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.407832, event: started, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9537> 2014-11-03 11:37:48.407867 7f290201f700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.407867, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9536> 2014-11-03 11:37:48.407965 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 71 ==== osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2032646018 0 3511520646) 0x2bd4f800 con 0x2308b840 -9535> 2014-11-03 11:37:48.408010 7f28ef985700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.407881, event: header_read, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9534> 2014-11-03 11:37:48.408029 7f28ef985700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.407883, event: throttled, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9533> 2014-11-03 11:37:48.408042 7f28ef985700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.407955, event: all_read, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9532> 2014-11-03 11:37:48.408057 7f28ef985700 5 -- op tracker -- seq: 2781, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9531> 2014-11-03 11:37:48.408493 7f28ff81a700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.408493, event: reached_pg, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9530> 2014-11-03 11:37:48.408522 7f28ff81a700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.408522, event: started, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9529> 2014-11-03 11:37:48.408609 7f28ff81a700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.408609, event: started, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9528> 2014-11-03 11:37:48.408645 7f28ff81a700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.408645, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9527> 2014-11-03 11:37:48.408656 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 72 ==== osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3562804110 0 3511520646) 0x2bd4ec00 con 0x2308b840 -9526> 2014-11-03 11:37:48.408687 7f28ef985700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.408571, event: header_read, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9525> 2014-11-03 11:37:48.408705 7f28ef985700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.408573, event: throttled, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9524> 2014-11-03 11:37:48.408721 7f28ef985700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.408643, event: all_read, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9523> 2014-11-03 11:37:48.408735 7f28ef985700 5 -- op tracker -- seq: 2782, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9522> 2014-11-03 11:37:48.408794 7f290201f700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.408793, event: reached_pg, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9521> 2014-11-03 11:37:48.408831 7f290201f700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.408831, event: started, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9520> 2014-11-03 11:37:48.408944 7f290201f700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.408944, event: started, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9519> 2014-11-03 11:37:48.408994 7f290201f700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.408994, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9518> 2014-11-03 11:37:48.409273 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 73 ==== osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2587291493 0 3511520646) 0x24e85000 con 0x2308b840 -9517> 2014-11-03 11:37:48.409310 7f28ef985700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.409188, event: header_read, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9516> 2014-11-03 11:37:48.409326 7f28ef985700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.409189, event: throttled, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9515> 2014-11-03 11:37:48.409340 7f28ef985700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.409260, event: all_read, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9514> 2014-11-03 11:37:48.409351 7f28ef985700 5 -- op tracker -- seq: 2783, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9513> 2014-11-03 11:37:48.409397 7f28ff81a700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.409397, event: reached_pg, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9512> 2014-11-03 11:37:48.409428 7f28ff81a700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.409428, event: started, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9511> 2014-11-03 11:37:48.409533 7f28ff81a700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.409532, event: started, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9510> 2014-11-03 11:37:48.409565 7f28ff81a700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.409565, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9509> 2014-11-03 11:37:48.409751 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 74 ==== osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2230394071 0 3511520646) 0x3349800 con 0x2308b840 -9508> 2014-11-03 11:37:48.409786 7f28ef985700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.409633, event: header_read, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9507> 2014-11-03 11:37:48.409802 7f28ef985700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.409635, event: throttled, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9506> 2014-11-03 11:37:48.409814 7f28ef985700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.409741, event: all_read, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9505> 2014-11-03 11:37:48.409825 7f28ef985700 5 -- op tracker -- seq: 2784, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9504> 2014-11-03 11:37:48.409867 7f290201f700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.409867, event: reached_pg, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9503> 2014-11-03 11:37:48.409892 7f290201f700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.409892, event: started, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9502> 2014-11-03 11:37:48.409997 7f290201f700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.409997, event: started, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9501> 2014-11-03 11:37:48.410027 7f290201f700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.410027, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9500> 2014-11-03 11:37:48.410195 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 75 ==== osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2721618183 0 3511520646) 0x28ea3000 con 0x2308b840 -9499> 2014-11-03 11:37:48.410259 7f28ef985700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.410046, event: header_read, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9498> 2014-11-03 11:37:48.410281 7f28ef985700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.410048, event: throttled, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9497> 2014-11-03 11:37:48.410299 7f28ef985700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.410167, event: all_read, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9496> 2014-11-03 11:37:48.410317 7f28ef985700 5 -- op tracker -- seq: 2785, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9495> 2014-11-03 11:37:48.410362 7f28ff81a700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.410361, event: reached_pg, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9494> 2014-11-03 11:37:48.410389 7f28ff81a700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.410389, event: started, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9493> 2014-11-03 11:37:48.410375 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 45 ==== osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3796085212 0 3511520646) 0x27807800 con 0x2308b000 -9492> 2014-11-03 11:37:48.410441 7f28ee470700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.410048, event: header_read, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9491> 2014-11-03 11:37:48.410472 7f28ee470700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.410054, event: throttled, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9490> 2014-11-03 11:37:48.410494 7f28ff81a700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.410494, event: started, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9489> 2014-11-03 11:37:48.410499 7f28ee470700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.410351, event: all_read, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9488> 2014-11-03 11:37:48.410526 7f28ee470700 5 -- op tracker -- seq: 2786, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9487> 2014-11-03 11:37:48.410538 7f28ff81a700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.410538, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9486> 2014-11-03 11:37:48.410630 7f290001b700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.410630, event: reached_pg, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9485> 2014-11-03 11:37:48.410671 7f290001b700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.410671, event: started, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9484> 2014-11-03 11:37:48.410729 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 76 ==== osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3398636159 0 3511520646) 0x28ea4800 con 0x2308b840 -9483> 2014-11-03 11:37:48.410768 7f28ef985700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.410643, event: header_read, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9482> 2014-11-03 11:37:48.410782 7f28ef985700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.410646, event: throttled, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9481> 2014-11-03 11:37:48.410790 7f28ef985700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.410719, event: all_read, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9480> 2014-11-03 11:37:48.410799 7f28ef985700 5 -- op tracker -- seq: 2787, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9479> 2014-11-03 11:37:48.410801 7f290001b700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.410801, event: started, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9478> 2014-11-03 11:37:48.410842 7f290201f700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.410842, event: reached_pg, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9477> 2014-11-03 11:37:48.410854 7f290001b700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.410854, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9476> 2014-11-03 11:37:48.410866 7f290201f700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.410866, event: started, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9475> 2014-11-03 11:37:48.410952 7f290201f700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.410952, event: started, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9474> 2014-11-03 11:37:48.410977 7f290201f700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.410977, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9473> 2014-11-03 11:37:48.411228 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 77 ==== osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1009876161 0 3511520646) 0x2605a800 con 0x2308b840 -9472> 2014-11-03 11:37:48.411258 7f28ef985700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.411122, event: header_read, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9471> 2014-11-03 11:37:48.411272 7f28ef985700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.411124, event: throttled, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9470> 2014-11-03 11:37:48.411281 7f28ef985700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.411215, event: all_read, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9469> 2014-11-03 11:37:48.411289 7f28ef985700 5 -- op tracker -- seq: 2788, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9468> 2014-11-03 11:37:48.411345 7f28ff81a700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.411344, event: reached_pg, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9467> 2014-11-03 11:37:48.411376 7f28ff81a700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.411376, event: started, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9466> 2014-11-03 11:37:48.411465 7f28ff81a700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.411465, event: started, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9465> 2014-11-03 11:37:48.411491 7f28ff81a700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.411491, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9464> 2014-11-03 11:37:48.411504 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 46 ==== osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (196793257 0 3511520646) 0x27806c00 con 0x2308b000 -9463> 2014-11-03 11:37:48.411548 7f28ee470700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.411362, event: header_read, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9462> 2014-11-03 11:37:48.411590 7f28ee470700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.411365, event: throttled, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9461> 2014-11-03 11:37:48.411611 7f28ee470700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.411488, event: all_read, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9460> 2014-11-03 11:37:48.411632 7f28ee470700 5 -- op tracker -- seq: 2789, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9459> 2014-11-03 11:37:48.411708 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 78 ==== osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2010878752 0 3511520646) 0x27072400 con 0x2308b840 -9458> 2014-11-03 11:37:48.411747 7f28ef985700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.411615, event: header_read, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9457> 2014-11-03 11:37:48.411745 7f28fd816700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.411744, event: reached_pg, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9456> 2014-11-03 11:37:48.411780 7f28ef985700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.411621, event: throttled, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9455> 2014-11-03 11:37:48.411827 7f28ef985700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.411696, event: all_read, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9454> 2014-11-03 11:37:48.411832 7f28fd816700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.411832, event: started, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9453> 2014-11-03 11:37:48.411848 7f28ef985700 5 -- op tracker -- seq: 2790, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9452> 2014-11-03 11:37:48.411916 7f290201f700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.411916, event: reached_pg, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9451> 2014-11-03 11:37:48.411951 7f290201f700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.411951, event: started, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9450> 2014-11-03 11:37:48.411970 7f28fd816700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.411970, event: started, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9449> 2014-11-03 11:37:48.412021 7f28fd816700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.412021, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9448> 2014-11-03 11:37:48.412074 7f290201f700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.412074, event: started, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9447> 2014-11-03 11:37:48.412118 7f290201f700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.412118, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9446> 2014-11-03 11:37:48.412167 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 79 ==== osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1867648253 0 3511520646) 0x23c3f800 con 0x2308b840 -9445> 2014-11-03 11:37:48.412194 7f28ef985700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.412077, event: header_read, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9444> 2014-11-03 11:37:48.412210 7f28ef985700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.412079, event: throttled, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9443> 2014-11-03 11:37:48.412219 7f28ef985700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.412159, event: all_read, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9442> 2014-11-03 11:37:48.412232 7f28ef985700 5 -- op tracker -- seq: 2791, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9441> 2014-11-03 11:37:48.412266 7f28ff81a700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.412266, event: reached_pg, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9440> 2014-11-03 11:37:48.412291 7f28ff81a700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.412291, event: started, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9439> 2014-11-03 11:37:48.412385 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 47 ==== osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1357079044 0 3511520646) 0x26151000 con 0x2308b000 -9438> 2014-11-03 11:37:48.412403 7f28ff81a700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.412403, event: started, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9437> 2014-11-03 11:37:48.412427 7f28ee470700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.412245, event: header_read, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9436> 2014-11-03 11:37:48.412433 7f28ff81a700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.412433, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9435> 2014-11-03 11:37:48.412461 7f28ee470700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.412247, event: throttled, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9434> 2014-11-03 11:37:48.412481 7f28ee470700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.412377, event: all_read, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9433> 2014-11-03 11:37:48.412500 7f28ee470700 5 -- op tracker -- seq: 2792, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9432> 2014-11-03 11:37:48.412567 7f290001b700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.412566, event: reached_pg, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9431> 2014-11-03 11:37:48.412606 7f290001b700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.412606, event: started, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9430> 2014-11-03 11:37:48.412686 7f290001b700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.412686, event: started, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9429> 2014-11-03 11:37:48.412715 7f290001b700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.412715, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9428> 2014-11-03 11:37:48.412855 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 80 ==== osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3549258070 0 3511520646) 0x23c40400 con 0x2308b840 -9427> 2014-11-03 11:37:48.412881 7f28ef985700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.412775, event: header_read, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9426> 2014-11-03 11:37:48.412900 7f28ef985700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.412785, event: throttled, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9425> 2014-11-03 11:37:48.412909 7f28ef985700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.412848, event: all_read, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9424> 2014-11-03 11:37:48.412921 7f28ef985700 5 -- op tracker -- seq: 2793, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9423> 2014-11-03 11:37:48.412962 7f290201f700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.412962, event: reached_pg, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9422> 2014-11-03 11:37:48.412996 7f290201f700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.412996, event: started, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9421> 2014-11-03 11:37:48.413105 7f290201f700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.413105, event: started, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9420> 2014-11-03 11:37:48.413133 7f290201f700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.413133, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9419> 2014-11-03 11:37:48.413387 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 81 ==== osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (806152323 0 3511520646) 0x23c3ec00 con 0x2308b840 -9418> 2014-11-03 11:37:48.413425 7f28ef985700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.413286, event: header_read, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9417> 2014-11-03 11:37:48.413452 7f28ef985700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.413288, event: throttled, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9416> 2014-11-03 11:37:48.413473 7f28ef985700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.413376, event: all_read, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9415> 2014-11-03 11:37:48.413501 7f28ef985700 5 -- op tracker -- seq: 2794, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9414> 2014-11-03 11:37:48.413498 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 48 ==== osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (4259881789 0 3511520646) 0x26942000 con 0x2308b000 -9413> 2014-11-03 11:37:48.413542 7f28ee470700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.413345, event: header_read, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9412> 2014-11-03 11:37:48.413554 7f28ff81a700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.413554, event: reached_pg, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9411> 2014-11-03 11:37:48.413574 7f28ff81a700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.413574, event: started, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9410> 2014-11-03 11:37:48.413568 7f28ee470700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.413349, event: throttled, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9409> 2014-11-03 11:37:48.413633 7f28ee470700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.413484, event: all_read, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9408> 2014-11-03 11:37:48.413654 7f28ee470700 5 -- op tracker -- seq: 2795, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9407> 2014-11-03 11:37:48.413687 7f28ff81a700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.413687, event: started, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9406> 2014-11-03 11:37:48.413720 7f28ff81a700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.413720, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9405> 2014-11-03 11:37:48.413719 7f28fd816700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.413718, event: reached_pg, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9404> 2014-11-03 11:37:48.413774 7f28fd816700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.413774, event: started, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9403> 2014-11-03 11:37:48.413912 7f28fd816700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.413912, event: started, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9402> 2014-11-03 11:37:48.413957 7f28fd816700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.413957, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9401> 2014-11-03 11:37:48.414053 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 47 ==== osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (584008076 0 3511520646) 0x2cff3c00 con 0x23000840 -9400> 2014-11-03 11:37:48.414086 7f28f14a0700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.413964, event: header_read, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9399> 2014-11-03 11:37:48.414156 7f28f14a0700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.413966, event: throttled, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9398> 2014-11-03 11:37:48.414179 7f28f14a0700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.414039, event: all_read, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9397> 2014-11-03 11:37:48.414193 7f28f14a0700 5 -- op tracker -- seq: 2796, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9396> 2014-11-03 11:37:48.414250 7f290201f700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.414249, event: reached_pg, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9395> 2014-11-03 11:37:48.414290 7f290201f700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.414290, event: started, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9394> 2014-11-03 11:37:48.414393 7f290201f700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.414393, event: started, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9393> 2014-11-03 11:37:48.414428 7f290201f700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.414428, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9392> 2014-11-03 11:37:48.414475 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 49 ==== osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1938070150 0 3511520646) 0x26942c00 con 0x2308b000 -9391> 2014-11-03 11:37:48.414516 7f28ee470700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.414356, event: header_read, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9390> 2014-11-03 11:37:48.414535 7f28ee470700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.414359, event: throttled, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9389> 2014-11-03 11:37:48.414548 7f28ee470700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.414459, event: all_read, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9388> 2014-11-03 11:37:48.414564 7f28ee470700 5 -- op tracker -- seq: 2797, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9387> 2014-11-03 11:37:48.414583 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 82 ==== osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (131149929 0 3511520646) 0x24867800 con 0x2308b840 -9386> 2014-11-03 11:37:48.414618 7f28ef985700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.414492, event: header_read, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9385> 2014-11-03 11:37:48.414633 7f28ef985700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.414495, event: throttled, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9384> 2014-11-03 11:37:48.414631 7f290001b700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.414631, event: reached_pg, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9383> 2014-11-03 11:37:48.414650 7f28ef985700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.414572, event: all_read, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9382> 2014-11-03 11:37:48.414674 7f290001b700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.414674, event: started, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9381> 2014-11-03 11:37:48.414699 7f28ef985700 5 -- op tracker -- seq: 2798, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9380> 2014-11-03 11:37:48.414759 7f28ff81a700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.414759, event: reached_pg, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9379> 2014-11-03 11:37:48.414778 7f290001b700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.414778, event: started, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9378> 2014-11-03 11:37:48.414784 7f28ff81a700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.414784, event: started, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9377> 2014-11-03 11:37:48.414799 7f290001b700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.414799, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9376> 2014-11-03 11:37:48.414908 7f28ff81a700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.414907, event: started, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9375> 2014-11-03 11:37:48.414939 7f28ff81a700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.414939, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9374> 2014-11-03 11:37:48.414952 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 83 ==== osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (225957428 0 3511520646) 0x2cff5400 con 0x2308b840 -9373> 2014-11-03 11:37:48.414975 7f28ef985700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.414853, event: header_read, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9372> 2014-11-03 11:37:48.414989 7f28ef985700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.414855, event: throttled, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9371> 2014-11-03 11:37:48.415004 7f28ef985700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.414942, event: all_read, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9370> 2014-11-03 11:37:48.415013 7f28ef985700 5 -- op tracker -- seq: 2799, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9369> 2014-11-03 11:37:48.415058 7f290201f700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.415058, event: reached_pg, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9368> 2014-11-03 11:37:48.415091 7f290201f700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.415091, event: started, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9367> 2014-11-03 11:37:48.415159 7f290201f700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.415159, event: started, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9366> 2014-11-03 11:37:48.415183 7f290201f700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.415183, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9365> 2014-11-03 11:37:48.415181 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 48 ==== osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3666686692 0 3511520646) 0x2ce3bc00 con 0x23000840 -9364> 2014-11-03 11:37:48.415203 7f28f14a0700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.415098, event: header_read, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9363> 2014-11-03 11:37:48.415220 7f28f14a0700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.415100, event: throttled, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9362> 2014-11-03 11:37:48.415228 7f28f14a0700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.415172, event: all_read, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9361> 2014-11-03 11:37:48.415236 7f28f14a0700 5 -- op tracker -- seq: 2800, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9360> 2014-11-03 11:37:48.415269 7f28ff81a700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.415269, event: reached_pg, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9359> 2014-11-03 11:37:48.415286 7f28ff81a700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.415286, event: started, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9358> 2014-11-03 11:37:48.415396 7f28ff81a700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.415396, event: started, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9357> 2014-11-03 11:37:48.415394 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 84 ==== osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (912267533 0 3511520646) 0x96ea000 con 0x2308b840 -9356> 2014-11-03 11:37:48.415428 7f28ef985700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.415318, event: header_read, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9355> 2014-11-03 11:37:48.415431 7f28ff81a700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.415430, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9354> 2014-11-03 11:37:48.415447 7f28ef985700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.415319, event: throttled, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9353> 2014-11-03 11:37:48.415469 7f28ef985700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.415385, event: all_read, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9352> 2014-11-03 11:37:48.415485 7f28ef985700 5 -- op tracker -- seq: 2801, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9351> 2014-11-03 11:37:48.415551 7f290201f700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.415551, event: reached_pg, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9350> 2014-11-03 11:37:48.415583 7f290201f700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.415583, event: started, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9349> 2014-11-03 11:37:48.415694 7f290201f700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.415694, event: started, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9348> 2014-11-03 11:37:48.415735 7f290201f700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.415734, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9347> 2014-11-03 11:37:48.415878 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 49 ==== osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (731865329 0 3511520646) 0x28fef400 con 0x23000840 -9346> 2014-11-03 11:37:48.415903 7f28f14a0700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.415808, event: header_read, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9345> 2014-11-03 11:37:48.415920 7f28f14a0700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.415809, event: throttled, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9344> 2014-11-03 11:37:48.415928 7f28f14a0700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.415869, event: all_read, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9343> 2014-11-03 11:37:48.415963 7f28f14a0700 5 -- op tracker -- seq: 2802, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9342> 2014-11-03 11:37:48.415909 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 50 ==== osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (497996957 0 3511520646) 0x2c691800 con 0x2308b000 -9341> 2014-11-03 11:37:48.415983 7f28ee470700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.415781, event: header_read, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9340> 2014-11-03 11:37:48.416009 7f28ff81a700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.416009, event: reached_pg, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9339> 2014-11-03 11:37:48.416011 7f28ee470700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.415792, event: throttled, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9338> 2014-11-03 11:37:48.416031 7f28ee470700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.415896, event: all_read, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9337> 2014-11-03 11:37:48.416046 7f28ff81a700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.416046, event: started, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9336> 2014-11-03 11:37:48.416007 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 85 ==== osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3805537411 0 3511520646) 0x2de0e800 con 0x2308b840 -9335> 2014-11-03 11:37:48.416054 7f28ee470700 5 -- op tracker -- seq: 2803, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9334> 2014-11-03 11:37:48.416065 7f28ef985700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.415871, event: header_read, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9333> 2014-11-03 11:37:48.416079 7f28ef985700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.415872, event: throttled, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9332> 2014-11-03 11:37:48.416097 7f28ef985700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.415996, event: all_read, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9331> 2014-11-03 11:37:48.416107 7f28ef985700 5 -- op tracker -- seq: 2804, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9330> 2014-11-03 11:37:48.416121 7f28fd816700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.416120, event: reached_pg, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9329> 2014-11-03 11:37:48.416146 7f290201f700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.416146, event: reached_pg, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9328> 2014-11-03 11:37:48.416150 7f28ff81a700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.416150, event: started, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9327> 2014-11-03 11:37:48.416160 7f28fd816700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.416160, event: started, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9326> 2014-11-03 11:37:48.416175 7f290201f700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.416175, event: started, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9325> 2014-11-03 11:37:48.416201 7f28ff81a700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.416201, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9324> 2014-11-03 11:37:48.416314 7f28fd816700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.416314, event: started, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9323> 2014-11-03 11:37:48.416335 7f290201f700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.416335, event: started, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9322> 2014-11-03 11:37:48.416361 7f28fd816700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.416361, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9321> 2014-11-03 11:37:48.416424 7f290201f700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.416424, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9320> 2014-11-03 11:37:48.416483 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 86 ==== osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (444047245 0 3511520646) 0x2de0d000 con 0x2308b840 -9319> 2014-11-03 11:37:48.416525 7f28ef985700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.416371, event: header_read, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9318> 2014-11-03 11:37:48.416541 7f28ef985700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.416373, event: throttled, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9317> 2014-11-03 11:37:48.416550 7f28ef985700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.416469, event: all_read, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9316> 2014-11-03 11:37:48.416570 7f28ef985700 5 -- op tracker -- seq: 2805, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9315> 2014-11-03 11:37:48.416599 7f28ff81a700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.416598, event: reached_pg, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9314> 2014-11-03 11:37:48.416589 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 50 ==== osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (496133042 0 3511520646) 0x24662800 con 0x23000840 -9313> 2014-11-03 11:37:48.416620 7f28ff81a700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.416620, event: started, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9312> 2014-11-03 11:37:48.416632 7f28f14a0700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.416521, event: header_read, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9311> 2014-11-03 11:37:48.416670 7f28f14a0700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.416522, event: throttled, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9310> 2014-11-03 11:37:48.416681 7f28f14a0700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.416581, event: all_read, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9309> 2014-11-03 11:37:48.416683 7f28ff81a700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.416683, event: started, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9308> 2014-11-03 11:37:48.416696 7f28f14a0700 5 -- op tracker -- seq: 2806, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9307> 2014-11-03 11:37:48.416714 7f28ff81a700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.416714, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9306> 2014-11-03 11:37:48.416727 7f290201f700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.416727, event: reached_pg, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9305> 2014-11-03 11:37:48.416740 7f290201f700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.416740, event: started, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9304> 2014-11-03 11:37:48.416882 7f290201f700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.416881, event: started, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9303> 2014-11-03 11:37:48.416921 7f290201f700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.416921, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9302> 2014-11-03 11:37:48.416916 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 87 ==== osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3807245891 0 3511520646) 0x263ce400 con 0x2308b840 -9301> 2014-11-03 11:37:48.416951 7f28ef985700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.416782, event: header_read, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9300> 2014-11-03 11:37:48.416977 7f28ef985700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.416783, event: throttled, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9299> 2014-11-03 11:37:48.417022 7f28ef985700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.416903, event: all_read, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9298> 2014-11-03 11:37:48.417044 7f28ef985700 5 -- op tracker -- seq: 2807, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9297> 2014-11-03 11:37:48.417108 7f28ff81a700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.417108, event: reached_pg, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9296> 2014-11-03 11:37:48.417144 7f28ff81a700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.417144, event: started, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9295> 2014-11-03 11:37:48.417224 7f28ff81a700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.417224, event: started, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9294> 2014-11-03 11:37:48.417249 7f28ff81a700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.417249, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9293> 2014-11-03 11:37:48.417280 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 51 ==== osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (657802267 0 3511520646) 0x2c692400 con 0x2308b000 -9292> 2014-11-03 11:37:48.417331 7f28ee470700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.417168, event: header_read, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9291> 2014-11-03 11:37:48.417356 7f28ee470700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.417171, event: throttled, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9290> 2014-11-03 11:37:48.417375 7f28ee470700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.417267, event: all_read, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9289> 2014-11-03 11:37:48.417399 7f28ee470700 5 -- op tracker -- seq: 2808, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9288> 2014-11-03 11:37:48.417478 7f290001b700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.417478, event: reached_pg, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9287> 2014-11-03 11:37:48.417504 7f290001b700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.417504, event: started, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9286> 2014-11-03 11:37:48.417591 7f290001b700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.417591, event: started, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9285> 2014-11-03 11:37:48.417620 7f290001b700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.417620, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9284> 2014-11-03 11:37:48.417708 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 88 ==== osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1579131031 0 3511520646) 0x263cc000 con 0x2308b840 -9283> 2014-11-03 11:37:48.417748 7f28ef985700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.417596, event: header_read, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9282> 2014-11-03 11:37:48.417772 7f28ef985700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.417598, event: throttled, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9281> 2014-11-03 11:37:48.417791 7f28ef985700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.417695, event: all_read, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9280> 2014-11-03 11:37:48.417804 7f28ef985700 5 -- op tracker -- seq: 2809, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9279> 2014-11-03 11:37:48.417860 7f290201f700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.417860, event: reached_pg, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9278> 2014-11-03 11:37:48.417889 7f290201f700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.417888, event: started, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9277> 2014-11-03 11:37:48.418019 7f290201f700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.418018, event: started, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9276> 2014-11-03 11:37:48.418056 7f290201f700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.418055, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9275> 2014-11-03 11:37:48.418063 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 51 ==== osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2828912873 0 3511520646) 0x27ed9800 con 0x23000840 -9274> 2014-11-03 11:37:48.418093 7f28f14a0700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.417990, event: header_read, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9273> 2014-11-03 11:37:48.418131 7f28f14a0700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.417991, event: throttled, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9272> 2014-11-03 11:37:48.418141 7f28f14a0700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.418054, event: all_read, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9271> 2014-11-03 11:37:48.418154 7f28f14a0700 5 -- op tracker -- seq: 2810, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9270> 2014-11-03 11:37:48.418238 7f28ff81a700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.418237, event: reached_pg, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9269> 2014-11-03 11:37:48.418272 7f28ff81a700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.418272, event: started, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9268> 2014-11-03 11:37:48.418392 7f28ff81a700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.418392, event: started, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9267> 2014-11-03 11:37:48.418425 7f28ff81a700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.418425, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9266> 2014-11-03 11:37:48.419330 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 52 ==== osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2772858574 0 3511520646) 0x2398e400 con 0x2308b000 -9265> 2014-11-03 11:37:48.419389 7f28ee470700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.419201, event: header_read, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9264> 2014-11-03 11:37:48.419442 7f28ee470700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.419218, event: throttled, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9263> 2014-11-03 11:37:48.419465 7f28ee470700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.419319, event: all_read, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9262> 2014-11-03 11:37:48.419486 7f28ee470700 5 -- op tracker -- seq: 2811, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9261> 2014-11-03 11:37:48.419564 7f28fd816700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.419564, event: reached_pg, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9260> 2014-11-03 11:37:48.419626 7f28fd816700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.419626, event: started, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9259> 2014-11-03 11:37:48.419732 7f28fd816700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.419732, event: started, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9258> 2014-11-03 11:37:48.419762 7f28fd816700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.419762, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9257> 2014-11-03 11:37:48.420370 7f28ee470700 1 -- 192.168.5.2:6803/7606 <== osd.63 192.168.5.5:6841/21346 53 ==== osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3945183084 0 3511520646) 0x2398d800 con 0x2308b000 -9256> 2014-11-03 11:37:48.420401 7f28ee470700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.420268, event: header_read, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9255> 2014-11-03 11:37:48.420424 7f28ee470700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.420271, event: throttled, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9254> 2014-11-03 11:37:48.420446 7f28ee470700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.420358, event: all_read, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9253> 2014-11-03 11:37:48.420464 7f28ee470700 5 -- op tracker -- seq: 2812, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9252> 2014-11-03 11:37:48.420505 7f290001b700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.420505, event: reached_pg, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9251> 2014-11-03 11:37:48.420532 7f290001b700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.420532, event: started, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9250> 2014-11-03 11:37:48.420619 7f290001b700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.420619, event: started, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9249> 2014-11-03 11:37:48.420654 7f290001b700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.420654, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9248> 2014-11-03 11:37:48.435242 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 32 ==== osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3860111431 0 3511520646) 0x265db800 con 0x22ff3580 -9247> 2014-11-03 11:37:48.435316 7f28f17a3700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.434926, event: header_read, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9246> 2014-11-03 11:37:48.435371 7f28f17a3700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.434931, event: throttled, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9245> 2014-11-03 11:37:48.435405 7f28f17a3700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.435215, event: all_read, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9244> 2014-11-03 11:37:48.435448 7f28f17a3700 5 -- op tracker -- seq: 2813, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9243> 2014-11-03 11:37:48.435584 7f28fd816700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.435584, event: reached_pg, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9242> 2014-11-03 11:37:48.435618 7f28fd816700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.435618, event: started, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9241> 2014-11-03 11:37:48.435710 7f28fd816700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.435709, event: started, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9240> 2014-11-03 11:37:48.435745 7f28fd816700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.435745, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9239> 2014-11-03 11:37:48.451496 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 31 ==== osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (774778591 0 3511520646) 0x3125bc00 con 0x22ff3420 -9238> 2014-11-03 11:37:48.451541 7f28f0b97700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.451366, event: header_read, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9237> 2014-11-03 11:37:48.451557 7f28f0b97700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.451370, event: throttled, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9236> 2014-11-03 11:37:48.451569 7f28f0b97700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.451480, event: all_read, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9235> 2014-11-03 11:37:48.451591 7f28f0b97700 5 -- op tracker -- seq: 2814, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9234> 2014-11-03 11:37:48.451664 7f290201f700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.451664, event: reached_pg, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9233> 2014-11-03 11:37:48.451721 7f290201f700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.451721, event: started, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9232> 2014-11-03 11:37:48.451964 7f290201f700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.451964, event: started, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9231> 2014-11-03 11:37:48.452002 7f290201f700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.452002, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9230> 2014-11-03 11:37:48.455491 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 32 ==== osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2267434258 0 3511520646) 0x3125d400 con 0x22ff3420 -9229> 2014-11-03 11:37:48.455534 7f28f0b97700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.455404, event: header_read, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9228> 2014-11-03 11:37:48.455556 7f28f0b97700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.455406, event: throttled, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9227> 2014-11-03 11:37:48.455573 7f28f0b97700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.455477, event: all_read, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9226> 2014-11-03 11:37:48.455596 7f28f0b97700 5 -- op tracker -- seq: 2815, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9225> 2014-11-03 11:37:48.455666 7f28ff81a700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.455666, event: reached_pg, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9224> 2014-11-03 11:37:48.455722 7f28ff81a700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.455722, event: started, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9223> 2014-11-03 11:37:48.455905 7f28ff81a700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.455905, event: started, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9222> 2014-11-03 11:37:48.455950 7f28ff81a700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.455950, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9221> 2014-11-03 11:37:48.457631 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 33 ==== osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1616071314 0 3511520646) 0x2fe0e400 con 0x22ff3420 -9220> 2014-11-03 11:37:48.457663 7f28f0b97700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.457532, event: header_read, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9219> 2014-11-03 11:37:48.457680 7f28f0b97700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.457533, event: throttled, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9218> 2014-11-03 11:37:48.457692 7f28f0b97700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.457617, event: all_read, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9217> 2014-11-03 11:37:48.457704 7f28f0b97700 5 -- op tracker -- seq: 2816, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9216> 2014-11-03 11:37:48.457751 7f290201f700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.457751, event: reached_pg, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9215> 2014-11-03 11:37:48.457779 7f290201f700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.457779, event: started, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9214> 2014-11-03 11:37:48.457883 7f290201f700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.457883, event: started, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9213> 2014-11-03 11:37:48.457918 7f290201f700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.457918, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9212> 2014-11-03 11:37:48.463171 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x191b3840 sd=235 :6802 s=0 pgs=0 cs=0 l=0 c=0x2741f420).accept sd=235 192.168.5.2:42821/0 -9211> 2014-11-03 11:37:48.463465 7f28c4099700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31525080 sd=239 :6802 s=0 pgs=0 cs=0 l=0 c=0x24537580).accept sd=239 192.168.5.201:58633/0 -9210> 2014-11-03 11:37:48.464267 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450787 192.168.5.2:0/1025968 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2be31e00 con 0x2741f420 -9209> 2014-11-03 11:37:48.464387 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419841 192.168.5.201:0/1028389 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "24.7d7"}) v1 ==== 80+0+0 (3756934197 0 0) 0x2ba4d800 con 0x24537580 -9208> 2014-11-03 11:37:48.464990 7f28fb812700 0 osd.21 105950 do_command r=0 -9207> 2014-11-03 11:37:48.465017 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025968 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x272dac40 con 0x2741f420 -9206> 2014-11-03 11:37:48.465694 7f28fb812700 0 osd.21 105950 do_command r=0 -9205> 2014-11-03 11:37:48.465732 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028389 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x272da1c0 con 0x24537580 -9204> 2014-11-03 11:37:48.468785 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 89 ==== osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3261495586 0 3511520646) 0x2de2ac00 con 0x2308b840 -9203> 2014-11-03 11:37:48.468835 7f28ef985700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.468577, event: header_read, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9202> 2014-11-03 11:37:48.468849 7f28ef985700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.468580, event: throttled, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9201> 2014-11-03 11:37:48.468857 7f28ef985700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.468767, event: all_read, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9200> 2014-11-03 11:37:48.468866 7f28ef985700 5 -- op tracker -- seq: 2817, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9199> 2014-11-03 11:37:48.468938 7f28ff81a700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.468938, event: reached_pg, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9198> 2014-11-03 11:37:48.468982 7f28ff81a700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.468981, event: started, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9197> 2014-11-03 11:37:48.469144 7f28ff81a700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.469144, event: started, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9196> 2014-11-03 11:37:48.469178 7f28ff81a700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469722 > 104857600 -9195> 2014-11-03 11:37:48.477426 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450787 192.168.5.2:0/1025968 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2350ac00 con 0x2741f420 -9194> 2014-11-03 11:37:48.478205 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419841 192.168.5.201:0/1028389 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "24.7d7"}) v1 ==== 74+0+0 (774687807 0 0) 0x2a8d7200 con 0x24537580 -9193> 2014-11-03 11:37:48.478653 7f28fb812700 0 osd.21 105950 do_command r=0 -9192> 2014-11-03 11:37:48.478680 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1025968 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x272daa80 con 0x2741f420 -9191> 2014-11-03 11:37:48.479727 7f28fb812700 0 osd.21 105950 do_command r=0 -9190> 2014-11-03 11:37:48.479763 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028389 -- command_reply(tid 2: 0 ) v1 -- ?+13413 0x245008c0 con 0x24537580 -9189> 2014-11-03 11:37:48.484781 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028389 pipe(0x31525080 sd=239 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537580).reader couldn't read tag, (0) Success -9188> 2014-11-03 11:37:48.484827 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028389 pipe(0x31525080 sd=239 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537580).fault (0) Success -9187> 2014-11-03 11:37:48.484953 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x24537580 session 0x23108c80 -9186> 2014-11-03 11:37:48.485220 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025968 pipe(0x191b3840 sd=235 :6802 s=2 pgs=2 cs=1 l=1 c=0x2741f420).reader couldn't read tag, (0) Success -9185> 2014-11-03 11:37:48.485269 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1025968 pipe(0x191b3840 sd=235 :6802 s=2 pgs=2 cs=1 l=1 c=0x2741f420).fault (0) Success -9184> 2014-11-03 11:37:48.485366 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2741f420 session 0x25045a00 -9183> 2014-11-03 11:37:48.499613 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 144 ==== osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (4068211267 0 3511520646) 0x3348000 con 0x22fefdc0 -9182> 2014-11-03 11:37:48.499682 7f28f36c2700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.499312, event: header_read, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9181> 2014-11-03 11:37:48.499713 7f28f36c2700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.499316, event: throttled, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9180> 2014-11-03 11:37:48.499733 7f28f36c2700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.499590, event: all_read, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9179> 2014-11-03 11:37:48.499766 7f28f36c2700 5 -- op tracker -- seq: 2818, time: 0.000000, event: dispatched, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9178> 2014-11-03 11:37:48.499879 7f290101d700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.499879, event: reached_pg, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9177> 2014-11-03 11:37:48.499931 7f290101d700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.499931, event: started, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9176> 2014-11-03 11:37:48.500058 7f290101d700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.500058, event: started, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9175> 2014-11-03 11:37:48.500081 7f290101d700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469727 > 104857600 -9174> 2014-11-03 11:37:48.544052 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 90 ==== osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (221951065 0 3511520646) 0x2d47d400 con 0x2308b840 -9173> 2014-11-03 11:37:48.544105 7f28ef985700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.543907, event: header_read, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9172> 2014-11-03 11:37:48.544127 7f28ef985700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.543910, event: throttled, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9171> 2014-11-03 11:37:48.544158 7f28ef985700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.544033, event: all_read, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9170> 2014-11-03 11:37:48.544184 7f28ef985700 5 -- op tracker -- seq: 2819, time: 0.000000, event: dispatched, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9169> 2014-11-03 11:37:48.544271 7f290081c700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.544270, event: reached_pg, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9168> 2014-11-03 11:37:48.544325 7f290081c700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.544325, event: started, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9167> 2014-11-03 11:37:48.544512 7f290081c700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.544512, event: started, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9166> 2014-11-03 11:37:48.544542 7f290081c700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469722 > 104857600 -9165> 2014-11-03 11:37:48.590706 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.35 192.168.5.3:0/6455 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.589778) v2 ==== 47+0+0 (1965989562 0 0) 0x1106ca00 con 0x2484a6e0 -9164> 2014-11-03 11:37:48.590795 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.589778) v2 -- ?+0 0x6425a00 con 0x2484a6e0 -9163> 2014-11-03 11:37:48.590871 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.35 192.168.5.3:0/6455 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.589778) v2 ==== 47+0+0 (1965989562 0 0) 0x6352a00 con 0x24321160 -9162> 2014-11-03 11:37:48.590940 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.589778) v2 -- ?+0 0x80d2000 con 0x24321160 -9161> 2014-11-03 11:37:48.597926 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.83 192.168.5.6:0/4005 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.597043) v2 ==== 47+0+0 (2744664902 0 0) 0x234d5200 con 0x2445c2c0 -9160> 2014-11-03 11:37:48.597969 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.597043) v2 -- ?+0 0x1106ca00 con 0x2445c2c0 -9159> 2014-11-03 11:37:48.598010 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.83 192.168.5.6:0/4005 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.597043) v2 ==== 47+0+0 (2744664902 0 0) 0x13d27a00 con 0x23ea4420 -9158> 2014-11-03 11:37:48.598069 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.597043) v2 -- ?+0 0x6352a00 con 0x23ea4420 -9157> 2014-11-03 11:37:48.614603 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 79 ==== osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (733267771 0 3511520646) 0x27758000 con 0x22ff3b00 -9156> 2014-11-03 11:37:48.614641 7f28f0d99700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.614409, event: header_read, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9155> 2014-11-03 11:37:48.614658 7f28f0d99700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.614412, event: throttled, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9154> 2014-11-03 11:37:48.614672 7f28f0d99700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.614592, event: all_read, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9153> 2014-11-03 11:37:48.614703 7f28f0d99700 5 -- op tracker -- seq: 2820, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9152> 2014-11-03 11:37:48.614785 7f28fe818700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.614784, event: reached_pg, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9151> 2014-11-03 11:37:48.614829 7f28fe818700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.614829, event: started, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9150> 2014-11-03 11:37:48.614955 7f28fe818700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.614955, event: started, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9149> 2014-11-03 11:37:48.614991 7f28fe818700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469722 > 104857600 -9148> 2014-11-03 11:37:48.616010 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 80 ==== osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3181839471 0 3511520646) 0x2de2b800 con 0x22ff3b00 -9147> 2014-11-03 11:37:48.616048 7f28f0d99700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.615920, event: header_read, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -9146> 2014-11-03 11:37:48.616067 7f28f0d99700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.615922, event: throttled, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -9145> 2014-11-03 11:37:48.616080 7f28f0d99700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.615995, event: all_read, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -9144> 2014-11-03 11:37:48.616092 7f28f0d99700 5 -- op tracker -- seq: 2821, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -9143> 2014-11-03 11:37:48.617511 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 81 ==== osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2959600436 0 3511520646) 0xd55d000 con 0x22ff3b00 -9142> 2014-11-03 11:37:48.617547 7f28f0d99700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.617408, event: header_read, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -9141> 2014-11-03 11:37:48.617571 7f28f0d99700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.617410, event: throttled, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -9140> 2014-11-03 11:37:48.617591 7f28f0d99700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.617494, event: all_read, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -9139> 2014-11-03 11:37:48.617609 7f28f0d99700 5 -- op tracker -- seq: 2822, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -9138> 2014-11-03 11:37:48.621322 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.99 192.168.5.8:0/26930 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.619861) v2 ==== 47+0+0 (1000866037 0 0) 0x2a8ef800 con 0x24205580 -9137> 2014-11-03 11:37:48.621374 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.619861) v2 -- ?+0 0x13d27a00 con 0x24205580 -9136> 2014-11-03 11:37:48.621363 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.99 192.168.5.8:0/26930 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.619861) v2 ==== 47+0+0 (1000866037 0 0) 0x2d329e00 con 0x239f8160 -9135> 2014-11-03 11:37:48.621411 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.619861) v2 -- ?+0 0x234d5200 con 0x239f8160 -9134> 2014-11-03 11:37:48.621615 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x279fc340 sd=235 :6802 s=0 pgs=0 cs=0 l=0 c=0x245376e0).accept sd=235 192.168.5.2:42826/0 -9133> 2014-11-03 11:37:48.622341 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450653 192.168.5.2:0/1026216 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2579ae00 con 0x245376e0 -9132> 2014-11-03 11:37:48.622856 7f28fb812700 0 osd.21 105950 do_command r=0 -9131> 2014-11-03 11:37:48.622875 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1026216 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x32f78c0 con 0x245376e0 -9130> 2014-11-03 11:37:48.627377 7f291450b700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.627377, event: journaled_completion_queued, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9129> 2014-11-03 11:37:48.627429 7f2911d06700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.627429, event: commit_sent, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9128> 2014-11-03 11:37:48.627460 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] ondisk, result = 0) v2 -- ?+0 0xa23e580 con 0x22ff3b00 -9127> 2014-11-03 11:37:48.627467 7f2914d0c700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.627467, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9126> 2014-11-03 11:37:48.627524 7f2914d0c700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.627524, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9125> 2014-11-03 11:37:48.627570 7f2914d0c700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.627570, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9124> 2014-11-03 11:37:48.627617 7f2914d0c700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.627616, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9123> 2014-11-03 11:37:48.627659 7f2914d0c700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.627659, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9122> 2014-11-03 11:37:48.627703 7f2914d0c700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.627703, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9121> 2014-11-03 11:37:48.627806 7f290101d700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469726 > 104857600 -9120> 2014-11-03 11:37:48.627833 7f290081c700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469721 > 104857600 -9119> 2014-11-03 11:37:48.627832 7f28ff81a700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.627832, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9118> 2014-11-03 11:37:48.627839 7f2912507700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.627839, event: sub_op_applied, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9117> 2014-11-03 11:37:48.627862 7f28fe818700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469721 > 104857600 -9116> 2014-11-03 11:37:48.627888 7f2912507700 5 -- op tracker -- seq: 2767, time: 2014-11-03 11:37:48.627888, event: done, op: osd_sub_op(client.7890490.0:244500 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222062 snapset=0=[]:[] snapc=0=[]) -9115> 2014-11-03 11:37:48.628395 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.104 192.168.5.8:0/2576 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.626532) v2 ==== 47+0+0 (876624664 0 0) 0x22a08000 con 0x239f8000 -9114> 2014-11-03 11:37:48.628419 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.104 192.168.5.8:0/2576 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.626532) v2 ==== 47+0+0 (876624664 0 0) 0x23821000 con 0x239f8dc0 -9113> 2014-11-03 11:37:48.628449 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.626532) v2 -- ?+0 0x2d329e00 con 0x239f8000 -9112> 2014-11-03 11:37:48.628505 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.626532) v2 -- ?+0 0x2a8ef800 con 0x239f8dc0 -9111> 2014-11-03 11:37:48.628818 7f2914d0c700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.628818, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9110> 2014-11-03 11:37:48.628868 7f2914d0c700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.628868, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9109> 2014-11-03 11:37:48.628918 7f2914d0c700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.628918, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9108> 2014-11-03 11:37:48.628960 7f2914d0c700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.628960, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9107> 2014-11-03 11:37:48.629973 7f2914d0c700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.629973, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9106> 2014-11-03 11:37:48.630025 7f2914d0c700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.630025, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9105> 2014-11-03 11:37:48.630075 7f2914d0c700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.630075, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9104> 2014-11-03 11:37:48.630185 7f2914d0c700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.630185, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9103> 2014-11-03 11:37:48.630240 7f2914d0c700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.630240, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9102> 2014-11-03 11:37:48.630286 7f2914d0c700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.630286, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9101> 2014-11-03 11:37:48.630341 7f2914d0c700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.630341, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9100> 2014-11-03 11:37:48.630376 7f2914d0c700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.630376, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9099> 2014-11-03 11:37:48.630428 7f2914d0c700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.630428, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9098> 2014-11-03 11:37:48.630464 7f2914d0c700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.630464, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9097> 2014-11-03 11:37:48.630507 7f2914d0c700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.630507, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9096> 2014-11-03 11:37:48.630547 7f2914d0c700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.630547, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9095> 2014-11-03 11:37:48.630591 7f2914d0c700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.630591, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9094> 2014-11-03 11:37:48.630629 7f2914d0c700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.630629, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9093> 2014-11-03 11:37:48.630668 7f2914d0c700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.630668, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9092> 2014-11-03 11:37:48.630709 7f2914d0c700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.630709, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9091> 2014-11-03 11:37:48.630747 7f2914d0c700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.630747, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9090> 2014-11-03 11:37:48.630792 7f2914d0c700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.630792, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9089> 2014-11-03 11:37:48.630831 7f2914d0c700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.630831, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9088> 2014-11-03 11:37:48.630873 7f2914d0c700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.630873, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9087> 2014-11-03 11:37:48.630911 7f2914d0c700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.630911, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9086> 2014-11-03 11:37:48.630956 7f2914d0c700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.630956, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9085> 2014-11-03 11:37:48.630995 7f2914d0c700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.630995, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9084> 2014-11-03 11:37:48.631031 7f2914d0c700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.631031, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9083> 2014-11-03 11:37:48.631066 7f2914d0c700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.631066, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9082> 2014-11-03 11:37:48.631106 7f2914d0c700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.631106, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9081> 2014-11-03 11:37:48.631137 7f2914d0c700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.631137, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -9080> 2014-11-03 11:37:48.631185 7f2914d0c700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.631185, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -9079> 2014-11-03 11:37:48.631223 7f2914d0c700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.631223, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -9078> 2014-11-03 11:37:48.631270 7f2914d0c700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.631270, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -9077> 2014-11-03 11:37:48.631322 7f2914d0c700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.631321, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -9076> 2014-11-03 11:37:48.631354 7f2914d0c700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.631354, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -9075> 2014-11-03 11:37:48.631393 7f2914d0c700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.631392, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -9074> 2014-11-03 11:37:48.631427 7f2914d0c700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.631427, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -9073> 2014-11-03 11:37:48.631468 7f2914d0c700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.631468, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -9072> 2014-11-03 11:37:48.631510 7f2914d0c700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.631510, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -9071> 2014-11-03 11:37:48.631556 7f2914d0c700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.631556, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -9070> 2014-11-03 11:37:48.631597 7f2914d0c700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.631596, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -9069> 2014-11-03 11:37:48.632536 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 105 ==== osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1807133462 0 3511520646) 0x26c42000 con 0x22fb76e0 -9068> 2014-11-03 11:37:48.632576 7f28edd69700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.632426, event: header_read, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -9067> 2014-11-03 11:37:48.632594 7f28edd69700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.632429, event: throttled, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -9066> 2014-11-03 11:37:48.632606 7f28edd69700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.632522, event: all_read, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -9065> 2014-11-03 11:37:48.632618 7f28edd69700 5 -- op tracker -- seq: 2823, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -9064> 2014-11-03 11:37:48.632676 7f290001b700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.632676, event: reached_pg, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -9063> 2014-11-03 11:37:48.632706 7f290001b700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.632706, event: started, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -9062> 2014-11-03 11:37:48.632802 7f290001b700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.632802, event: started, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -9061> 2014-11-03 11:37:48.632824 7f290001b700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469721 > 104857600 -9060> 2014-11-03 11:37:48.637661 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450653 192.168.5.2:0/1026216 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x26bdb400 con 0x245376e0 -9059> 2014-11-03 11:37:48.639235 7f28fb812700 0 osd.21 105950 do_command r=0 -9058> 2014-11-03 11:37:48.639260 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1026216 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x2b23e000 con 0x245376e0 -9057> 2014-11-03 11:37:48.646509 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1026216 pipe(0x279fc340 sd=235 :6802 s=2 pgs=2 cs=1 l=1 c=0x245376e0).reader couldn't read tag, (0) Success -9056> 2014-11-03 11:37:48.646581 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1026216 pipe(0x279fc340 sd=235 :6802 s=2 pgs=2 cs=1 l=1 c=0x245376e0).fault (0) Success -9055> 2014-11-03 11:37:48.646756 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x245376e0 session 0x25045a00 -9054> 2014-11-03 11:37:48.663423 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.107 192.168.5.8:0/17474 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.662210) v2 ==== 47+0+0 (384673475 0 0) 0x2470a800 con 0x239f8580 -9053> 2014-11-03 11:37:48.663476 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.662210) v2 -- ?+0 0x22a08000 con 0x239f8580 -9052> 2014-11-03 11:37:48.663484 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.107 192.168.5.8:0/17474 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.662210) v2 ==== 47+0+0 (384673475 0 0) 0x1a327200 con 0x238b86e0 -9051> 2014-11-03 11:37:48.663546 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.662210) v2 -- ?+0 0x23821000 con 0x238b86e0 -9050> 2014-11-03 11:37:48.705389 7f291450b700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.705388, event: journaled_completion_queued, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9049> 2014-11-03 11:37:48.705440 7f2911d06700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.705440, event: commit_sent, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9048> 2014-11-03 11:37:48.705494 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa23e2c0 con 0x22fbc840 -9047> 2014-11-03 11:37:48.705452 7f291450b700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.705452, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9046> 2014-11-03 11:37:48.705549 7f291450b700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.705549, event: journaled_completion_queued, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9045> 2014-11-03 11:37:48.705573 7f291450b700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.705573, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -9044> 2014-11-03 11:37:48.705593 7f291450b700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.705593, event: journaled_completion_queued, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -9043> 2014-11-03 11:37:48.705611 7f291450b700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.705611, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -9042> 2014-11-03 11:37:48.705635 7f291450b700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.705635, event: journaled_completion_queued, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -9041> 2014-11-03 11:37:48.705653 7f291450b700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.705653, event: journaled_completion_queued, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -9040> 2014-11-03 11:37:48.705671 7f291450b700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.705671, event: journaled_completion_queued, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -9039> 2014-11-03 11:37:48.705690 7f291450b700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.705690, event: journaled_completion_queued, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -9038> 2014-11-03 11:37:48.705709 7f291450b700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.705708, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -9037> 2014-11-03 11:37:48.705727 7f291450b700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.705727, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -9036> 2014-11-03 11:37:48.705745 7f291450b700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.705745, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -9035> 2014-11-03 11:37:48.705763 7f291450b700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.705763, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -9034> 2014-11-03 11:37:48.705780 7f291450b700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.705780, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -9033> 2014-11-03 11:37:48.705797 7f291450b700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.705797, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -9032> 2014-11-03 11:37:48.705815 7f291450b700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.705815, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -9031> 2014-11-03 11:37:48.705832 7f291450b700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.705832, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -9030> 2014-11-03 11:37:48.705850 7f291450b700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.705850, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -9029> 2014-11-03 11:37:48.705867 7f291450b700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.705867, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -9028> 2014-11-03 11:37:48.705875 7f290081c700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469730 > 104857600 -9027> 2014-11-03 11:37:48.705867 7f2912507700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.705867, event: sub_op_applied, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9026> 2014-11-03 11:37:48.705885 7f291450b700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.705885, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -9025> 2014-11-03 11:37:48.705905 7f2912507700 5 -- op tracker -- seq: 2768, time: 2014-11-03 11:37:48.705905, event: done, op: osd_sub_op(client.7869425.0:381939 6.a8a 442a7a8a/rbd_data.4b73002ae8944a.0000000000000000/head//6 [] v 105950'8195957 snapset=0=[]:[] snapc=0=[]) -9024> 2014-11-03 11:37:48.705882 7f290101d700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.705882, event: commit_queued_for_journal_write, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -9023> 2014-11-03 11:37:48.705917 7f291450b700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.705917, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -9022> 2014-11-03 11:37:48.706074 7f28fe818700 2 filestore(/var/lib/ceph/osd/cloud-21) waiting 51 > 50 ops || 8469725 > 104857600 -9021> 2014-11-03 11:37:48.706086 7f2912507700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.706086, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -9020> 2014-11-03 11:37:48.706083 7f290081c700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.706083, event: commit_queued_for_journal_write, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -9019> 2014-11-03 11:37:48.706138 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ack, result = 0) v2 -- ?+0 0x23df9dc0 con 0x22fefdc0 -9018> 2014-11-03 11:37:48.706187 7f291450b700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.706187, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -9017> 2014-11-03 11:37:48.706215 7f291450b700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.706215, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -9016> 2014-11-03 11:37:48.706234 7f291450b700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.706233, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -9015> 2014-11-03 11:37:48.706251 7f291450b700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.706251, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -9014> 2014-11-03 11:37:48.706266 7f291450b700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.706266, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -9013> 2014-11-03 11:37:48.706282 7f291450b700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.706282, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -9012> 2014-11-03 11:37:48.706299 7f291450b700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.706299, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -9011> 2014-11-03 11:37:48.706316 7f291450b700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.706316, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -9010> 2014-11-03 11:37:48.706307 7f28fe818700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.706307, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -9009> 2014-11-03 11:37:48.706337 7f291450b700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.706337, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -9008> 2014-11-03 11:37:48.706355 7f291450b700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.706355, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -9007> 2014-11-03 11:37:48.706373 7f291450b700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.706373, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -9006> 2014-11-03 11:37:48.706371 7f290101d700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.706371, event: reached_pg, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -9005> 2014-11-03 11:37:48.706399 7f291450b700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.706399, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -9004> 2014-11-03 11:37:48.706418 7f291450b700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.706418, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -9003> 2014-11-03 11:37:48.706373 7f2912507700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.706372, event: sub_op_applied, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -9002> 2014-11-03 11:37:48.706441 7f291450b700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.706441, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -9001> 2014-11-03 11:37:48.706447 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ack, result = 0) v2 -- ?+0 0x23df9000 con 0x22fbc840 -9000> 2014-11-03 11:37:48.706459 7f291450b700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.706459, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -8999> 2014-11-03 11:37:48.706476 7f291450b700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.706476, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -8998> 2014-11-03 11:37:48.706493 7f291450b700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.706493, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -8997> 2014-11-03 11:37:48.706513 7f291450b700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.706513, event: journaled_completion_queued, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -8996> 2014-11-03 11:37:48.706531 7f291450b700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.706531, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -8995> 2014-11-03 11:37:48.706548 7f291450b700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.706547, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -8994> 2014-11-03 11:37:48.706564 7f291450b700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.706564, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -8993> 2014-11-03 11:37:48.706554 7f290001b700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.706554, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -8992> 2014-11-03 11:37:48.706581 7f291450b700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.706581, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -8991> 2014-11-03 11:37:48.706588 7f2912507700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.706588, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -8990> 2014-11-03 11:37:48.706598 7f291450b700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.706598, event: journaled_completion_queued, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -8989> 2014-11-03 11:37:48.706616 7f291450b700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.706616, event: journaled_completion_queued, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -8988> 2014-11-03 11:37:48.706619 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] ack, result = 0) v2 -- ?+0 0x240d22c0 con 0x231ab580 -8987> 2014-11-03 11:37:48.706634 7f291450b700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.706634, event: journaled_completion_queued, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -8986> 2014-11-03 11:37:48.706699 7f291450b700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.706698, event: journaled_completion_queued, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -8985> 2014-11-03 11:37:48.706716 7f2911d06700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.706716, event: commit_sent, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -8984> 2014-11-03 11:37:48.706655 7f2912507700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.706655, event: sub_op_applied, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -8983> 2014-11-03 11:37:48.706745 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ack, result = 0) v2 -- ?+0 0x2905a600 con 0x22fbc840 -8982> 2014-11-03 11:37:48.706743 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0xa23e000 con 0x22fefdc0 -8981> 2014-11-03 11:37:48.706816 7f2911d06700 5 -- op tracker -- seq: 2769, time: 2014-11-03 11:37:48.706816, event: done, op: osd_sub_op(client.8703713.0:32254336 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953125 snapset=0=[]:[] snapc=0=[]) -8980> 2014-11-03 11:37:48.706417 7f290101d700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.706417, event: started, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8979> 2014-11-03 11:37:48.706882 7f2911d06700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.706882, event: commit_sent, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -8978> 2014-11-03 11:37:48.706910 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b41e2c0 con 0x22fbc840 -8977> 2014-11-03 11:37:48.706947 7f2911d06700 5 -- op tracker -- seq: 2770, time: 2014-11-03 11:37:48.706947, event: done, op: osd_sub_op(client.7867945.0:15039326 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195958 snapset=0=[]:[] snapc=0=[]) -8976> 2014-11-03 11:37:48.706992 7f2911d06700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.706992, event: commit_sent, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -8975> 2014-11-03 11:37:48.707012 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23dfa8c0 con 0x231ab580 -8974> 2014-11-03 11:37:48.707022 7f290101d700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.707022, event: started, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8973> 2014-11-03 11:37:48.707051 7f2911d06700 5 -- op tracker -- seq: 2771, time: 2014-11-03 11:37:48.707050, event: done, op: osd_sub_op(client.8236383.0:16630592 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629959 snapset=0=[]:[] snapc=0=[]) -8972> 2014-11-03 11:37:48.707056 7f290101d700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.707056, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8971> 2014-11-03 11:37:48.707087 7f2911d06700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.707087, event: commit_sent, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -8970> 2014-11-03 11:37:48.707108 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x239268c0 con 0x22fbc840 -8969> 2014-11-03 11:37:48.707113 7f28fe818700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.707113, event: reached_pg, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8968> 2014-11-03 11:37:48.707134 7f28fe818700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.707134, event: started, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8967> 2014-11-03 11:37:48.707138 7f2911d06700 5 -- op tracker -- seq: 2772, time: 2014-11-03 11:37:48.707138, event: done, op: osd_sub_op(client.7867945.0:15039327 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195959 snapset=0=[]:[] snapc=0=[]) -8966> 2014-11-03 11:37:48.707169 7f2911d06700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.707168, event: commit_sent, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -8965> 2014-11-03 11:37:48.707191 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x922d600 con 0x22ff3b00 -8964> 2014-11-03 11:37:48.707226 7f2911d06700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.707226, event: commit_sent, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -8963> 2014-11-03 11:37:48.707247 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] ondisk, result = 0) v2 -- ?+0 0x922cdc0 con 0x22fefdc0 -8962> 2014-11-03 11:37:48.707280 7f2911d06700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.707280, event: commit_sent, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -8961> 2014-11-03 11:37:48.707295 7f28fe818700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.707294, event: started, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8960> 2014-11-03 11:37:48.707302 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] ondisk, result = 0) v2 -- ?+0 0x922d8c0 con 0x22fbcb00 -8959> 2014-11-03 11:37:48.707323 7f28fe818700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.707323, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8958> 2014-11-03 11:37:48.707334 7f2911d06700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.707333, event: commit_sent, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -8957> 2014-11-03 11:37:48.707355 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] ondisk, result = 0) v2 -- ?+0 0x248b6b00 con 0x22fbc420 -8956> 2014-11-03 11:37:48.707381 7f2911d06700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.707381, event: commit_sent, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -8955> 2014-11-03 11:37:48.707498 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23926600 con 0x22fbc420 -8954> 2014-11-03 11:37:48.707555 7f2911d06700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.707555, event: commit_sent, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -8953> 2014-11-03 11:37:48.707583 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x314a0080 con 0x2308b840 -8952> 2014-11-03 11:37:48.707587 7f2912507700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.707587, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -8951> 2014-11-03 11:37:48.707609 7f2911d06700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.707609, event: commit_sent, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -8950> 2014-11-03 11:37:48.707635 7f2912507700 5 -- op tracker -- seq: 2773, time: 2014-11-03 11:37:48.707635, event: done, op: osd_sub_op(client.12146205.0:439294 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222063 snapset=0=[]:[] snapc=0=[]) -8949> 2014-11-03 11:37:48.707671 7f2912507700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.707671, event: sub_op_applied, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -8948> 2014-11-03 11:37:48.707676 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23dfa080 con 0x2308b840 -8947> 2014-11-03 11:37:48.707697 7f2911d06700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.707697, event: commit_sent, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -8946> 2014-11-03 11:37:48.707697 7f2912507700 5 -- op tracker -- seq: 2775, time: 2014-11-03 11:37:48.707697, event: done, op: osd_sub_op(client.12404072.0:246010 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033669 snapset=0=[]:[] snapc=0=[]) -8945> 2014-11-03 11:37:48.707735 7f2914d0c700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.707735, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -8944> 2014-11-03 11:37:48.707745 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x31525340 con 0x2308b840 -8943> 2014-11-03 11:37:48.707766 7f2911d06700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.707766, event: commit_sent, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -8942> 2014-11-03 11:37:48.707770 7f2914d0c700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.707770, event: write_thread_in_journal_buffer, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -8941> 2014-11-03 11:37:48.707784 7f2912507700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.707783, event: sub_op_applied, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -8940> 2014-11-03 11:37:48.707805 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x29ce3b80 con 0x2308b840 -8939> 2014-11-03 11:37:48.707803 7f2914d0c700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.707803, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -8938> 2014-11-03 11:37:48.707825 7f2911d06700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.707825, event: commit_sent, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -8937> 2014-11-03 11:37:48.707823 7f2912507700 5 -- op tracker -- seq: 2776, time: 2014-11-03 11:37:48.707823, event: done, op: osd_sub_op(client.9561337.0:516028 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134359 snapset=0=[]:[] snapc=0=[]) -8936> 2014-11-03 11:37:48.707835 7f2914d0c700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.707835, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -8935> 2014-11-03 11:37:48.707862 7f2914d0c700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.707862, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8934> 2014-11-03 11:37:48.707868 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d34a600 con 0x2308b840 -8933> 2014-11-03 11:37:48.707889 7f2914d0c700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.707888, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8932> 2014-11-03 11:37:48.707900 7f2911d06700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.707900, event: commit_sent, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -8931> 2014-11-03 11:37:48.707951 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x270d2580 con 0x2308b840 -8930> 2014-11-03 11:37:48.707985 7f2911d06700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.707985, event: commit_sent, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -8929> 2014-11-03 11:37:48.708002 7f2912507700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.708002, event: sub_op_applied, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -8928> 2014-11-03 11:37:48.708033 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x239252c0 con 0x2308b000 -8927> 2014-11-03 11:37:48.708037 7f2912507700 5 -- op tracker -- seq: 2777, time: 2014-11-03 11:37:48.708037, event: done, op: osd_sub_op(client.9561337.0:516022 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965047 snapset=0=[]:[] snapc=0=[]) -8926> 2014-11-03 11:37:48.708072 7f2911d06700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.708072, event: commit_sent, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -8925> 2014-11-03 11:37:48.708099 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23926b80 con 0x2308b840 -8924> 2014-11-03 11:37:48.708135 7f2911d06700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.708135, event: commit_sent, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -8923> 2014-11-03 11:37:48.708162 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23925000 con 0x2308b840 -8922> 2014-11-03 11:37:48.708191 7f2911d06700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.708191, event: commit_sent, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -8921> 2014-11-03 11:37:48.708218 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2de67000 con 0x2308b000 -8920> 2014-11-03 11:37:48.708243 7f2911d06700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.708243, event: commit_sent, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -8919> 2014-11-03 11:37:48.708266 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2de688c0 con 0x2308b840 -8918> 2014-11-03 11:37:48.708287 7f2911d06700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.708287, event: commit_sent, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -8917> 2014-11-03 11:37:48.708308 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2de672c0 con 0x2308b840 -8916> 2014-11-03 11:37:48.708331 7f2911d06700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.708331, event: commit_sent, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -8915> 2014-11-03 11:37:48.708353 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2de68600 con 0x2308b000 -8914> 2014-11-03 11:37:48.708377 7f2911d06700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.708377, event: commit_sent, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -8913> 2014-11-03 11:37:48.708365 7f2912507700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.708365, event: sub_op_applied, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -8912> 2014-11-03 11:37:48.708404 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2de68080 con 0x2308b840 -8911> 2014-11-03 11:37:48.708410 7f2912507700 5 -- op tracker -- seq: 2778, time: 2014-11-03 11:37:48.708410, event: done, op: osd_sub_op(client.9561337.0:516023 6.665 9db5ce65/rbd_data.43ef922ae8944a.0000000000000001/head//6 [] v 105950'9965048 snapset=0=[]:[] snapc=0=[]) -8910> 2014-11-03 11:37:48.708426 7f2911d06700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.708426, event: commit_sent, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -8909> 2014-11-03 11:37:48.708448 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23bd98c0 con 0x2308b840 -8908> 2014-11-03 11:37:48.708469 7f2911d06700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.708469, event: commit_sent, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -8907> 2014-11-03 11:37:48.708491 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23bd9600 con 0x2308b000 -8906> 2014-11-03 11:37:48.708513 7f2911d06700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.708513, event: commit_sent, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -8905> 2014-11-03 11:37:48.708538 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] ondisk, result = 0) v2 -- ?+0 0x316d1580 con 0x23000840 -8904> 2014-11-03 11:37:48.708571 7f2911d06700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.708571, event: commit_sent, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -8903> 2014-11-03 11:37:48.708591 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x316d2340 con 0x2308b000 -8902> 2014-11-03 11:37:48.708612 7f2911d06700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.708612, event: commit_sent, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -8901> 2014-11-03 11:37:48.708633 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269af580 con 0x2308b840 -8900> 2014-11-03 11:37:48.708655 7f2911d06700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.708654, event: commit_sent, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -8899> 2014-11-03 11:37:48.708675 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269af840 con 0x2308b840 -8898> 2014-11-03 11:37:48.708697 7f2911d06700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.708697, event: commit_sent, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -8897> 2014-11-03 11:37:48.708718 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23dfa340 con 0x23000840 -8896> 2014-11-03 11:37:48.708747 7f2911d06700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.708747, event: commit_sent, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -8895> 2014-11-03 11:37:48.708774 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9b8c0 con 0x2308b840 -8894> 2014-11-03 11:37:48.708793 7f2911d06700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.708793, event: commit_sent, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -8893> 2014-11-03 11:37:48.708816 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9a840 con 0x23000840 -8892> 2014-11-03 11:37:48.708844 7f2911d06700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.708843, event: commit_sent, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -8891> 2014-11-03 11:37:48.708864 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272ebb80 con 0x2308b000 -8890> 2014-11-03 11:37:48.708885 7f2911d06700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.708885, event: commit_sent, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -8889> 2014-11-03 11:37:48.708912 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eb600 con 0x2308b840 -8888> 2014-11-03 11:37:48.708937 7f2911d06700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.708937, event: commit_sent, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -8887> 2014-11-03 11:37:48.708965 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272ea580 con 0x2308b840 -8886> 2014-11-03 11:37:48.708983 7f2911d06700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.708983, event: commit_sent, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -8885> 2014-11-03 11:37:48.709000 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eb080 con 0x23000840 -8884> 2014-11-03 11:37:48.709024 7f2911d06700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.709024, event: commit_sent, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -8883> 2014-11-03 11:37:48.709037 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272ea000 con 0x2308b840 -8882> 2014-11-03 11:37:48.709048 7f2911d06700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.709048, event: commit_sent, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -8881> 2014-11-03 11:37:48.709060 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272ea2c0 con 0x2308b000 -8880> 2014-11-03 11:37:48.709071 7f2911d06700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.709071, event: commit_sent, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -8879> 2014-11-03 11:37:48.709083 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eb340 con 0x2308b840 -8878> 2014-11-03 11:37:48.709099 7f2911d06700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.709099, event: commit_sent, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -8877> 2014-11-03 11:37:48.709116 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcac000 con 0x23000840 -8876> 2014-11-03 11:37:48.709138 7f2911d06700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.709138, event: commit_sent, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -8875> 2014-11-03 11:37:48.709164 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcac2c0 con 0x2308b000 -8874> 2014-11-03 11:37:48.709190 7f2911d06700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.709190, event: commit_sent, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -8873> 2014-11-03 11:37:48.709219 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6841/21346 -- osd_sub_op_reply(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcac580 con 0x2308b000 -8872> 2014-11-03 11:37:48.709242 7f2911d06700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.709242, event: commit_sent, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -8871> 2014-11-03 11:37:48.709268 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcad080 con 0x22ff3580 -8870> 2014-11-03 11:37:48.709307 7f2911d06700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.709307, event: commit_sent, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -8869> 2014-11-03 11:37:48.709335 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op_reply(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcacdc0 con 0x22ff3420 -8868> 2014-11-03 11:37:48.709367 7f2911d06700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.709366, event: commit_sent, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -8867> 2014-11-03 11:37:48.709389 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op_reply(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcacb00 con 0x22ff3420 -8866> 2014-11-03 11:37:48.709407 7f2911d06700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.709407, event: commit_sent, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -8865> 2014-11-03 11:37:48.709428 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op_reply(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcad8c0 con 0x22ff3420 -8864> 2014-11-03 11:37:48.709445 7f2911d06700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.709444, event: commit_sent, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -8863> 2014-11-03 11:37:48.709464 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcad600 con 0x2308b840 -8862> 2014-11-03 11:37:48.710291 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 145 ==== osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3000464283 0 3511520646) 0x27eac000 con 0x22fefdc0 -8861> 2014-11-03 11:37:48.710338 7f28f36c2700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.710105, event: header_read, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8860> 2014-11-03 11:37:48.710359 7f28f36c2700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.710107, event: throttled, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8859> 2014-11-03 11:37:48.710373 7f28f36c2700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.710276, event: all_read, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8858> 2014-11-03 11:37:48.710386 7f28f36c2700 5 -- op tracker -- seq: 2824, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8857> 2014-11-03 11:37:48.710449 7f290101d700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.710448, event: reached_pg, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8856> 2014-11-03 11:37:48.710479 7f290101d700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.710479, event: started, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8855> 2014-11-03 11:37:48.710604 7f290101d700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.710604, event: started, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8854> 2014-11-03 11:37:48.710640 7f290101d700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.710640, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8853> 2014-11-03 11:37:48.711521 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 146 ==== osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (454809256 0 3511520646) 0x2d575800 con 0x22fefdc0 -8852> 2014-11-03 11:37:48.711577 7f28f36c2700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.711390, event: header_read, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8851> 2014-11-03 11:37:48.711604 7f28f36c2700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.711392, event: throttled, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8850> 2014-11-03 11:37:48.711619 7f28f36c2700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.711505, event: all_read, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8849> 2014-11-03 11:37:48.711667 7f28f36c2700 5 -- op tracker -- seq: 2825, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8848> 2014-11-03 11:37:48.711817 7f28fe818700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.711816, event: reached_pg, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8847> 2014-11-03 11:37:48.711853 7f28fe818700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.711853, event: started, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8846> 2014-11-03 11:37:48.711992 7f28fe818700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.711992, event: started, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8845> 2014-11-03 11:37:48.712035 7f28fe818700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.712035, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8844> 2014-11-03 11:37:48.727464 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 82 ==== osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2872088322 0 3511520646) 0x22f9f400 con 0x22ff3b00 -8843> 2014-11-03 11:37:48.727521 7f28f0d99700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.727319, event: header_read, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8842> 2014-11-03 11:37:48.727562 7f28f0d99700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.727322, event: throttled, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8841> 2014-11-03 11:37:48.727583 7f28f0d99700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.727441, event: all_read, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8840> 2014-11-03 11:37:48.727604 7f28f0d99700 5 -- op tracker -- seq: 2826, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8839> 2014-11-03 11:37:48.727702 7f28fd816700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.727702, event: reached_pg, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8838> 2014-11-03 11:37:48.727758 7f28fd816700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.727757, event: started, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8837> 2014-11-03 11:37:48.727940 7f28fd816700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.727939, event: started, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8836> 2014-11-03 11:37:48.727996 7f28fd816700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.727996, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8835> 2014-11-03 11:37:48.734895 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.50 192.168.5.4:0/26902 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.733515) v2 ==== 47+0+0 (3809185336 0 0) 0x2350ae00 con 0x23335160 -8834> 2014-11-03 11:37:48.734933 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.50 192.168.5.4:0/26902 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.733515) v2 ==== 47+0+0 (3809185336 0 0) 0x23323a00 con 0x234ed160 -8833> 2014-11-03 11:37:48.734971 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/26902 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.733515) v2 -- ?+0 0x2470a800 con 0x23335160 -8832> 2014-11-03 11:37:48.735139 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/26902 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.733515) v2 -- ?+0 0x1a327200 con 0x234ed160 -8831> 2014-11-03 11:37:48.770442 7f291450b700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.770441, event: journaled_completion_queued, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -8830> 2014-11-03 11:37:48.770491 7f291450b700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.770491, event: journaled_completion_queued, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -8829> 2014-11-03 11:37:48.770508 7f291450b700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.770507, event: journaled_completion_queued, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -8828> 2014-11-03 11:37:48.770527 7f291450b700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.770527, event: journaled_completion_queued, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -8827> 2014-11-03 11:37:48.770551 7f291450b700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.770551, event: journaled_completion_queued, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8826> 2014-11-03 11:37:48.770545 7f2911d06700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.770545, event: commit_sent, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -8825> 2014-11-03 11:37:48.770636 7f291450b700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.770636, event: journaled_completion_queued, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8824> 2014-11-03 11:37:48.770685 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2fcadb80 con 0x22fefdc0 -8823> 2014-11-03 11:37:48.770707 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6807/16411 -- MOSDPGPushReply(6.167 105950 [PushReplyOp(a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6)]) v2 -- ?+0 0x8025e00 con 0x230c29a0 -8822> 2014-11-03 11:37:48.770736 7f2911d06700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.770736, event: commit_sent, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -8821> 2014-11-03 11:37:48.770764 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294addc0 con 0x2308b840 -8820> 2014-11-03 11:37:48.770767 7f2914d0c700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.770766, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8819> 2014-11-03 11:37:48.770815 7f2911d06700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.770815, event: commit_sent, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -8818> 2014-11-03 11:37:48.770844 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294ad580 con 0x22ff3b00 -8817> 2014-11-03 11:37:48.770840 7f2914d0c700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.770840, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8816> 2014-11-03 11:37:48.770902 7f2911d06700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.770902, event: commit_sent, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -8815> 2014-11-03 11:37:48.770917 7f2914d0c700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.770917, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8814> 2014-11-03 11:37:48.770943 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294ae8c0 con 0x22fb76e0 -8813> 2014-11-03 11:37:48.770969 7f2912507700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.770969, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -8812> 2014-11-03 11:37:48.770980 7f2911d06700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.770980, event: commit_sent, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8811> 2014-11-03 11:37:48.771033 7f2912507700 5 -- op tracker -- seq: 2780, time: 2014-11-03 11:37:48.771033, event: done, op: osd_sub_op(client.7868656.0:3476924 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899990 snapset=0=[]:[] snapc=0=[]) -8810> 2014-11-03 11:37:48.771035 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263ae000 con 0x22ff3b00 -8809> 2014-11-03 11:37:48.771060 7f2911d06700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.771060, event: commit_sent, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8808> 2014-11-03 11:37:48.771095 7f2912507700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.771094, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -8807> 2014-11-03 11:37:48.771124 7f2912507700 5 -- op tracker -- seq: 2781, time: 2014-11-03 11:37:48.771124, event: done, op: osd_sub_op(client.7868656.0:3476925 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899991 snapset=0=[]:[] snapc=0=[]) -8806> 2014-11-03 11:37:48.771137 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263aeb00 con 0x22ff3b00 -8805> 2014-11-03 11:37:48.771311 7f2912507700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.771311, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -8804> 2014-11-03 11:37:48.771354 7f2912507700 5 -- op tracker -- seq: 2782, time: 2014-11-03 11:37:48.771354, event: done, op: osd_sub_op(client.7868656.0:3476926 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899992 snapset=0=[]:[] snapc=0=[]) -8803> 2014-11-03 11:37:48.771431 7f2912507700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.771431, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -8802> 2014-11-03 11:37:48.771458 7f2912507700 5 -- op tracker -- seq: 2783, time: 2014-11-03 11:37:48.771458, event: done, op: osd_sub_op(client.7868656.0:3476927 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899993 snapset=0=[]:[] snapc=0=[]) -8801> 2014-11-03 11:37:48.771625 7f2912507700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.771624, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -8800> 2014-11-03 11:37:48.771681 7f2912507700 5 -- op tracker -- seq: 2784, time: 2014-11-03 11:37:48.771680, event: done, op: osd_sub_op(client.7868656.0:3476928 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899994 snapset=0=[]:[] snapc=0=[]) -8799> 2014-11-03 11:37:48.771749 7f2912507700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.771749, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -8798> 2014-11-03 11:37:48.771784 7f2912507700 5 -- op tracker -- seq: 2785, time: 2014-11-03 11:37:48.771784, event: done, op: osd_sub_op(client.7868656.0:3476929 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899995 snapset=0=[]:[] snapc=0=[]) -8797> 2014-11-03 11:37:48.772072 7f2912507700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.772072, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -8796> 2014-11-03 11:37:48.772116 7f2912507700 5 -- op tracker -- seq: 2786, time: 2014-11-03 11:37:48.772116, event: done, op: osd_sub_op(client.7914641.0:33713261 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199781 snapset=0=[]:[] snapc=0=[]) -8795> 2014-11-03 11:37:48.772369 7f2912507700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.772368, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -8794> 2014-11-03 11:37:48.772452 7f2912507700 5 -- op tracker -- seq: 2787, time: 2014-11-03 11:37:48.772452, event: done, op: osd_sub_op(client.7868656.0:3476933 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899996 snapset=0=[]:[] snapc=0=[]) -8793> 2014-11-03 11:37:48.772554 7f28ede6a700 1 -- 192.168.5.2:6803/7606 <== osd.57 192.168.5.5:6807/16411 35 ==== osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4921 (2568387149 0 555681506) 0x2444e000 con 0x230c29a0 -8792> 2014-11-03 11:37:48.772586 7f28ede6a700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.772459, event: header_read, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8791> 2014-11-03 11:37:48.772595 7f2912507700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.772595, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -8790> 2014-11-03 11:37:48.772604 7f28ede6a700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.772461, event: throttled, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8789> 2014-11-03 11:37:48.772645 7f28ede6a700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.772542, event: all_read, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8788> 2014-11-03 11:37:48.772658 7f28ede6a700 5 -- op tracker -- seq: 2827, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8787> 2014-11-03 11:37:48.772655 7f2912507700 5 -- op tracker -- seq: 2788, time: 2014-11-03 11:37:48.772655, event: done, op: osd_sub_op(client.7868656.0:3476934 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899997 snapset=0=[]:[] snapc=0=[]) -8786> 2014-11-03 11:37:48.772722 7f290001b700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.772721, event: reached_pg, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8785> 2014-11-03 11:37:48.772749 7f290001b700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.772749, event: started, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8784> 2014-11-03 11:37:48.772777 7f2912507700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.772776, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -8783> 2014-11-03 11:37:48.772829 7f290001b700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.772829, event: started, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8782> 2014-11-03 11:37:48.772833 7f2912507700 5 -- op tracker -- seq: 2789, time: 2014-11-03 11:37:48.772833, event: done, op: osd_sub_op(client.7914641.0:33713262 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199782 snapset=0=[]:[] snapc=0=[]) -8781> 2014-11-03 11:37:48.772859 7f290001b700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.772859, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8780> 2014-11-03 11:37:48.772891 7f2912507700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.772891, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -8779> 2014-11-03 11:37:48.772920 7f2912507700 5 -- op tracker -- seq: 2790, time: 2014-11-03 11:37:48.772920, event: done, op: osd_sub_op(client.7868656.0:3476935 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899998 snapset=0=[]:[] snapc=0=[]) -8778> 2014-11-03 11:37:48.773079 7f2912507700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.773079, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -8777> 2014-11-03 11:37:48.773134 7f2912507700 5 -- op tracker -- seq: 2791, time: 2014-11-03 11:37:48.773134, event: done, op: osd_sub_op(client.7868656.0:3476936 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28899999 snapset=0=[]:[] snapc=0=[]) -8776> 2014-11-03 11:37:48.773243 7f2912507700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.773243, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -8775> 2014-11-03 11:37:48.773288 7f2912507700 5 -- op tracker -- seq: 2792, time: 2014-11-03 11:37:48.773287, event: done, op: osd_sub_op(client.7914641.0:33713263 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199783 snapset=0=[]:[] snapc=0=[]) -8774> 2014-11-03 11:37:48.773354 7f2912507700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.773353, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -8773> 2014-11-03 11:37:48.773395 7f2912507700 5 -- op tracker -- seq: 2793, time: 2014-11-03 11:37:48.773395, event: done, op: osd_sub_op(client.7868656.0:3476937 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900000 snapset=0=[]:[] snapc=0=[]) -8772> 2014-11-03 11:37:48.773556 7f2912507700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.773555, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -8771> 2014-11-03 11:37:48.773600 7f2912507700 5 -- op tracker -- seq: 2794, time: 2014-11-03 11:37:48.773600, event: done, op: osd_sub_op(client.7868656.0:3476938 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900001 snapset=0=[]:[] snapc=0=[]) -8770> 2014-11-03 11:37:48.773718 7f2912507700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.773717, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -8769> 2014-11-03 11:37:48.773762 7f2912507700 5 -- op tracker -- seq: 2795, time: 2014-11-03 11:37:48.773762, event: done, op: osd_sub_op(client.7914641.0:33713264 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199784 snapset=0=[]:[] snapc=0=[]) -8768> 2014-11-03 11:37:48.773864 7f2912507700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.773864, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -8767> 2014-11-03 11:37:48.773903 7f2912507700 5 -- op tracker -- seq: 2796, time: 2014-11-03 11:37:48.773903, event: done, op: osd_sub_op(client.7914641.0:33713287 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931196 snapset=0=[]:[] snapc=0=[]) -8766> 2014-11-03 11:37:48.773998 7f2912507700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.773998, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -8765> 2014-11-03 11:37:48.774024 7f2912507700 5 -- op tracker -- seq: 2797, time: 2014-11-03 11:37:48.774024, event: done, op: osd_sub_op(client.7914641.0:33713265 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199785 snapset=0=[]:[] snapc=0=[]) -8764> 2014-11-03 11:37:48.774173 7f2912507700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.774173, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -8763> 2014-11-03 11:37:48.774221 7f2912507700 5 -- op tracker -- seq: 2798, time: 2014-11-03 11:37:48.774220, event: done, op: osd_sub_op(client.7868656.0:3476939 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900002 snapset=0=[]:[] snapc=0=[]) -8762> 2014-11-03 11:37:48.774373 7f2912507700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.774373, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -8761> 2014-11-03 11:37:48.774407 7f2912507700 5 -- op tracker -- seq: 2799, time: 2014-11-03 11:37:48.774407, event: done, op: osd_sub_op(client.7868656.0:3476940 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900003 snapset=0=[]:[] snapc=0=[]) -8760> 2014-11-03 11:37:48.774523 7f2912507700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.774523, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -8759> 2014-11-03 11:37:48.774562 7f2912507700 5 -- op tracker -- seq: 2800, time: 2014-11-03 11:37:48.774562, event: done, op: osd_sub_op(client.7914641.0:33713288 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931197 snapset=0=[]:[] snapc=0=[]) -8758> 2014-11-03 11:37:48.774741 7f2912507700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.774741, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -8757> 2014-11-03 11:37:48.774779 7f2912507700 5 -- op tracker -- seq: 2801, time: 2014-11-03 11:37:48.774779, event: done, op: osd_sub_op(client.7868656.0:3476941 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900004 snapset=0=[]:[] snapc=0=[]) -8756> 2014-11-03 11:37:48.774881 7f2912507700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.774881, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -8755> 2014-11-03 11:37:48.774906 7f2912507700 5 -- op tracker -- seq: 2802, time: 2014-11-03 11:37:48.774906, event: done, op: osd_sub_op(client.7914641.0:33713289 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931198 snapset=0=[]:[] snapc=0=[]) -8754> 2014-11-03 11:37:48.775169 7f2912507700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.775169, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -8753> 2014-11-03 11:37:48.775213 7f2912507700 5 -- op tracker -- seq: 2803, time: 2014-11-03 11:37:48.775213, event: done, op: osd_sub_op(client.7914641.0:33713266 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199786 snapset=0=[]:[] snapc=0=[]) -8752> 2014-11-03 11:37:48.775404 7f2912507700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.775403, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -8751> 2014-11-03 11:37:48.775443 7f2912507700 5 -- op tracker -- seq: 2804, time: 2014-11-03 11:37:48.775443, event: done, op: osd_sub_op(client.7868656.0:3476942 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900005 snapset=0=[]:[] snapc=0=[]) -8750> 2014-11-03 11:37:48.775549 7f2912507700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.775548, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -8749> 2014-11-03 11:37:48.775594 7f2912507700 5 -- op tracker -- seq: 2805, time: 2014-11-03 11:37:48.775594, event: done, op: osd_sub_op(client.7868656.0:3476943 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900006 snapset=0=[]:[] snapc=0=[]) -8748> 2014-11-03 11:37:48.775704 7f2912507700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.775704, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -8747> 2014-11-03 11:37:48.775745 7f2912507700 5 -- op tracker -- seq: 2806, time: 2014-11-03 11:37:48.775745, event: done, op: osd_sub_op(client.7914641.0:33713290 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931199 snapset=0=[]:[] snapc=0=[]) -8746> 2014-11-03 11:37:48.775844 7f2912507700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.775844, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -8745> 2014-11-03 11:37:48.775882 7f2912507700 5 -- op tracker -- seq: 2807, time: 2014-11-03 11:37:48.775882, event: done, op: osd_sub_op(client.7868656.0:3476944 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900007 snapset=0=[]:[] snapc=0=[]) -8744> 2014-11-03 11:37:48.776003 7f2912507700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.776002, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -8743> 2014-11-03 11:37:48.776031 7f2912507700 5 -- op tracker -- seq: 2808, time: 2014-11-03 11:37:48.776031, event: done, op: osd_sub_op(client.7914641.0:33713267 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199787 snapset=0=[]:[] snapc=0=[]) -8742> 2014-11-03 11:37:48.776150 7f2912507700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.776150, event: sub_op_applied, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -8741> 2014-11-03 11:37:48.776176 7f2912507700 5 -- op tracker -- seq: 2809, time: 2014-11-03 11:37:48.776176, event: done, op: osd_sub_op(client.7868656.0:3476945 6.690 bfbab690/rbd_data.509c0f2ae8944a.0000000000000400/head//6 [] v 105950'28900008 snapset=0=[]:[] snapc=0=[]) -8740> 2014-11-03 11:37:48.776317 7f2912507700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.776316, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -8739> 2014-11-03 11:37:48.776364 7f2912507700 5 -- op tracker -- seq: 2810, time: 2014-11-03 11:37:48.776363, event: done, op: osd_sub_op(client.7914641.0:33713291 6.a23 cf6fea23/rbd_data.49e9f52ae8944a.00000000000011c4/head//6 [] v 105950'7931200 snapset=0=[]:[] snapc=0=[]) -8738> 2014-11-03 11:37:48.776656 7f2912507700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.776656, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -8737> 2014-11-03 11:37:48.776705 7f2912507700 5 -- op tracker -- seq: 2811, time: 2014-11-03 11:37:48.776705, event: done, op: osd_sub_op(client.7914641.0:33713268 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199788 snapset=0=[]:[] snapc=0=[]) -8736> 2014-11-03 11:37:48.776905 7f2912507700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.776905, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -8735> 2014-11-03 11:37:48.776945 7f2912507700 5 -- op tracker -- seq: 2812, time: 2014-11-03 11:37:48.776945, event: done, op: osd_sub_op(client.7914641.0:33713269 6.31f 2d19531f/rbd_data.49e9f52ae8944a.00000000000011c3/head//6 [] v 105950'18199789 snapset=0=[]:[] snapc=0=[]) -8734> 2014-11-03 11:37:48.777201 7f2912507700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.777201, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -8733> 2014-11-03 11:37:48.777255 7f2912507700 5 -- op tracker -- seq: 2813, time: 2014-11-03 11:37:48.777255, event: done, op: osd_sub_op(client.7914641.0:33713395 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649536 snapset=0=[]:[] snapc=0=[]) -8732> 2014-11-03 11:37:48.777447 7f2912507700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.777447, event: sub_op_applied, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -8731> 2014-11-03 11:37:48.777490 7f2912507700 5 -- op tracker -- seq: 2814, time: 2014-11-03 11:37:48.777489, event: done, op: osd_sub_op(client.7868825.0:12014090 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502024 snapset=0=[]:[] snapc=0=[]) -8730> 2014-11-03 11:37:48.777676 7f2912507700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.777676, event: sub_op_applied, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -8729> 2014-11-03 11:37:48.777718 7f2912507700 5 -- op tracker -- seq: 2815, time: 2014-11-03 11:37:48.777718, event: done, op: osd_sub_op(client.7868825.0:12014091 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502025 snapset=0=[]:[] snapc=0=[]) -8728> 2014-11-03 11:37:48.777923 7f2912507700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.777923, event: sub_op_applied, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -8727> 2014-11-03 11:37:48.777963 7f2912507700 5 -- op tracker -- seq: 2816, time: 2014-11-03 11:37:48.777962, event: done, op: osd_sub_op(client.7868825.0:12014092 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502026 snapset=0=[]:[] snapc=0=[]) -8726> 2014-11-03 11:37:48.778183 7f2912507700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.778182, event: sub_op_applied, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -8725> 2014-11-03 11:37:48.778221 7f2912507700 5 -- op tracker -- seq: 2817, time: 2014-11-03 11:37:48.778221, event: done, op: osd_sub_op(client.7869041.0:12286803 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900009 snapset=0=[]:[] snapc=0=[]) -8724> 2014-11-03 11:37:48.778496 7f2912507700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.778496, event: sub_op_applied, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -8723> 2014-11-03 11:37:48.778545 7f2912507700 5 -- op tracker -- seq: 2818, time: 2014-11-03 11:37:48.778545, event: done, op: osd_sub_op(client.7870481.0:43903885 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953126 snapset=0=[]:[] snapc=0=[]) -8722> 2014-11-03 11:37:48.778761 7f2912507700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.778760, event: sub_op_applied, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -8721> 2014-11-03 11:37:48.778814 7f2912507700 5 -- op tracker -- seq: 2819, time: 2014-11-03 11:37:48.778814, event: done, op: osd_sub_op(client.11363859.0:247720 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000601 snapset=0=[]:[] snapc=0=[]) -8720> 2014-11-03 11:37:48.779023 7f2912507700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.779022, event: sub_op_applied, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -8719> 2014-11-03 11:37:48.779066 7f2912507700 5 -- op tracker -- seq: 2820, time: 2014-11-03 11:37:48.779066, event: done, op: osd_sub_op(client.7869083.0:23485453 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795118 snapset=0=[]:[] snapc=0=[]) -8718> 2014-11-03 11:37:48.779291 7f2912507700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.779291, event: sub_op_applied, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -8717> 2014-11-03 11:37:48.779329 7f2912507700 5 -- op tracker -- seq: 2823, time: 2014-11-03 11:37:48.779329, event: done, op: osd_sub_op(client.7912219.0:1154960 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187819 snapset=0=[]:[] snapc=0=[]) -8716> 2014-11-03 11:37:48.779525 7f2912507700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.779525, event: sub_op_applied, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8715> 2014-11-03 11:37:48.779590 7f2912507700 5 -- op tracker -- seq: 2821, time: 2014-11-03 11:37:48.779589, event: done, op: osd_sub_op(client.7869083.0:23485454 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795119 snapset=0=[]:[] snapc=0=[]) -8714> 2014-11-03 11:37:48.779778 7f2912507700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.779778, event: sub_op_applied, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8713> 2014-11-03 11:37:48.779814 7f2912507700 5 -- op tracker -- seq: 2822, time: 2014-11-03 11:37:48.779814, event: done, op: osd_sub_op(client.7869083.0:23485455 6.3b3 4bbe1bb3/rbd_data.4f607c22a87210.00000000000000e2/head//6 [] v 105950'11795120 snapset=0=[]:[] snapc=0=[]) -8712> 2014-11-03 11:37:48.790792 7f28e9326700 1 -- 192.168.5.2:6802/7606 <== client.9598152 192.168.9.2:0/1027858 10 ==== osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) v4 ==== 229+0+4096 (432475747 0 3331964654) 0x7c35500 con 0x2311c9a0 -8711> 2014-11-03 11:37:48.790855 7f28e9326700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.790666, event: header_read, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8710> 2014-11-03 11:37:48.790886 7f28e9326700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.790672, event: throttled, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8709> 2014-11-03 11:37:48.790921 7f28e9326700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.790775, event: all_read, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8708> 2014-11-03 11:37:48.790941 7f28e9326700 5 -- op tracker -- seq: 2828, time: 0.000000, event: dispatched, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8707> 2014-11-03 11:37:48.791082 7f290101d700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.791082, event: reached_pg, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8706> 2014-11-03 11:37:48.791191 7f290101d700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.791191, event: started, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8705> 2014-11-03 11:37:48.791216 7f290101d700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.791216, event: started, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8704> 2014-11-03 11:37:48.791277 7f290101d700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.791277, event: waiting for subops from 45,107, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8703> 2014-11-03 11:37:48.791312 7f290101d700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131094 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4925 0x24867800 con 0x22ff3420 -8702> 2014-11-03 11:37:48.791389 7f290101d700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6833/17474 -- osd_sub_op(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131094 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4925 0x3125bc00 con 0x22fe0dc0 -8701> 2014-11-03 11:37:48.791498 7f290101d700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.791498, event: commit_queued_for_journal_write, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8700> 2014-11-03 11:37:48.792916 7f28f52de700 1 -- 192.168.5.2:6803/7606 <== osd.107 192.168.5.8:6833/17474 33 ==== osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (954060464 0 0) 0x28a7c840 con 0x22fe0dc0 -8699> 2014-11-03 11:37:48.792965 7f28f52de700 5 -- op tracker -- seq: 2829, time: 2014-11-03 11:37:48.792822, event: header_read, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8698> 2014-11-03 11:37:48.792993 7f28f52de700 5 -- op tracker -- seq: 2829, time: 2014-11-03 11:37:48.792824, event: throttled, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8697> 2014-11-03 11:37:48.793012 7f28f52de700 5 -- op tracker -- seq: 2829, time: 2014-11-03 11:37:48.792903, event: all_read, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8696> 2014-11-03 11:37:48.793030 7f28f52de700 5 -- op tracker -- seq: 2829, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8695> 2014-11-03 11:37:48.793105 7f28fe818700 5 -- op tracker -- seq: 2829, time: 2014-11-03 11:37:48.793104, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8694> 2014-11-03 11:37:48.793153 7f28fe818700 5 -- op tracker -- seq: 2829, time: 2014-11-03 11:37:48.793153, event: started, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8693> 2014-11-03 11:37:48.793186 7f28fe818700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.793186, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8692> 2014-11-03 11:37:48.793227 7f28fe818700 5 -- op tracker -- seq: 2829, time: 2014-11-03 11:37:48.793227, event: done, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8691> 2014-11-03 11:37:48.793697 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 34 ==== osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (1604322031 0 0) 0x2fcad8c0 con 0x22ff3420 -8690> 2014-11-03 11:37:48.793752 7f28f0b97700 5 -- op tracker -- seq: 2830, time: 2014-11-03 11:37:48.793585, event: header_read, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8689> 2014-11-03 11:37:48.793774 7f28f0b97700 5 -- op tracker -- seq: 2830, time: 2014-11-03 11:37:48.793589, event: throttled, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8688> 2014-11-03 11:37:48.793786 7f28f0b97700 5 -- op tracker -- seq: 2830, time: 2014-11-03 11:37:48.793674, event: all_read, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8687> 2014-11-03 11:37:48.793797 7f28f0b97700 5 -- op tracker -- seq: 2830, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8686> 2014-11-03 11:37:48.793896 7f290101d700 5 -- op tracker -- seq: 2830, time: 2014-11-03 11:37:48.793896, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8685> 2014-11-03 11:37:48.793949 7f290101d700 5 -- op tracker -- seq: 2830, time: 2014-11-03 11:37:48.793948, event: started, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8684> 2014-11-03 11:37:48.793975 7f290101d700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.793975, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8683> 2014-11-03 11:37:48.794010 7f290101d700 5 -- op tracker -- seq: 2830, time: 2014-11-03 11:37:48.794010, event: done, op: osd_sub_op_reply(client.9598152.0:459474 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -8682> 2014-11-03 11:37:48.818259 7f291450b700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.818259, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8681> 2014-11-03 11:37:48.818301 7f291450b700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.818301, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8680> 2014-11-03 11:37:48.818321 7f291450b700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.818321, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8679> 2014-11-03 11:37:48.818335 7f2911d06700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.818335, event: commit_sent, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8678> 2014-11-03 11:37:48.818396 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2494b000 con 0x22fefdc0 -8677> 2014-11-03 11:37:48.818409 7f2914d0c700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.818408, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8676> 2014-11-03 11:37:48.818446 7f2911d06700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.818445, event: commit_sent, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8675> 2014-11-03 11:37:48.818469 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2650d840 con 0x22fefdc0 -8674> 2014-11-03 11:37:48.818463 7f2914d0c700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.818463, event: write_thread_in_journal_buffer, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8673> 2014-11-03 11:37:48.818672 7f2911d06700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.818672, event: commit_sent, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8672> 2014-11-03 11:37:48.818741 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2dfba000 con 0x22ff3b00 -8671> 2014-11-03 11:37:48.818775 7f2912507700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.818774, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8670> 2014-11-03 11:37:48.818816 7f2912507700 5 -- op tracker -- seq: 2824, time: 2014-11-03 11:37:48.818816, event: done, op: osd_sub_op(client.8703713.0:32254337 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953127 snapset=0=[]:[] snapc=0=[]) -8669> 2014-11-03 11:37:48.818945 7f2912507700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.818944, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8668> 2014-11-03 11:37:48.818990 7f2912507700 5 -- op tracker -- seq: 2825, time: 2014-11-03 11:37:48.818990, event: done, op: osd_sub_op(client.8703713.0:32254338 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953128 snapset=0=[]:[] snapc=0=[]) -8667> 2014-11-03 11:37:48.819111 7f2912507700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.819110, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8666> 2014-11-03 11:37:48.819162 7f2912507700 5 -- op tracker -- seq: 2826, time: 2014-11-03 11:37:48.819162, event: done, op: osd_sub_op(client.12146205.0:439297 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222064 snapset=0=[]:[] snapc=0=[]) -8665> 2014-11-03 11:37:48.821527 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.7 192.168.5.1:0/3511 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.821146) v2 ==== 47+0+0 (59955474 0 0) 0x2372b200 con 0x24925000 -8664> 2014-11-03 11:37:48.821579 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.821146) v2 -- ?+0 0x2350ae00 con 0x24925000 -8663> 2014-11-03 11:37:48.821649 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.7 192.168.5.1:0/3511 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.821146) v2 ==== 47+0+0 (59955474 0 0) 0x22fd4400 con 0x23ea4c60 -8662> 2014-11-03 11:37:48.821746 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.821146) v2 -- ?+0 0x23323a00 con 0x23ea4c60 -8661> 2014-11-03 11:37:48.822225 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 147 ==== osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2671666291 0 3511520646) 0x2799a800 con 0x22fefdc0 -8660> 2014-11-03 11:37:48.822275 7f28f36c2700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.821926, event: header_read, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8659> 2014-11-03 11:37:48.822305 7f28f36c2700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.821929, event: throttled, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8658> 2014-11-03 11:37:48.822349 7f28f36c2700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.822206, event: all_read, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8657> 2014-11-03 11:37:48.822371 7f28f36c2700 5 -- op tracker -- seq: 2831, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8656> 2014-11-03 11:37:48.822498 7f28fe818700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.822498, event: reached_pg, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8655> 2014-11-03 11:37:48.822535 7f28fe818700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.822535, event: started, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8654> 2014-11-03 11:37:48.822630 7f28fe818700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.822630, event: started, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8653> 2014-11-03 11:37:48.822669 7f28fe818700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.822669, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8652> 2014-11-03 11:37:48.840468 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x23bd8580 sd=97 :6802 s=0 pgs=0 cs=0 l=0 c=0x24537840).accept sd=97 192.168.5.2:42834/0 -8651> 2014-11-03 11:37:48.841560 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450659 192.168.5.2:0/1026349 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x242e2e00 con 0x24537840 -8650> 2014-11-03 11:37:48.842248 7f28fb812700 0 osd.21 105950 do_command r=0 -8649> 2014-11-03 11:37:48.842282 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1026349 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x4e2a1c0 con 0x24537840 -8648> 2014-11-03 11:37:48.851016 7f28ead39700 1 -- 192.168.5.2:6803/7606 <== osd.28 192.168.5.3:6812/13976 32 ==== osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2770692200 0 3511520646) 0x2cff3000 con 0x23157dc0 -8647> 2014-11-03 11:37:48.851087 7f28ead39700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.850831, event: header_read, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8646> 2014-11-03 11:37:48.851114 7f28ead39700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.850837, event: throttled, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8645> 2014-11-03 11:37:48.851123 7f28ead39700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.850990, event: all_read, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8644> 2014-11-03 11:37:48.851156 7f28ead39700 5 -- op tracker -- seq: 2832, time: 0.000000, event: dispatched, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8643> 2014-11-03 11:37:48.851265 7f290201f700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.851265, event: reached_pg, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8642> 2014-11-03 11:37:48.851302 7f290201f700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.851302, event: started, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8641> 2014-11-03 11:37:48.851415 7f290201f700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.851415, event: started, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8640> 2014-11-03 11:37:48.851452 7f290201f700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.851452, event: commit_queued_for_journal_write, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8639> 2014-11-03 11:37:48.854723 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450659 192.168.5.2:0/1026349 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2b1ff000 con 0x24537840 -8638> 2014-11-03 11:37:48.855637 7f28fb812700 0 osd.21 105950 do_command r=0 -8637> 2014-11-03 11:37:48.855657 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1026349 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0xccda540 con 0x24537840 -8636> 2014-11-03 11:37:48.861493 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1026349 pipe(0x23bd8580 sd=97 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537840).reader couldn't read tag, (0) Success -8635> 2014-11-03 11:37:48.861545 7f28c4e00700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1026349 pipe(0x23bd8580 sd=97 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537840).fault (0) Success -8634> 2014-11-03 11:37:48.861655 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x24537840 session 0x7c35780 -8633> 2014-11-03 11:37:48.870404 7f291450b700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.870403, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8632> 2014-11-03 11:37:48.870477 7f2911d06700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.870477, event: commit_sent, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8631> 2014-11-03 11:37:48.870495 7f291450b700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.870494, event: journaled_completion_queued, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8630> 2014-11-03 11:37:48.870520 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6807/16411 -- osd_sub_op_reply(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfba580 con 0x230c29a0 -8629> 2014-11-03 11:37:48.870559 7f2911d06700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.870558, event: op_commit, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8628> 2014-11-03 11:37:48.870579 7f2914d0c700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.870579, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8627> 2014-11-03 11:37:48.870602 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.2:0/1027858 -- osd_op_reply(459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] v105950'131094 uv131094 ondisk = 0) v6 -- ?+0 0x2fc9bb80 con 0x2311c9a0 -8626> 2014-11-03 11:37:48.870620 7f2914d0c700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.870620, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8625> 2014-11-03 11:37:48.870634 7f2911d06700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.870634, event: commit_sent, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8624> 2014-11-03 11:37:48.871045 7f2912507700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.871045, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8623> 2014-11-03 11:37:48.871114 7f2912507700 5 -- op tracker -- seq: 2827, time: 2014-11-03 11:37:48.871113, event: done, op: osd_sub_op(client.8236383.0:16630616 6.167 a1215167/rbd_data.7e655d2ae8944a.0000000000003d28/head//6 [] v 105950'5727269 snapset=0=[]:[] snapc=0=[]) -8622> 2014-11-03 11:37:48.871652 7f2912507700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.871651, event: op_applied, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8621> 2014-11-03 11:37:48.871744 7f2912507700 5 -- op tracker -- seq: 2828, time: 2014-11-03 11:37:48.871744, event: done, op: osd_op(client.9598152.0:459474 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3084288~4096] 24.8f258d3e ack+ondisk+write e105950) -8620> 2014-11-03 11:37:48.880295 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.71 192.168.5.6:0/14378 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.879569) v2 ==== 47+0+0 (2214990315 0 0) 0x13defc00 con 0x233352c0 -8619> 2014-11-03 11:37:48.880357 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.71 192.168.5.6:0/14378 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.879569) v2 ==== 47+0+0 (2214990315 0 0) 0x2a25b200 con 0x234ed2c0 -8618> 2014-11-03 11:37:48.880392 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.879569) v2 -- ?+0 0x2372b200 con 0x233352c0 -8617> 2014-11-03 11:37:48.880480 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.879569) v2 -- ?+0 0x22fd4400 con 0x234ed2c0 -8616> 2014-11-03 11:37:48.894917 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.52 192.168.5.4:0/3307 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.893534) v2 ==== 47+0+0 (1582034888 0 0) 0x23408200 con 0x242c0000 -8615> 2014-11-03 11:37:48.894991 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/3307 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.893534) v2 -- ?+0 0x13defc00 con 0x242c0000 -8614> 2014-11-03 11:37:48.894995 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.52 192.168.5.4:0/3307 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.893534) v2 ==== 47+0+0 (1582034888 0 0) 0x5f22000 con 0x2469e9a0 -8613> 2014-11-03 11:37:48.895141 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/3307 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.893534) v2 -- ?+0 0x2a25b200 con 0x2469e9a0 -8612> 2014-11-03 11:37:48.903760 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.60 192.168.5.5:0/5983 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.902475) v2 ==== 47+0+0 (645261581 0 0) 0x226b5800 con 0x2445c9a0 -8611> 2014-11-03 11:37:48.903809 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.902475) v2 -- ?+0 0x23408200 con 0x2445c9a0 -8610> 2014-11-03 11:37:48.903806 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.60 192.168.5.5:0/5983 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.902475) v2 ==== 47+0+0 (645261581 0 0) 0x2355b400 con 0x23ea4b00 -8609> 2014-11-03 11:37:48.903874 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.902475) v2 -- ?+0 0x5f22000 con 0x23ea4b00 -8608> 2014-11-03 11:37:48.908049 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.118 192.168.5.9:0/23392 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.906433) v2 ==== 47+0+0 (27677782 0 0) 0x8664a00 con 0x2469ec60 -8607> 2014-11-03 11:37:48.908101 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/23392 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.906433) v2 -- ?+0 0x2355b400 con 0x2469ec60 -8606> 2014-11-03 11:37:48.908208 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.118 192.168.5.9:0/23392 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:48.906433) v2 ==== 47+0+0 (27677782 0 0) 0x2b1ff600 con 0x23c15000 -8605> 2014-11-03 11:37:48.908263 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/23392 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:48.906433) v2 -- ?+0 0x226b5800 con 0x23c15000 -8604> 2014-11-03 11:37:48.971000 7f291450b700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.970999, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8603> 2014-11-03 11:37:48.971109 7f291450b700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.971109, event: journaled_completion_queued, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8602> 2014-11-03 11:37:48.971117 7f2911d06700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.971116, event: commit_sent, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8601> 2014-11-03 11:37:48.971170 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x258be8c0 con 0x22fefdc0 -8600> 2014-11-03 11:37:48.971223 7f2911d06700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.971223, event: commit_sent, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8599> 2014-11-03 11:37:48.971256 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6812/13976 -- osd_sub_op_reply(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bdb00 con 0x23157dc0 -8598> 2014-11-03 11:37:48.971721 7f2912507700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.971721, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8597> 2014-11-03 11:37:48.971791 7f2912507700 5 -- op tracker -- seq: 2831, time: 2014-11-03 11:37:48.971791, event: done, op: osd_sub_op(client.8703713.0:32254342 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953129 snapset=0=[]:[] snapc=0=[]) -8596> 2014-11-03 11:37:48.972151 7f2912507700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.972150, event: sub_op_applied, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8595> 2014-11-03 11:37:48.972261 7f2912507700 5 -- op tracker -- seq: 2832, time: 2014-11-03 11:37:48.972261, event: done, op: osd_sub_op(client.7907476.0:2631964 6.7e9 17f8c7e9/rbd_data.4cfea248a30bc6.0000000000001537/head//6 [] v 105950'8627705 snapset=0=[]:[] snapc=0=[]) -8594> 2014-11-03 11:37:48.987077 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 148 ==== osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3431867752 0 3511520646) 0x26151000 con 0x22fefdc0 -8593> 2014-11-03 11:37:48.987114 7f28f36c2700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.986921, event: header_read, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8592> 2014-11-03 11:37:48.987143 7f28f36c2700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.986923, event: throttled, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8591> 2014-11-03 11:37:48.987157 7f28f36c2700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.987062, event: all_read, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8590> 2014-11-03 11:37:48.987173 7f28f36c2700 5 -- op tracker -- seq: 2833, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8589> 2014-11-03 11:37:48.987302 7f290101d700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.987302, event: reached_pg, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8588> 2014-11-03 11:37:48.987419 7f290101d700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.987419, event: started, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8587> 2014-11-03 11:37:48.987660 7f290101d700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.987660, event: started, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8586> 2014-11-03 11:37:48.987736 7f290101d700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.987736, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8585> 2014-11-03 11:37:48.987902 7f2914d0c700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.987901, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8584> 2014-11-03 11:37:48.988260 7f291450b700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.988259, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8583> 2014-11-03 11:37:48.988396 7f2911d06700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.988395, event: commit_sent, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8582> 2014-11-03 11:37:48.988463 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x258bd000 con 0x22fefdc0 -8581> 2014-11-03 11:37:48.988849 7f2912507700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.988848, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8580> 2014-11-03 11:37:48.988917 7f2912507700 5 -- op tracker -- seq: 2833, time: 2014-11-03 11:37:48.988917, event: done, op: osd_sub_op(client.8703713.0:32254344 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953130 snapset=0=[]:[] snapc=0=[]) -8579> 2014-11-03 11:37:48.989826 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x29fcd000 con 0x22fbc420 -8578> 2014-11-03 11:37:48.993007 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 149 ==== osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3101748050 0 3511520646) 0x28fef400 con 0x22fefdc0 -8577> 2014-11-03 11:37:48.993083 7f28f36c2700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.992830, event: header_read, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8576> 2014-11-03 11:37:48.993128 7f28f36c2700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.992833, event: throttled, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8575> 2014-11-03 11:37:48.993176 7f28f36c2700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.992985, event: all_read, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8574> 2014-11-03 11:37:48.993190 7f28f36c2700 5 -- op tracker -- seq: 2834, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8573> 2014-11-03 11:37:48.993283 7f28fe818700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.993282, event: reached_pg, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8572> 2014-11-03 11:37:48.993351 7f28fe818700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.993351, event: started, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8571> 2014-11-03 11:37:48.993483 7f28fe818700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.993483, event: started, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8570> 2014-11-03 11:37:48.993532 7f28fe818700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.993532, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8569> 2014-11-03 11:37:48.993660 7f2914d0c700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.993660, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8568> 2014-11-03 11:37:48.993979 7f291450b700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.993978, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8567> 2014-11-03 11:37:48.994080 7f2911d06700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.994080, event: commit_sent, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8566> 2014-11-03 11:37:48.994179 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x269b0600 con 0x22fefdc0 -8565> 2014-11-03 11:37:48.994420 7f2912507700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.994420, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8564> 2014-11-03 11:37:48.994497 7f2912507700 5 -- op tracker -- seq: 2834, time: 2014-11-03 11:37:48.994497, event: done, op: osd_sub_op(client.8703713.0:32254346 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953131 snapset=0=[]:[] snapc=0=[]) -8563> 2014-11-03 11:37:49.003139 7f28fb011700 5 osd.21 105950 heartbeat: osd_stat(1844 GB used, 1867 GB avail, 3712 GB total, peers [1,3,4,6,7,8,10,12,13,20,23,28,30,31,32,33,35,37,40,42,43,44,45,46,48,49,50,54,55,61,66,68,69,70,75,76,77,81,82,83,90,93,97,98,99,102,103,104,105,107,108,109,110,112,113,114,116,118,121,122,123,124,125]/[] op hist []) -8562> 2014-11-03 11:37:49.003184 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6808/11054 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2369fc00 con 0x239f4dc0 -8561> 2014-11-03 11:37:49.003235 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6809/11054 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2369fa00 con 0x239f4c60 -8560> 2014-11-03 11:37:49.003271 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6858/28138 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x24800600 con 0x2445a2c0 -8559> 2014-11-03 11:37:49.003298 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6860/28138 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x24800c00 con 0x2445a160 -8558> 2014-11-03 11:37:49.003329 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6868/31636 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x24800800 con 0x2439c000 -8557> 2014-11-03 11:37:49.003386 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6869/31636 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x240f8200 con 0x2445ac60 -8556> 2014-11-03 11:37:49.003415 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6882/1181 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x240f8800 con 0x233da580 -8555> 2014-11-03 11:37:49.003454 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6883/1181 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23430600 con 0x233da420 -8554> 2014-11-03 11:37:49.003480 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6887/3511 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23430800 con 0x233da000 -8553> 2014-11-03 11:37:49.003508 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6888/3511 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23430e00 con 0x233da2c0 -8552> 2014-11-03 11:37:49.003542 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6893/8894 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23430200 con 0x23d636e0 -8551> 2014-11-03 11:37:49.003576 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6894/8894 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23430400 con 0x23d63580 -8550> 2014-11-03 11:37:49.003614 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6817/15344 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23e31400 con 0x23d63c60 -8549> 2014-11-03 11:37:49.003643 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6821/15344 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23e31c00 con 0x23d63b00 -8548> 2014-11-03 11:37:49.003677 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6829/19844 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23e31600 con 0x2416eb00 -8547> 2014-11-03 11:37:49.003711 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6847/19844 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23e31e00 con 0x2416e9a0 -8546> 2014-11-03 11:37:49.003737 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6840/22281 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23e31200 con 0x23ea02c0 -8545> 2014-11-03 11:37:49.003757 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6852/22281 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23e31a00 con 0x23ea0160 -8544> 2014-11-03 11:37:49.003806 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6834/4963 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x26bdbc00 con 0x2342c420 -8543> 2014-11-03 11:37:49.003859 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6837/4963 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x1adc4200 con 0x2342c2c0 -8542> 2014-11-03 11:37:49.003887 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6852/14826 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xaf2e200 con 0x2342c160 -8541> 2014-11-03 11:37:49.003911 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6853/14826 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xaf2e600 con 0x2342c000 -8540> 2014-11-03 11:37:49.003952 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6813/13976 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xaf2ea00 con 0x2445ab00 -8539> 2014-11-03 11:37:49.003987 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6817/13976 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x147be200 con 0x2445a9a0 -8538> 2014-11-03 11:37:49.003982 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.1 192.168.5.1:6808/11054 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x240f8c00 con 0x239f4dc0 -8537> 2014-11-03 11:37:49.004025 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6818/11504 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x30fde00 con 0x2387dc60 -8536> 2014-11-03 11:37:49.004053 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6820/11504 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2be31e00 con 0x2387db00 -8535> 2014-11-03 11:37:49.004086 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6826/20618 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x80d2c00 con 0x233d2c60 -8534> 2014-11-03 11:37:49.004130 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6827/20618 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x80d2400 con 0x233d2b00 -8533> 2014-11-03 11:37:49.004164 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6830/25327 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x80d2000 con 0x233dac60 -8532> 2014-11-03 11:37:49.004205 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6831/25327 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2b1ffa00 con 0x233dab00 -8531> 2014-11-03 11:37:49.004241 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6840/29378 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23408200 con 0x2387fb00 -8530> 2014-11-03 11:37:49.004273 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6841/29378 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23354c00 con 0x2387f9a0 -8529> 2014-11-03 11:37:49.004319 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6847/6455 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23a9ec00 con 0x2387d160 -8528> 2014-11-03 11:37:49.004353 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6848/6455 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x5f22000 con 0x2387d000 -8527> 2014-11-03 11:37:49.004396 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6858/13652 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2a347200 con 0x2387fdc0 -8526> 2014-11-03 11:37:49.004427 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6859/13652 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x243ff400 con 0x2387fc60 -8525> 2014-11-03 11:37:49.004477 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6869/25675 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xde23000 con 0x235f99a0 -8524> 2014-11-03 11:37:49.004512 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6870/25675 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x6526e00 con 0x235f9840 -8523> 2014-11-03 11:37:49.004542 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6808/9946 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x6526c00 con 0x23ea0580 -8522> 2014-11-03 11:37:49.004575 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6809/9946 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x6526a00 con 0x23ea0420 -8521> 2014-11-03 11:37:49.004652 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6813/19461 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126e00 con 0x233da160 -8520> 2014-11-03 11:37:49.004706 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6814/19461 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126c00 con 0x233da6e0 -8519> 2014-11-03 11:37:49.004753 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6817/27523 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126a00 con 0x2416e000 -8518> 2014-11-03 11:37:49.004782 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6818/27523 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126800 con 0x24448420 -8517> 2014-11-03 11:37:49.004806 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6822/1781 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126600 con 0x2342c9a0 -8516> 2014-11-03 11:37:49.004828 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6823/1781 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126400 con 0x2342c840 -8515> 2014-11-03 11:37:49.004861 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6827/7678 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126200 con 0x245abc60 -8514> 2014-11-03 11:37:49.004887 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6828/7678 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25126000 con 0x245abb00 -8513> 2014-11-03 11:37:49.004914 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6837/17727 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329ee00 con 0x23ea0000 -8512> 2014-11-03 11:37:49.004947 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6839/17727 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329ec00 con 0x2387ddc0 -8511> 2014-11-03 11:37:49.004997 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6843/21769 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329ea00 con 0x244482c0 -8510> 2014-11-03 11:37:49.005037 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6844/21769 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329e800 con 0x23eb86e0 -8509> 2014-11-03 11:37:49.005078 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6848/26902 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329e600 con 0x245ab9a0 -8508> 2014-11-03 11:37:49.005112 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6849/26902 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329e400 con 0x245ab840 -8507> 2014-11-03 11:37:49.005157 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6869/8391 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329e200 con 0x242056e0 -8506> 2014-11-03 11:37:49.005206 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6870/8391 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x2329e000 con 0x9921000 -8505> 2014-11-03 11:37:49.005248 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6872/12368 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23ff5e00 con 0x233d2580 -8504> 2014-11-03 11:37:49.005277 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6873/12368 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23ff5c00 con 0x233d2420 -8503> 2014-11-03 11:37:49.005304 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6832/12481 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23ff5a00 con 0x239f42c0 -8502> 2014-11-03 11:37:49.005332 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6833/12481 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5e00 con 0x239f4160 -8501> 2014-11-03 11:37:49.005356 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6853/919 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5c00 con 0x233d22c0 -8500> 2014-11-03 11:37:49.005382 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6859/919 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5a00 con 0x233d2160 -8499> 2014-11-03 11:37:49.005424 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6867/8133 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5800 con 0x245ab420 -8498> 2014-11-03 11:37:49.005449 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6868/8133 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5600 con 0x245ab2c0 -8497> 2014-11-03 11:37:49.005486 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6872/11297 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5400 con 0x23eb8580 -8496> 2014-11-03 11:37:49.005507 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6873/11297 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5200 con 0x23eb8420 -8495> 2014-11-03 11:37:49.005530 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6809/3622 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x71f5000 con 0x239f4b00 -8494> 2014-11-03 11:37:49.005560 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6810/3622 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23f8ce00 con 0x239f49a0 -8493> 2014-11-03 11:37:49.005604 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6832/12612 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25e25e00 con 0x2445a840 -8492> 2014-11-03 11:37:49.005635 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6833/12612 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25e25c00 con 0x2445a6e0 -8491> 2014-11-03 11:37:49.005670 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6837/22493 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25e25a00 con 0x23d639a0 -8490> 2014-11-03 11:37:49.005728 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6838/22493 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xa053e00 con 0x23d63840 -8489> 2014-11-03 11:37:49.005756 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6842/32590 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xa053c00 con 0x239f4000 -8488> 2014-11-03 11:37:49.005804 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6843/32590 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xa053a00 con 0x23d63dc0 -8487> 2014-11-03 11:37:49.005831 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6863/27170 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x17cace00 con 0x23d63160 -8486> 2014-11-03 11:37:49.005872 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6864/27170 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x17cacc00 con 0x23d63000 -8485> 2014-11-03 11:37:49.005904 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6867/31491 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x17caca00 con 0x235f9c60 -8484> 2014-11-03 11:37:49.005932 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6869/31491 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23d2be00 con 0x235f9b00 -8483> 2014-11-03 11:37:49.006029 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6872/4005 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23d2bc00 con 0x2387d9a0 -8482> 2014-11-03 11:37:49.006066 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6873/4005 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x23d2ba00 con 0x2387d840 -8481> 2014-11-03 11:37:49.006120 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6832/27918 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xc02ee00 con 0x233d29a0 -8480> 2014-11-03 11:37:49.006167 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6833/27918 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x9591e00 con 0x233d2840 -8479> 2014-11-03 11:37:49.006195 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6842/7095 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x9591c00 con 0x235f96e0 -8478> 2014-11-03 11:37:49.006257 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6843/7095 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x9591a00 con 0x235f92c0 -8477> 2014-11-03 11:37:49.006288 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6862/21285 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x619fe00 con 0x233d2000 -8476> 2014-11-03 11:37:49.006196 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 150 ==== osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1956485319 0 3511520646) 0x24662800 con 0x22fefdc0 -8475> 2014-11-03 11:37:49.006311 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6863/21285 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x619fc00 con 0x233d2dc0 -8474> 2014-11-03 11:37:49.006345 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6821/14722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x619fa00 con 0x239f4580 -8473> 2014-11-03 11:37:49.006379 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6823/14722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1ae00 con 0x239f4420 -8472> 2014-11-03 11:37:49.006308 7f28f36c2700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.005973, event: header_read, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8471> 2014-11-03 11:37:49.006404 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6831/26930 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1ac00 con 0x2387d420 -8470> 2014-11-03 11:37:49.006469 7f28f36c2700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.005975, event: throttled, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8469> 2014-11-03 11:37:49.006484 7f28f36c2700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.006181, event: all_read, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8468> 2014-11-03 11:37:49.006499 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6832/26930 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1aa00 con 0x2387d2c0 -8467> 2014-11-03 11:37:49.006502 7f28f36c2700 5 -- op tracker -- seq: 2835, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8466> 2014-11-03 11:37:49.006530 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6812/21153 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1a800 con 0x23ea0b00 -8465> 2014-11-03 11:37:49.006559 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6814/21153 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1a600 con 0x23ea09a0 -8464> 2014-11-03 11:37:49.006588 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6827/29722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1a400 con 0x233da9a0 -8463> 2014-11-03 11:37:49.006614 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6830/29722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1a200 con 0x233da840 -8462> 2014-11-03 11:37:49.006608 7f290101d700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.006607, event: reached_pg, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8461> 2014-11-03 11:37:49.006668 7f290101d700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.006668, event: started, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8460> 2014-11-03 11:37:49.006698 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6843/2576 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10c1a000 con 0x245ab160 -8459> 2014-11-03 11:37:49.006744 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6844/2576 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x70a2e00 con 0x245ab000 -8458> 2014-11-03 11:37:49.006782 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6841/6777 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x70a2c00 con 0x2387d6e0 -8457> 2014-11-03 11:37:49.006821 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6842/6777 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x70a2a00 con 0x2387d580 -8456> 2014-11-03 11:37:49.006820 7f290101d700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.006820, event: started, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8455> 2014-11-03 11:37:49.006854 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6839/17474 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708fe00 con 0x2342c6e0 -8454> 2014-11-03 11:37:49.006900 7f290101d700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.006900, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8453> 2014-11-03 11:37:49.006900 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6855/17474 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708fc00 con 0x2342c580 -8452> 2014-11-03 11:37:49.007018 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6820/21567 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708fa00 con 0x2416e2c0 -8451> 2014-11-03 11:37:49.007062 7f2914d0c700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.007062, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8450> 2014-11-03 11:37:49.007077 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6857/21567 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708f800 con 0x2416e160 -8449> 2014-11-03 11:37:49.007198 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6860/25883 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708f600 con 0x2416e580 -8448> 2014-11-03 11:37:49.007245 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6861/25883 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708f400 con 0x2416e420 -8447> 2014-11-03 11:37:49.007279 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6863/29518 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708f200 con 0x2445a580 -8446> 2014-11-03 11:37:49.007273 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 151 ==== osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3908943028 0 3511520646) 0x22f9f400 con 0x22fefdc0 -8445> 2014-11-03 11:37:49.007330 7f28f36c2700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007029, event: header_read, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8444> 2014-11-03 11:37:49.007327 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6864/29518 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x708f000 con 0x2445a420 -8443> 2014-11-03 11:37:49.007384 7f28f36c2700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007030, event: throttled, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8442> 2014-11-03 11:37:49.007412 7f28f36c2700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007261, event: all_read, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8441> 2014-11-03 11:37:49.007424 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6810/24552 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x5d49e00 con 0x2387f840 -8440> 2014-11-03 11:37:49.007442 7f28f36c2700 5 -- op tracker -- seq: 2836, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8439> 2014-11-03 11:37:49.007482 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6811/24552 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x5d49c00 con 0x2387f6e0 -8438> 2014-11-03 11:37:49.007507 7f291450b700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.007507, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8437> 2014-11-03 11:37:49.007535 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6821/5555 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x5d49a00 con 0x23d63420 -8436> 2014-11-03 11:37:49.007539 7f28fe818700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007539, event: reached_pg, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8435> 2014-11-03 11:37:49.007618 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6829/5555 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5e00 con 0x23d632c0 -8434> 2014-11-03 11:37:49.007630 7f28fe818700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007630, event: started, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8433> 2014-11-03 11:37:49.007645 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6817/17408 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5c00 con 0x233dadc0 -8432> 2014-11-03 11:37:49.007733 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6818/17408 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5a00 con 0x233d26e0 -8431> 2014-11-03 11:37:49.007763 7f28fe818700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007763, event: started, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8430> 2014-11-03 11:37:49.007768 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6803/6451 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5800 con 0x2416e840 -8429> 2014-11-03 11:37:49.007814 7f28fe818700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007814, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8428> 2014-11-03 11:37:49.007832 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6828/6451 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5600 con 0x2416e6e0 -8427> 2014-11-03 11:37:49.007876 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6839/23392 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5400 con 0x245ab6e0 -8426> 2014-11-03 11:37:49.007878 7f2911d06700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.007878, event: commit_sent, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8425> 2014-11-03 11:37:49.007907 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6840/23392 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5200 con 0x245ab580 -8424> 2014-11-03 11:37:49.007931 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x269af000 con 0x22fefdc0 -8423> 2014-11-03 11:37:49.007947 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6850/12205 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x248b5000 con 0x2445a000 -8422> 2014-11-03 11:37:49.007892 7f2914d0c700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.007892, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8421> 2014-11-03 11:37:49.007971 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6851/12205 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xedfde00 con 0x2445adc0 -8420> 2014-11-03 11:37:49.007973 7f2912507700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.007973, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8419> 2014-11-03 11:37:49.007996 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6855/17563 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xedfdc00 con 0x239f4840 -8418> 2014-11-03 11:37:49.008017 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6856/17563 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0xedfda00 con 0x239f46e0 -8417> 2014-11-03 11:37:49.008023 7f2912507700 5 -- op tracker -- seq: 2835, time: 2014-11-03 11:37:49.008023, event: done, op: osd_sub_op(client.8703713.0:32254347 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953132 snapset=0=[]:[] snapc=0=[]) -8416> 2014-11-03 11:37:49.008045 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6861/22948 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25bd9e00 con 0x23ea0840 -8415> 2014-11-03 11:37:49.008064 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6862/22948 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25bd9c00 con 0x23ea06e0 -8414> 2014-11-03 11:37:49.008099 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6865/28288 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x25bd9a00 con 0x23ea0dc0 -8413> 2014-11-03 11:37:49.008140 7f291450b700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.008140, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8412> 2014-11-03 11:37:49.008139 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6866/28288 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10a86e00 con 0x23ea0c60 -8411> 2014-11-03 11:37:49.008172 7f2911d06700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.008172, event: commit_sent, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8410> 2014-11-03 11:37:49.008194 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6871/31543 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10a86c00 con 0x2416edc0 -8409> 2014-11-03 11:37:49.008218 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6872/31543 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:49.003180) v2 -- ?+0 0x10a86a00 con 0x2416ec60 -8408> 2014-11-03 11:37:49.008217 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23c1d8c0 con 0x22fefdc0 -8407> 2014-11-03 11:37:49.008298 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.1 192.168.5.1:6809/11054 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x226aae00 con 0x239f4c60 -8406> 2014-11-03 11:37:49.008364 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.70 192.168.5.6:6809/3622 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xa248800 con 0x239f4b00 -8405> 2014-11-03 11:37:49.008412 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.70 192.168.5.6:6810/3622 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xa248e00 con 0x239f49a0 -8404> 2014-11-03 11:37:49.008529 7f2912507700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.008529, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8403> 2014-11-03 11:37:49.008530 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.98 192.168.5.8:6821/14722 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x24dce400 con 0x239f4580 -8402> 2014-11-03 11:37:49.008593 7f2912507700 5 -- op tracker -- seq: 2836, time: 2014-11-03 11:37:49.008592, event: done, op: osd_sub_op(client.8703713.0:32254348 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953133 snapset=0=[]:[] snapc=0=[]) -8401> 2014-11-03 11:37:49.008677 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.98 192.168.5.8:6823/14722 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2576c800 con 0x239f4420 -8400> 2014-11-03 11:37:49.008746 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.61 192.168.5.5:6832/12481 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x1ada2800 con 0x239f42c0 -8399> 2014-11-03 11:37:49.008795 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.61 192.168.5.5:6833/12481 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x147be400 con 0x239f4160 -8398> 2014-11-03 11:37:49.008857 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.77 192.168.5.6:6842/32590 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x27969e00 con 0x239f4000 -8397> 2014-11-03 11:37:49.008941 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.77 192.168.5.6:6843/32590 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2d164e00 con 0x23d63dc0 -8396> 2014-11-03 11:37:49.009004 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.10 192.168.5.1:6817/15344 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x6315800 con 0x23d63c60 -8395> 2014-11-03 11:37:49.009072 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.10 192.168.5.1:6821/15344 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x6c82e00 con 0x23d63b00 -8394> 2014-11-03 11:37:49.009118 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.76 192.168.5.6:6837/22493 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xe5b5e00 con 0x23d639a0 -8393> 2014-11-03 11:37:49.009161 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.76 192.168.5.6:6838/22493 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xecb2800 con 0x23d63840 -8392> 2014-11-03 11:37:49.009206 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.8 192.168.5.1:6893/8894 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x22d40e00 con 0x23d636e0 -8391> 2014-11-03 11:37:49.009235 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.8 192.168.5.1:6894/8894 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xaf3b200 con 0x23d63580 -8390> 2014-11-03 11:37:49.009285 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.113 192.168.5.9:6821/5555 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x10a86800 con 0x23d63420 -8389> 2014-11-03 11:37:49.009330 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.113 192.168.5.9:6829/5555 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2ba4d200 con 0x23d632c0 -8388> 2014-11-03 11:37:49.009366 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.81 192.168.5.6:6863/27170 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2d164800 con 0x23d63160 -8387> 2014-11-03 11:37:49.009408 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.81 192.168.5.6:6864/27170 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x242e2400 con 0x23d63000 -8386> 2014-11-03 11:37:49.009441 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.124 192.168.5.9:6865/28288 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x27783800 con 0x23ea0dc0 -8385> 2014-11-03 11:37:49.009477 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.124 192.168.5.9:6866/28288 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23821800 con 0x23ea0c60 -8384> 2014-11-03 11:37:49.009510 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.102 192.168.5.8:6812/21153 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2d344800 con 0x23ea0b00 -8383> 2014-11-03 11:37:49.009547 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.102 192.168.5.8:6814/21153 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xc2d1800 con 0x23ea09a0 -8382> 2014-11-03 11:37:49.009580 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.123 192.168.5.9:6861/22948 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2d6ec800 con 0x23ea0840 -8381> 2014-11-03 11:37:49.009628 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.123 192.168.5.9:6862/22948 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x896d800 con 0x23ea06e0 -8380> 2014-11-03 11:37:49.009666 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.42 192.168.5.4:6808/9946 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x54a3800 con 0x23ea0580 -8379> 2014-11-03 11:37:49.009703 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.42 192.168.5.4:6809/9946 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23ff5800 con 0x23ea0420 -8378> 2014-11-03 11:37:49.009738 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.13 192.168.5.1:6840/22281 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23561e00 con 0x23ea02c0 -8377> 2014-11-03 11:37:49.009776 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.13 192.168.5.1:6852/22281 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x73d3800 con 0x23ea0160 -8376> 2014-11-03 11:37:49.009819 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.48 192.168.5.4:6837/17727 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23f8cc00 con 0x23ea0000 -8375> 2014-11-03 11:37:49.009855 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.48 192.168.5.4:6839/17727 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23937800 con 0x2387ddc0 -8374> 2014-11-03 11:37:49.009887 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.30 192.168.5.3:6818/11504 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2de29400 con 0x2387dc60 -8373> 2014-11-03 11:37:49.009924 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.30 192.168.5.3:6820/11504 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x73d3e00 con 0x2387db00 -8372> 2014-11-03 11:37:49.009958 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.83 192.168.5.6:6872/4005 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xecb2e00 con 0x2387d9a0 -8371> 2014-11-03 11:37:49.009993 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.83 192.168.5.6:6873/4005 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x70a2800 con 0x2387d840 -8370> 2014-11-03 11:37:49.010045 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.105 192.168.5.8:6841/6777 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x229e7c00 con 0x2387d6e0 -8369> 2014-11-03 11:37:49.010084 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.105 192.168.5.8:6842/6777 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x229e7e00 con 0x2387d580 -8368> 2014-11-03 11:37:49.010162 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.99 192.168.5.8:6831/26930 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x80ff800 con 0x2387d420 -8367> 2014-11-03 11:37:49.010201 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.99 192.168.5.8:6832/26930 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x328e800 con 0x2387d2c0 -8366> 2014-11-03 11:37:49.010236 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.35 192.168.5.3:6847/6455 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2295ae00 con 0x2387d160 -8365> 2014-11-03 11:37:49.010278 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.35 192.168.5.3:6848/6455 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x26fb1e00 con 0x2387d000 -8364> 2014-11-03 11:37:49.010314 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.46 192.168.5.4:6827/7678 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x25e25800 con 0x245abc60 -8363> 2014-11-03 11:37:49.010349 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.46 192.168.5.4:6828/7678 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x1adc4600 con 0x245abb00 -8362> 2014-11-03 11:37:49.010382 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.50 192.168.5.4:6848/26902 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23d2b800 con 0x245ab9a0 -8361> 2014-11-03 11:37:49.010420 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.50 192.168.5.4:6849/26902 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x17cac800 con 0x245ab840 -8360> 2014-11-03 11:37:49.010452 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.118 192.168.5.9:6839/23392 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x6023200 con 0x245ab6e0 -8359> 2014-11-03 11:37:49.010486 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.118 192.168.5.9:6840/23392 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x26a9f000 con 0x245ab580 -8358> 2014-11-03 11:37:49.010518 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.68 192.168.5.5:6867/8133 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x261a4800 con 0x245ab420 -8357> 2014-11-03 11:37:49.010559 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.68 192.168.5.5:6868/8133 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xc02e800 con 0x245ab2c0 -8356> 2014-11-03 11:37:49.010591 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.104 192.168.5.8:6843/2576 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x187f4400 con 0x245ab160 -8355> 2014-11-03 11:37:49.010624 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.104 192.168.5.8:6844/2576 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x229e7800 con 0x245ab000 -8354> 2014-11-03 11:37:49.010657 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.125 192.168.5.9:6871/31543 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2438f000 con 0x2416edc0 -8353> 2014-11-03 11:37:49.010688 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.125 192.168.5.9:6872/31543 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x6023c00 con 0x2416ec60 -8352> 2014-11-03 11:37:49.010719 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.12 192.168.5.1:6829/19844 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xddd0e00 con 0x2416eb00 -8351> 2014-11-03 11:37:49.010753 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.12 192.168.5.1:6847/19844 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x3a32800 con 0x2416e9a0 -8350> 2014-11-03 11:37:49.010785 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.116 192.168.5.9:6803/6451 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xc157200 con 0x2416e840 -8349> 2014-11-03 11:37:49.010823 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.116 192.168.5.9:6828/6451 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x24d78e00 con 0x2416e6e0 -8348> 2014-11-03 11:37:49.010861 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.109 192.168.5.8:6860/25883 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23a41800 con 0x2416e580 -8347> 2014-11-03 11:37:49.010895 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.109 192.168.5.8:6861/25883 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xf637800 con 0x2416e420 -8346> 2014-11-03 11:37:49.010926 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.108 192.168.5.8:6820/21567 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x24092800 con 0x2416e2c0 -8345> 2014-11-03 11:37:49.010960 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.108 192.168.5.8:6857/21567 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2386ce00 con 0x2416e160 -8344> 2014-11-03 11:37:49.011000 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.44 192.168.5.4:6817/27523 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x256b9600 con 0x2416e000 -8343> 2014-11-03 11:37:49.011036 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.44 192.168.5.4:6818/27523 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x42a3800 con 0x24448420 -8342> 2014-11-03 11:37:49.011069 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.49 192.168.5.4:6843/21769 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x1474a800 con 0x244482c0 -8341> 2014-11-03 11:37:49.011105 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.49 192.168.5.4:6844/21769 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xc02ec00 con 0x23eb86e0 -8340> 2014-11-03 11:37:49.011138 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.69 192.168.5.5:6872/11297 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xe5b5800 con 0x23eb8580 -8339> 2014-11-03 11:37:49.011170 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.69 192.168.5.5:6873/11297 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x22a35e00 con 0x23eb8420 -8338> 2014-11-03 11:37:49.011219 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.37 192.168.5.3:6858/13652 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x26fb1800 con 0x2387fdc0 -8337> 2014-11-03 11:37:49.011255 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.37 192.168.5.3:6859/13652 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23250e00 con 0x2387fc60 -8336> 2014-11-03 11:37:49.011287 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.33 192.168.5.3:6840/29378 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23250800 con 0x2387fb00 -8335> 2014-11-03 11:37:49.011320 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.33 192.168.5.3:6841/29378 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2de29e00 con 0x2387f9a0 -8334> 2014-11-03 11:37:49.011352 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.112 192.168.5.9:6810/24552 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x8664200 con 0x2387f840 -8333> 2014-11-03 11:37:49.011389 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.112 192.168.5.9:6811/24552 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x41c2800 con 0x2387f6e0 -8332> 2014-11-03 11:37:49.011423 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.7 192.168.5.1:6887/3511 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xaf3b400 con 0x233da000 -8331> 2014-11-03 11:37:49.011456 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.7 192.168.5.1:6888/3511 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xa1a7a00 con 0x233da2c0 -8330> 2014-11-03 11:37:49.011488 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.43 192.168.5.4:6813/19461 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23482800 con 0x233da160 -8329> 2014-11-03 11:37:49.011521 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.43 192.168.5.4:6814/19461 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23f8c800 con 0x233da6e0 -8328> 2014-11-03 11:37:49.011553 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.6 192.168.5.1:6882/1181 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x22d40800 con 0x233da580 -8327> 2014-11-03 11:37:49.011585 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.6 192.168.5.1:6883/1181 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x6315e00 con 0x233da420 -8326> 2014-11-03 11:37:49.011616 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.32 192.168.5.3:6830/25327 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x1ae44e00 con 0x233dac60 -8325> 2014-11-03 11:37:49.011652 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.32 192.168.5.3:6831/25327 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x286bbe00 con 0x233dab00 -8324> 2014-11-03 11:37:49.011685 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.103 192.168.5.8:6827/29722 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x328ee00 con 0x233da9a0 -8323> 2014-11-03 11:37:49.011717 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.103 192.168.5.8:6830/29722 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2d344e00 con 0x233da840 -8322> 2014-11-03 11:37:49.011749 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.114 192.168.5.9:6817/17408 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x1a22c800 con 0x233dadc0 -8321> 2014-11-03 11:37:49.011781 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.114 192.168.5.9:6818/17408 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x1a22ce00 con 0x233d26e0 -8320> 2014-11-03 11:37:49.011812 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.55 192.168.5.4:6872/12368 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x9591800 con 0x233d2580 -8319> 2014-11-03 11:37:49.011845 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.55 192.168.5.4:6873/12368 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x25835800 con 0x233d2420 -8318> 2014-11-03 11:37:49.011877 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.66 192.168.5.5:6853/919 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x261a4e00 con 0x233d22c0 -8317> 2014-11-03 11:37:49.011915 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.66 192.168.5.5:6859/919 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x22a35800 con 0x233d2160 -8316> 2014-11-03 11:37:49.011955 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.97 192.168.5.7:6862/21285 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xf637e00 con 0x233d2000 -8315> 2014-11-03 11:37:49.011989 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.97 192.168.5.7:6863/21285 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2d329c00 con 0x233d2dc0 -8314> 2014-11-03 11:37:49.012020 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.31 192.168.5.3:6826/20618 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x405b800 con 0x233d2c60 -8313> 2014-11-03 11:37:49.012060 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.31 192.168.5.3:6827/20618 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2295a800 con 0x233d2b00 -8312> 2014-11-03 11:37:49.012092 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.90 192.168.5.7:6832/27918 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xc2d1e00 con 0x233d29a0 -8311> 2014-11-03 11:37:49.012126 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.90 192.168.5.7:6833/27918 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2576ce00 con 0x233d2840 -8310> 2014-11-03 11:37:49.012157 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.45 192.168.5.4:6822/1781 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x8714200 con 0x2342c9a0 -8309> 2014-11-03 11:37:49.012195 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.45 192.168.5.4:6823/1781 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23482e00 con 0x2342c840 -8308> 2014-11-03 11:37:49.012227 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.107 192.168.5.8:6839/17474 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x24092e00 con 0x2342c6e0 -8307> 2014-11-03 11:37:49.012259 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.107 192.168.5.8:6855/17474 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x5d49800 con 0x2342c580 -8306> 2014-11-03 11:37:49.012315 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.20 192.168.5.2:6834/4963 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x78cbe00 con 0x2342c420 -8305> 2014-11-03 11:37:49.012341 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.20 192.168.5.2:6837/4963 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x3a32e00 con 0x2342c2c0 -8304> 2014-11-03 11:37:49.012364 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.23 192.168.5.2:6852/14826 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x78cb800 con 0x2342c160 -8303> 2014-11-03 11:37:49.012387 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.23 192.168.5.2:6853/14826 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x6526800 con 0x2342c000 -8302> 2014-11-03 11:37:49.012408 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.4 192.168.5.1:6868/31636 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x218c4a00 con 0x2439c000 -8301> 2014-11-03 11:37:49.012441 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.4 192.168.5.1:6869/31636 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x218c4e00 con 0x2445ac60 -8300> 2014-11-03 11:37:49.012476 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.28 192.168.5.3:6813/13976 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x1ae44800 con 0x2445ab00 -8299> 2014-11-03 11:37:49.012516 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.28 192.168.5.3:6817/13976 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x73d3c00 con 0x2445a9a0 -8298> 2014-11-03 11:37:49.012551 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.75 192.168.5.6:6832/12612 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x42a3e00 con 0x2445a840 -8297> 2014-11-03 11:37:49.012597 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.75 192.168.5.6:6833/12612 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x619f800 con 0x2445a6e0 -8296> 2014-11-03 11:37:49.012632 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.110 192.168.5.8:6863/29518 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xedfd800 con 0x2445a580 -8295> 2014-11-03 11:37:49.012667 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.110 192.168.5.8:6864/29518 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x25bd9800 con 0x2445a420 -8294> 2014-11-03 11:37:49.012702 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.3 192.168.5.1:6858/28138 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xddd0800 con 0x2445a2c0 -8293> 2014-11-03 11:37:49.012739 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.3 192.168.5.1:6860/28138 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x6c82800 con 0x2445a160 -8292> 2014-11-03 11:37:49.012769 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.121 192.168.5.9:6850/12205 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x24d78800 con 0x2445a000 -8291> 2014-11-03 11:37:49.012802 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.121 192.168.5.9:6851/12205 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x27783e00 con 0x2445adc0 -8290> 2014-11-03 11:37:49.012854 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.54 192.168.5.4:6869/8391 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x22a08200 con 0x242056e0 -8289> 2014-11-03 11:37:49.012893 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.54 192.168.5.4:6870/8391 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0xa053800 con 0x9921000 -8288> 2014-11-03 11:37:49.012924 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.82 192.168.5.6:6867/31491 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x27969800 con 0x235f9c60 -8287> 2014-11-03 11:37:49.012957 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.82 192.168.5.6:6869/31491 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x2ba4d000 con 0x235f9b00 -8286> 2014-11-03 11:37:49.012990 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.40 192.168.5.3:6869/25675 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23937e00 con 0x235f99a0 -8285> 2014-11-03 11:37:49.013026 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.40 192.168.5.3:6870/25675 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x5f22e00 con 0x235f9840 -8284> 2014-11-03 11:37:49.013061 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.93 192.168.5.7:6842/7095 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x24800a00 con 0x235f96e0 -8283> 2014-11-03 11:37:49.013095 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.93 192.168.5.7:6843/7095 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x24dcea00 con 0x235f92c0 -8282> 2014-11-03 11:37:49.013123 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.122 192.168.5.9:6855/17563 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x226aae00 con 0x239f4840 -8281> 2014-11-03 11:37:49.013192 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.122 192.168.5.9:6856/17563 11 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.003180) v2 ==== 47+0+0 (2184667619 0 0) 0x23821200 con 0x239f46e0 -8280> 2014-11-03 11:37:49.049492 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 44 ==== osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2395785264 0 3511520646) 0x23c3e000 con 0x22fbc000 -8279> 2014-11-03 11:37:49.049541 7f28f71fd700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049374, event: header_read, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8278> 2014-11-03 11:37:49.049557 7f28f71fd700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049377, event: throttled, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8277> 2014-11-03 11:37:49.049566 7f28f71fd700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049477, event: all_read, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8276> 2014-11-03 11:37:49.049576 7f28f71fd700 5 -- op tracker -- seq: 2837, time: 0.000000, event: dispatched, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8275> 2014-11-03 11:37:49.049641 7f28fd816700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049641, event: reached_pg, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8274> 2014-11-03 11:37:49.049678 7f28fd816700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049678, event: started, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8273> 2014-11-03 11:37:49.049792 7f28fd816700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049791, event: started, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8272> 2014-11-03 11:37:49.049860 7f28fd816700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049860, event: commit_queued_for_journal_write, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8271> 2014-11-03 11:37:49.049923 7f2914d0c700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.049922, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8270> 2014-11-03 11:37:49.050135 7f291450b700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.050135, event: journaled_completion_queued, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8269> 2014-11-03 11:37:49.050192 7f2911d06700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.050192, event: commit_sent, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8268> 2014-11-03 11:37:49.050221 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1d340 con 0x22fbc000 -8267> 2014-11-03 11:37:49.050622 7f2912507700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.050622, event: sub_op_applied, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8266> 2014-11-03 11:37:49.050658 7f2912507700 5 -- op tracker -- seq: 2837, time: 2014-11-03 11:37:49.050658, event: done, op: osd_sub_op(client.9538981.0:653233 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100461 snapset=0=[]:[] snapc=0=[]) -8265> 2014-11-03 11:37:49.058452 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 45 ==== osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1080351507 0 3511520646) 0x24661c00 con 0x22fbc000 -8264> 2014-11-03 11:37:49.058493 7f28f71fd700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058368, event: header_read, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8263> 2014-11-03 11:37:49.058511 7f28f71fd700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058369, event: throttled, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8262> 2014-11-03 11:37:49.058520 7f28f71fd700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058438, event: all_read, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8261> 2014-11-03 11:37:49.058528 7f28f71fd700 5 -- op tracker -- seq: 2838, time: 0.000000, event: dispatched, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8260> 2014-11-03 11:37:49.058578 7f290001b700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058578, event: reached_pg, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8259> 2014-11-03 11:37:49.058611 7f290001b700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058611, event: started, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8258> 2014-11-03 11:37:49.058713 7f290001b700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058712, event: started, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8257> 2014-11-03 11:37:49.058746 7f290001b700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058745, event: commit_queued_for_journal_write, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8256> 2014-11-03 11:37:49.058841 7f2914d0c700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.058841, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8255> 2014-11-03 11:37:49.059085 7f291450b700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.059085, event: journaled_completion_queued, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8254> 2014-11-03 11:37:49.059162 7f2911d06700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.059162, event: commit_sent, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8253> 2014-11-03 11:37:49.059197 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1d600 con 0x22fbc000 -8252> 2014-11-03 11:37:49.059458 7f2912507700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.059458, event: sub_op_applied, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8251> 2014-11-03 11:37:49.059494 7f2912507700 5 -- op tracker -- seq: 2838, time: 2014-11-03 11:37:49.059494, event: done, op: osd_sub_op(client.9538981.0:653234 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100462 snapset=0=[]:[] snapc=0=[]) -8250> 2014-11-03 11:37:49.073534 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.124 192.168.5.9:0/28288 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.072387) v2 ==== 47+0+0 (2733323297 0 0) 0x1a327800 con 0x2389f160 -8249> 2014-11-03 11:37:49.073559 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.124 192.168.5.9:0/28288 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.072387) v2 ==== 47+0+0 (2733323297 0 0) 0x2386c000 con 0x31a9160 -8248> 2014-11-03 11:37:49.073590 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/28288 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.072387) v2 -- ?+0 0x2b1ff600 con 0x31a9160 -8247> 2014-11-03 11:37:49.073648 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/28288 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.072387) v2 -- ?+0 0x8664a00 con 0x2389f160 -8246> 2014-11-03 11:37:49.080007 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 83 ==== osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1526443568 0 3511520646) 0x2d47b000 con 0x22ff3b00 -8245> 2014-11-03 11:37:49.080086 7f28f0d99700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.079884, event: header_read, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8244> 2014-11-03 11:37:49.080117 7f28f0d99700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.079886, event: throttled, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8243> 2014-11-03 11:37:49.080129 7f28f0d99700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.079992, event: all_read, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8242> 2014-11-03 11:37:49.080150 7f28f0d99700 5 -- op tracker -- seq: 2839, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8241> 2014-11-03 11:37:49.080229 7f28fd816700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.080228, event: reached_pg, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8240> 2014-11-03 11:37:49.080261 7f28fd816700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.080261, event: started, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8239> 2014-11-03 11:37:49.080432 7f28fd816700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.080432, event: started, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8238> 2014-11-03 11:37:49.080479 7f28fd816700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.080479, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8237> 2014-11-03 11:37:49.080559 7f2914d0c700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.080559, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8236> 2014-11-03 11:37:49.080798 7f291450b700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.080797, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8235> 2014-11-03 11:37:49.080868 7f2911d06700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.080868, event: commit_sent, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8234> 2014-11-03 11:37:49.080903 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x278032c0 con 0x22ff3b00 -8233> 2014-11-03 11:37:49.081139 7f2912507700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.081138, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8232> 2014-11-03 11:37:49.081181 7f2912507700 5 -- op tracker -- seq: 2839, time: 2014-11-03 11:37:49.081181, event: done, op: osd_sub_op(client.12146205.0:439300 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222065 snapset=0=[]:[] snapc=0=[]) -8231> 2014-11-03 11:37:49.116833 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 46 ==== osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1087596341 0 3511520646) 0x28ea3c00 con 0x22fbc000 -8230> 2014-11-03 11:37:49.116882 7f28f71fd700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.116737, event: header_read, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8229> 2014-11-03 11:37:49.116900 7f28f71fd700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.116739, event: throttled, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8228> 2014-11-03 11:37:49.116913 7f28f71fd700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.116819, event: all_read, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8227> 2014-11-03 11:37:49.116922 7f28f71fd700 5 -- op tracker -- seq: 2840, time: 0.000000, event: dispatched, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8226> 2014-11-03 11:37:49.117017 7f290001b700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117017, event: reached_pg, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8225> 2014-11-03 11:37:49.117052 7f290001b700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117052, event: started, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8224> 2014-11-03 11:37:49.117144 7f290001b700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117144, event: started, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8223> 2014-11-03 11:37:49.117172 7f290001b700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117172, event: commit_queued_for_journal_write, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8222> 2014-11-03 11:37:49.117229 7f2914d0c700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117229, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8221> 2014-11-03 11:37:49.117467 7f291450b700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117467, event: journaled_completion_queued, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8220> 2014-11-03 11:37:49.117542 7f2911d06700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117542, event: commit_sent, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8219> 2014-11-03 11:37:49.117580 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2905a600 con 0x22fbc000 -8218> 2014-11-03 11:37:49.117801 7f2912507700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117800, event: sub_op_applied, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8217> 2014-11-03 11:37:49.117844 7f2912507700 5 -- op tracker -- seq: 2840, time: 2014-11-03 11:37:49.117844, event: done, op: osd_sub_op(client.9538981.0:653238 6.4f0 b6ca7cf0/rbd_data.91871c2ae8944a.00000000000001e7/head//6 [] v 105950'13100463 snapset=0=[]:[] snapc=0=[]) -8216> 2014-11-03 11:37:49.121317 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 45 ==== osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3208613172 0 3511520646) 0x295a4400 con 0x22fbc160 -8215> 2014-11-03 11:37:49.121352 7f28f6df9700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121228, event: header_read, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8214> 2014-11-03 11:37:49.121378 7f28f6df9700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121231, event: throttled, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8213> 2014-11-03 11:37:49.121398 7f28f6df9700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121307, event: all_read, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8212> 2014-11-03 11:37:49.121417 7f28f6df9700 5 -- op tracker -- seq: 2841, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8211> 2014-11-03 11:37:49.121549 7f290101d700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121548, event: reached_pg, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8210> 2014-11-03 11:37:49.121620 7f290101d700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121620, event: started, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8209> 2014-11-03 11:37:49.121815 7f290101d700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121815, event: started, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8208> 2014-11-03 11:37:49.121852 7f290101d700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121852, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8207> 2014-11-03 11:37:49.121911 7f2914d0c700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.121911, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8206> 2014-11-03 11:37:49.122433 7f291450b700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.122433, event: journaled_completion_queued, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8205> 2014-11-03 11:37:49.122526 7f2911d06700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.122526, event: commit_sent, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8204> 2014-11-03 11:37:49.122553 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- osd_sub_op_reply(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcac000 con 0x22fbc160 -8203> 2014-11-03 11:37:49.123082 7f2912507700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.123082, event: sub_op_applied, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8202> 2014-11-03 11:37:49.123131 7f2912507700 5 -- op tracker -- seq: 2841, time: 2014-11-03 11:37:49.123130, event: done, op: osd_sub_op(client.7867498.0:12568317 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539476 snapset=0=[]:[] snapc=0=[]) -8201> 2014-11-03 11:37:49.133537 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.31 192.168.5.3:0/20618 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.132513) v2 ==== 47+0+0 (2025851133 0 0) 0x30c1400 con 0x24925840 -8200> 2014-11-03 11:37:49.133582 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.31 192.168.5.3:0/20618 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.132513) v2 ==== 47+0+0 (2025851133 0 0) 0x8714000 con 0x24229580 -8199> 2014-11-03 11:37:49.133617 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.132513) v2 -- ?+0 0x2386c000 con 0x24925840 -8198> 2014-11-03 11:37:49.133709 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.132513) v2 -- ?+0 0x1a327800 con 0x24229580 -8197> 2014-11-03 11:37:49.146054 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 55 ==== osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3577797407 0 3511520646) 0x3348000 con 0x230c2000 -8196> 2014-11-03 11:37:49.146202 7f28ebc48700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.145864, event: header_read, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8195> 2014-11-03 11:37:49.146291 7f28ebc48700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.145869, event: throttled, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8194> 2014-11-03 11:37:49.146324 7f28ebc48700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.146036, event: all_read, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8193> 2014-11-03 11:37:49.146357 7f28ebc48700 5 -- op tracker -- seq: 2842, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8192> 2014-11-03 11:37:49.146439 7f28fe818700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.146439, event: reached_pg, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8191> 2014-11-03 11:37:49.146486 7f28fe818700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.146486, event: started, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8190> 2014-11-03 11:37:49.146634 7f28fe818700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.146634, event: started, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8189> 2014-11-03 11:37:49.146687 7f28fe818700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.146687, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8188> 2014-11-03 11:37:49.146798 7f2914d0c700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.146797, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8187> 2014-11-03 11:37:49.146856 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 56 ==== osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2450706149 0 3511520646) 0x2d575800 con 0x230c2000 -8186> 2014-11-03 11:37:49.146923 7f28ebc48700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.146686, event: header_read, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8185> 2014-11-03 11:37:49.146965 7f28ebc48700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.146688, event: throttled, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8184> 2014-11-03 11:37:49.146997 7f28ebc48700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.146840, event: all_read, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8183> 2014-11-03 11:37:49.147019 7f28ebc48700 5 -- op tracker -- seq: 2843, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8182> 2014-11-03 11:37:49.147091 7f290101d700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.147091, event: reached_pg, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8181> 2014-11-03 11:37:49.147117 7f291450b700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.147117, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8180> 2014-11-03 11:37:49.147136 7f290101d700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.147136, event: started, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8179> 2014-11-03 11:37:49.147267 7f290101d700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.147267, event: started, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8178> 2014-11-03 11:37:49.147311 7f290101d700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.147311, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8177> 2014-11-03 11:37:49.147441 7f2911d06700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.147441, event: commit_sent, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8176> 2014-11-03 11:37:49.147452 7f2914d0c700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.147451, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8175> 2014-11-03 11:37:49.147473 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcad8c0 con 0x230c2000 -8174> 2014-11-03 11:37:49.147546 7f2912507700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.147545, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8173> 2014-11-03 11:37:49.147605 7f2912507700 5 -- op tracker -- seq: 2842, time: 2014-11-03 11:37:49.147605, event: done, op: osd_sub_op(client.7914488.0:1855172 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166697 snapset=0=[]:[] snapc=0=[]) -8172> 2014-11-03 11:37:49.147644 7f291450b700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.147643, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8171> 2014-11-03 11:37:49.147705 7f2911d06700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.147704, event: commit_sent, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8170> 2014-11-03 11:37:49.147759 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 57 ==== osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4195828206 0 3511520646) 0x2d576400 con 0x230c2000 -8169> 2014-11-03 11:37:49.147792 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcadb80 con 0x230c2000 -8168> 2014-11-03 11:37:49.147796 7f28ebc48700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.147581, event: header_read, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8167> 2014-11-03 11:37:49.147864 7f28ebc48700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.147583, event: throttled, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8166> 2014-11-03 11:37:49.147886 7f28ebc48700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.147745, event: all_read, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8165> 2014-11-03 11:37:49.147908 7f28ebc48700 5 -- op tracker -- seq: 2844, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8164> 2014-11-03 11:37:49.147978 7f28fe818700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.147978, event: reached_pg, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8163> 2014-11-03 11:37:49.148030 7f28fe818700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.148029, event: started, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8162> 2014-11-03 11:37:49.148194 7f28fe818700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.148193, event: started, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8161> 2014-11-03 11:37:49.148238 7f28fe818700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.148238, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8160> 2014-11-03 11:37:49.148306 7f2912507700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.148306, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8159> 2014-11-03 11:37:49.148317 7f2914d0c700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.148317, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8158> 2014-11-03 11:37:49.148375 7f2912507700 5 -- op tracker -- seq: 2843, time: 2014-11-03 11:37:49.148375, event: done, op: osd_sub_op(client.7914488.0:1855173 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166698 snapset=0=[]:[] snapc=0=[]) -8157> 2014-11-03 11:37:49.148678 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 58 ==== osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3956212182 0 3511520646) 0x2d47d400 con 0x230c2000 -8156> 2014-11-03 11:37:49.148720 7f28ebc48700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.148558, event: header_read, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8155> 2014-11-03 11:37:49.148746 7f28ebc48700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.148560, event: throttled, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8154> 2014-11-03 11:37:49.148765 7f28ebc48700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.148663, event: all_read, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8153> 2014-11-03 11:37:49.148784 7f28ebc48700 5 -- op tracker -- seq: 2845, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8152> 2014-11-03 11:37:49.148874 7f290101d700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.148873, event: reached_pg, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8151> 2014-11-03 11:37:49.148916 7f290101d700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.148916, event: started, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8150> 2014-11-03 11:37:49.149037 7f290101d700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.149037, event: started, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8149> 2014-11-03 11:37:49.149077 7f290101d700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.149077, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8148> 2014-11-03 11:37:49.149299 7f291450b700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.149299, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8147> 2014-11-03 11:37:49.149368 7f2911d06700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.149367, event: commit_sent, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8146> 2014-11-03 11:37:49.149376 7f2914d0c700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.149376, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8145> 2014-11-03 11:37:49.149438 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eb080 con 0x230c2000 -8144> 2014-11-03 11:37:49.149570 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 59 ==== osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3477443455 0 3511520646) 0x2799a800 con 0x230c2000 -8143> 2014-11-03 11:37:49.149606 7f28ebc48700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.149435, event: header_read, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8142> 2014-11-03 11:37:49.149628 7f28ebc48700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.149438, event: throttled, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8141> 2014-11-03 11:37:49.149643 7f28ebc48700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.149537, event: all_read, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8140> 2014-11-03 11:37:49.149658 7f28ebc48700 5 -- op tracker -- seq: 2846, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8139> 2014-11-03 11:37:49.149690 7f2912507700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.149690, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8138> 2014-11-03 11:37:49.149745 7f2912507700 5 -- op tracker -- seq: 2844, time: 2014-11-03 11:37:49.149744, event: done, op: osd_sub_op(client.7914488.0:1855174 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166699 snapset=0=[]:[] snapc=0=[]) -8137> 2014-11-03 11:37:49.149801 7f28fe818700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.149801, event: reached_pg, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8136> 2014-11-03 11:37:49.149841 7f28fe818700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.149841, event: started, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8135> 2014-11-03 11:37:49.149958 7f28fe818700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.149957, event: started, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8134> 2014-11-03 11:37:49.150000 7f28fe818700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.149999, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8133> 2014-11-03 11:37:49.150395 7f291450b700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.150394, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8132> 2014-11-03 11:37:49.150442 7f2911d06700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.150441, event: commit_sent, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8131> 2014-11-03 11:37:49.150488 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9a840 con 0x230c2000 -8130> 2014-11-03 11:37:49.150496 7f2914d0c700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.150495, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8129> 2014-11-03 11:37:49.150758 7f2912507700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.150758, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8128> 2014-11-03 11:37:49.150768 7f291450b700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.150768, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8127> 2014-11-03 11:37:49.150818 7f2912507700 5 -- op tracker -- seq: 2845, time: 2014-11-03 11:37:49.150818, event: done, op: osd_sub_op(client.7914488.0:1855175 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166700 snapset=0=[]:[] snapc=0=[]) -8126> 2014-11-03 11:37:49.150906 7f2911d06700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.150905, event: commit_sent, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8125> 2014-11-03 11:37:49.150947 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2ea7a000 con 0x230c2000 -8124> 2014-11-03 11:37:49.151261 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 60 ==== osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (724029705 0 3511520646) 0x2dcb3400 con 0x230c2000 -8123> 2014-11-03 11:37:49.151258 7f2912507700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.151257, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8122> 2014-11-03 11:37:49.151299 7f28ebc48700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151137, event: header_read, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8121> 2014-11-03 11:37:49.151314 7f28ebc48700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151151, event: throttled, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8120> 2014-11-03 11:37:49.151321 7f2912507700 5 -- op tracker -- seq: 2846, time: 2014-11-03 11:37:49.151321, event: done, op: osd_sub_op(client.7914488.0:1855176 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166701 snapset=0=[]:[] snapc=0=[]) -8119> 2014-11-03 11:37:49.151329 7f28ebc48700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151250, event: all_read, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8118> 2014-11-03 11:37:49.151340 7f28ebc48700 5 -- op tracker -- seq: 2847, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8117> 2014-11-03 11:37:49.151391 7f290101d700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151390, event: reached_pg, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8116> 2014-11-03 11:37:49.151423 7f290101d700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151423, event: started, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8115> 2014-11-03 11:37:49.151454 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.4 192.168.5.1:0/31636 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.150796) v2 ==== 47+0+0 (383422905 0 0) 0x23482800 con 0x2389f580 -8114> 2014-11-03 11:37:49.151507 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.4 192.168.5.1:0/31636 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.150796) v2 ==== 47+0+0 (383422905 0 0) 0x25835800 con 0x31a9580 -8113> 2014-11-03 11:37:49.151555 7f290101d700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151555, event: started, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8112> 2014-11-03 11:37:49.151524 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/31636 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.150796) v2 -- ?+0 0x8714000 con 0x2389f580 -8111> 2014-11-03 11:37:49.151604 7f290101d700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151604, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8110> 2014-11-03 11:37:49.151640 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/31636 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.150796) v2 -- ?+0 0x30c1400 con 0x31a9580 -8109> 2014-11-03 11:37:49.151691 7f2914d0c700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151690, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8108> 2014-11-03 11:37:49.151941 7f291450b700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151941, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8107> 2014-11-03 11:37:49.151999 7f2911d06700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.151998, event: commit_sent, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8106> 2014-11-03 11:37:49.152040 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7c840 con 0x230c2000 -8105> 2014-11-03 11:37:49.152248 7f2912507700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.152248, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8104> 2014-11-03 11:37:49.152310 7f2912507700 5 -- op tracker -- seq: 2847, time: 2014-11-03 11:37:49.152309, event: done, op: osd_sub_op(client.7914488.0:1855177 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166702 snapset=0=[]:[] snapc=0=[]) -8103> 2014-11-03 11:37:49.154263 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 61 ==== osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1437305679 0 3511520646) 0x2d3a8800 con 0x230c2000 -8102> 2014-11-03 11:37:49.154298 7f28ebc48700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154100, event: header_read, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8101> 2014-11-03 11:37:49.154312 7f28ebc48700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154102, event: throttled, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8100> 2014-11-03 11:37:49.154325 7f28ebc48700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154251, event: all_read, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8099> 2014-11-03 11:37:49.154333 7f28ebc48700 5 -- op tracker -- seq: 2848, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8098> 2014-11-03 11:37:49.154376 7f28fe818700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154376, event: reached_pg, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8097> 2014-11-03 11:37:49.154400 7f28fe818700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154400, event: started, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8096> 2014-11-03 11:37:49.154482 7f28fe818700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154482, event: started, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8095> 2014-11-03 11:37:49.154507 7f28fe818700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154507, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8094> 2014-11-03 11:37:49.154591 7f2914d0c700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154591, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8093> 2014-11-03 11:37:49.154793 7f291450b700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154793, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8092> 2014-11-03 11:37:49.154866 7f2911d06700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.154866, event: commit_sent, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8091> 2014-11-03 11:37:49.154903 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258be8c0 con 0x230c2000 -8090> 2014-11-03 11:37:49.155123 7f2912507700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.155123, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8089> 2014-11-03 11:37:49.155123 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 62 ==== osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1745732415 0 3511520646) 0x27000000 con 0x230c2000 -8088> 2014-11-03 11:37:49.155183 7f2912507700 5 -- op tracker -- seq: 2848, time: 2014-11-03 11:37:49.155183, event: done, op: osd_sub_op(client.7914488.0:1855178 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166703 snapset=0=[]:[] snapc=0=[]) -8087> 2014-11-03 11:37:49.155185 7f28ebc48700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.154934, event: header_read, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8086> 2014-11-03 11:37:49.155203 7f28ebc48700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.154936, event: throttled, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8085> 2014-11-03 11:37:49.155220 7f28ebc48700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.155004, event: all_read, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8084> 2014-11-03 11:37:49.155229 7f28ebc48700 5 -- op tracker -- seq: 2849, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8083> 2014-11-03 11:37:49.155273 7f290101d700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.155273, event: reached_pg, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8082> 2014-11-03 11:37:49.155304 7f290101d700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.155304, event: started, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8081> 2014-11-03 11:37:49.155388 7f290101d700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.155388, event: started, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8080> 2014-11-03 11:37:49.155413 7f290101d700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.155413, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8079> 2014-11-03 11:37:49.155488 7f2914d0c700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.155487, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8078> 2014-11-03 11:37:49.156206 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 63 ==== osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2986517304 0 3511520646) 0x3348000 con 0x230c2000 -8077> 2014-11-03 11:37:49.156233 7f28ebc48700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156084, event: header_read, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8076> 2014-11-03 11:37:49.156245 7f28ebc48700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156085, event: throttled, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8075> 2014-11-03 11:37:49.156259 7f28ebc48700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156197, event: all_read, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8074> 2014-11-03 11:37:49.156272 7f28ebc48700 5 -- op tracker -- seq: 2850, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8073> 2014-11-03 11:37:49.156316 7f28fe818700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156315, event: reached_pg, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8072> 2014-11-03 11:37:49.156339 7f28fe818700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156339, event: started, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8071> 2014-11-03 11:37:49.156412 7f28fe818700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156412, event: started, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8070> 2014-11-03 11:37:49.156436 7f28fe818700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156436, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8069> 2014-11-03 11:37:49.156513 7f291450b700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.156513, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8068> 2014-11-03 11:37:49.156572 7f2911d06700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.156572, event: commit_sent, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8067> 2014-11-03 11:37:49.156587 7f2914d0c700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156587, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8066> 2014-11-03 11:37:49.156609 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bd840 con 0x230c2000 -8065> 2014-11-03 11:37:49.156849 7f291450b700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.156849, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8064> 2014-11-03 11:37:49.156889 7f2912507700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.156888, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8063> 2014-11-03 11:37:49.156922 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 64 ==== osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1696824032 0 3511520646) 0x24662800 con 0x230c2000 -8062> 2014-11-03 11:37:49.156929 7f2912507700 5 -- op tracker -- seq: 2849, time: 2014-11-03 11:37:49.156929, event: done, op: osd_sub_op(client.7914488.0:1855179 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166704 snapset=0=[]:[] snapc=0=[]) -8061> 2014-11-03 11:37:49.156956 7f28ebc48700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.156835, event: header_read, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8060> 2014-11-03 11:37:49.156973 7f28ebc48700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.156836, event: throttled, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8059> 2014-11-03 11:37:49.157025 7f2911d06700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.157025, event: commit_sent, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8058> 2014-11-03 11:37:49.157047 7f28ebc48700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.156912, event: all_read, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8057> 2014-11-03 11:37:49.157070 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bd000 con 0x230c2000 -8056> 2014-11-03 11:37:49.157078 7f28ebc48700 5 -- op tracker -- seq: 2851, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8055> 2014-11-03 11:37:49.157142 7f2912507700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.157142, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8054> 2014-11-03 11:37:49.157182 7f2912507700 5 -- op tracker -- seq: 2850, time: 2014-11-03 11:37:49.157182, event: done, op: osd_sub_op(client.7914488.0:1855180 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166705 snapset=0=[]:[] snapc=0=[]) -8053> 2014-11-03 11:37:49.157246 7f290101d700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.157245, event: reached_pg, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8052> 2014-11-03 11:37:49.157278 7f290101d700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.157278, event: started, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8051> 2014-11-03 11:37:49.157425 7f290101d700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.157424, event: started, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8050> 2014-11-03 11:37:49.157458 7f290101d700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.157458, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8049> 2014-11-03 11:37:49.157522 7f2914d0c700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.157521, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8048> 2014-11-03 11:37:49.157658 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 106 ==== osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2868418763 0 3511520646) 0x2fdf4c00 con 0x22fb76e0 -8047> 2014-11-03 11:37:49.157687 7f28edd69700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.157563, event: header_read, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8046> 2014-11-03 11:37:49.157736 7f28edd69700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.157565, event: throttled, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8045> 2014-11-03 11:37:49.157748 7f28edd69700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.157646, event: all_read, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8044> 2014-11-03 11:37:49.157760 7f28edd69700 5 -- op tracker -- seq: 2852, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8043> 2014-11-03 11:37:49.157772 7f291450b700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.157772, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8042> 2014-11-03 11:37:49.157814 7f28fd816700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.157813, event: reached_pg, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8041> 2014-11-03 11:37:49.157818 7f2911d06700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.157818, event: commit_sent, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8040> 2014-11-03 11:37:49.157857 7f28fd816700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.157857, event: started, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8039> 2014-11-03 11:37:49.157868 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2494b000 con 0x230c2000 -8038> 2014-11-03 11:37:49.157979 7f28fd816700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.157979, event: started, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8037> 2014-11-03 11:37:49.158026 7f28fd816700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.158026, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8036> 2014-11-03 11:37:49.158097 7f2914d0c700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.158096, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8035> 2014-11-03 11:37:49.158153 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 65 ==== osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (396751153 0 3511520646) 0x24661c00 con 0x230c2000 -8034> 2014-11-03 11:37:49.158178 7f28ebc48700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158039, event: header_read, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8033> 2014-11-03 11:37:49.158189 7f2912507700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.158189, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8032> 2014-11-03 11:37:49.158212 7f28ebc48700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158040, event: throttled, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8031> 2014-11-03 11:37:49.158231 7f28ebc48700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158133, event: all_read, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8030> 2014-11-03 11:37:49.158234 7f2912507700 5 -- op tracker -- seq: 2851, time: 2014-11-03 11:37:49.158234, event: done, op: osd_sub_op(client.7914488.0:1855181 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166706 snapset=0=[]:[] snapc=0=[]) -8029> 2014-11-03 11:37:49.158243 7f28ebc48700 5 -- op tracker -- seq: 2853, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8028> 2014-11-03 11:37:49.158321 7f28fe818700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158321, event: reached_pg, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8027> 2014-11-03 11:37:49.158328 7f291450b700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.158328, event: journaled_completion_queued, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8026> 2014-11-03 11:37:49.158354 7f28fe818700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158354, event: started, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8025> 2014-11-03 11:37:49.158395 7f2911d06700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.158394, event: commit_sent, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8024> 2014-11-03 11:37:49.158431 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] ondisk, result = 0) v2 -- ?+0 0x25f9d580 con 0x22fb76e0 -8023> 2014-11-03 11:37:49.158463 7f28fe818700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158462, event: started, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8022> 2014-11-03 11:37:49.158507 7f28fe818700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158507, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8021> 2014-11-03 11:37:49.158605 7f2914d0c700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158605, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8020> 2014-11-03 11:37:49.158682 7f2912507700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.158681, event: sub_op_applied, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8019> 2014-11-03 11:37:49.158743 7f2912507700 5 -- op tracker -- seq: 2852, time: 2014-11-03 11:37:49.158743, event: done, op: osd_sub_op(client.7867546.0:2630233 6.24d 885e524d/rbd_data.4c8d092ae8944a.0000000000001a81/head//6 [] v 105950'12187820 snapset=0=[]:[] snapc=0=[]) -8018> 2014-11-03 11:37:49.158859 7f291450b700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158859, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8017> 2014-11-03 11:37:49.158914 7f2911d06700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.158914, event: commit_sent, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8016> 2014-11-03 11:37:49.158935 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x246ba8c0 con 0x230c2000 -8015> 2014-11-03 11:37:49.159274 7f2912507700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.159274, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8014> 2014-11-03 11:37:49.159307 7f2912507700 5 -- op tracker -- seq: 2853, time: 2014-11-03 11:37:49.159306, event: done, op: osd_sub_op(client.7914488.0:1855182 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166707 snapset=0=[]:[] snapc=0=[]) -8013> 2014-11-03 11:37:49.160349 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 66 ==== osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3790469155 0 3511520646) 0x26151000 con 0x230c2000 -8012> 2014-11-03 11:37:49.160376 7f28ebc48700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160278, event: header_read, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8011> 2014-11-03 11:37:49.160393 7f28ebc48700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160279, event: throttled, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8010> 2014-11-03 11:37:49.160406 7f28ebc48700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160341, event: all_read, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8009> 2014-11-03 11:37:49.160417 7f28ebc48700 5 -- op tracker -- seq: 2854, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8008> 2014-11-03 11:37:49.160473 7f290101d700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160473, event: reached_pg, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8007> 2014-11-03 11:37:49.160518 7f290101d700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160518, event: started, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8006> 2014-11-03 11:37:49.160635 7f290101d700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160635, event: started, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8005> 2014-11-03 11:37:49.160702 7f290101d700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160701, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8004> 2014-11-03 11:37:49.160838 7f2914d0c700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.160838, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8003> 2014-11-03 11:37:49.161110 7f291450b700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.161110, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8002> 2014-11-03 11:37:49.161147 7f2911d06700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.161147, event: commit_sent, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -8001> 2014-11-03 11:37:49.161169 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28228580 con 0x230c2000 -8000> 2014-11-03 11:37:49.161350 7f2912507700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.161349, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -7999> 2014-11-03 11:37:49.161386 7f2912507700 5 -- op tracker -- seq: 2854, time: 2014-11-03 11:37:49.161386, event: done, op: osd_sub_op(client.7914488.0:1855183 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166708 snapset=0=[]:[] snapc=0=[]) -7998> 2014-11-03 11:37:49.162708 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 67 ==== osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3348671578 0 3511520646) 0x27000c00 con 0x230c2000 -7997> 2014-11-03 11:37:49.162745 7f28ebc48700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.162622, event: header_read, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7996> 2014-11-03 11:37:49.162757 7f28ebc48700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.162628, event: throttled, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7995> 2014-11-03 11:37:49.162765 7f28ebc48700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.162700, event: all_read, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7994> 2014-11-03 11:37:49.162773 7f28ebc48700 5 -- op tracker -- seq: 2855, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7993> 2014-11-03 11:37:49.162836 7f28fe818700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.162836, event: reached_pg, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7992> 2014-11-03 11:37:49.162867 7f28fe818700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.162867, event: started, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7991> 2014-11-03 11:37:49.162955 7f28fe818700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.162955, event: started, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7990> 2014-11-03 11:37:49.162980 7f28fe818700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.162980, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7989> 2014-11-03 11:37:49.163034 7f2914d0c700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.163034, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7988> 2014-11-03 11:37:49.163308 7f291450b700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.163308, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7987> 2014-11-03 11:37:49.163382 7f2911d06700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.163382, event: commit_sent, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7986> 2014-11-03 11:37:49.163417 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28229340 con 0x230c2000 -7985> 2014-11-03 11:37:49.163796 7f2912507700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.163796, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7984> 2014-11-03 11:37:49.163829 7f2912507700 5 -- op tracker -- seq: 2855, time: 2014-11-03 11:37:49.163829, event: done, op: osd_sub_op(client.7914488.0:1855184 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166709 snapset=0=[]:[] snapc=0=[]) -7983> 2014-11-03 11:37:49.164375 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 68 ==== osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2728355934 0 3511520646) 0x2dcb3400 con 0x230c2000 -7982> 2014-11-03 11:37:49.164467 7f28ebc48700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.164262, event: header_read, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7981> 2014-11-03 11:37:49.164490 7f28ebc48700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.164264, event: throttled, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7980> 2014-11-03 11:37:49.164509 7f28ebc48700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.164357, event: all_read, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7979> 2014-11-03 11:37:49.164543 7f28ebc48700 5 -- op tracker -- seq: 2856, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7978> 2014-11-03 11:37:49.164629 7f290101d700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.164629, event: reached_pg, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7977> 2014-11-03 11:37:49.164690 7f290101d700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.164690, event: started, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7976> 2014-11-03 11:37:49.164832 7f290101d700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.164831, event: started, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7975> 2014-11-03 11:37:49.164886 7f290101d700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.164886, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7974> 2014-11-03 11:37:49.165000 7f2914d0c700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.165000, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7973> 2014-11-03 11:37:49.165319 7f291450b700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.165318, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7972> 2014-11-03 11:37:49.165287 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 69 ==== osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2941537490 0 3511520646) 0x2799a800 con 0x230c2000 -7971> 2014-11-03 11:37:49.165387 7f2911d06700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.165387, event: commit_sent, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7970> 2014-11-03 11:37:49.165417 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28228dc0 con 0x230c2000 -7969> 2014-11-03 11:37:49.165402 7f28ebc48700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.165130, event: header_read, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7968> 2014-11-03 11:37:49.165481 7f28ebc48700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.165134, event: throttled, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7967> 2014-11-03 11:37:49.165513 7f28ebc48700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.165267, event: all_read, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7966> 2014-11-03 11:37:49.165541 7f28ebc48700 5 -- op tracker -- seq: 2857, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7965> 2014-11-03 11:37:49.165598 7f2912507700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.165597, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7964> 2014-11-03 11:37:49.165633 7f2912507700 5 -- op tracker -- seq: 2856, time: 2014-11-03 11:37:49.165633, event: done, op: osd_sub_op(client.7914488.0:1855185 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166710 snapset=0=[]:[] snapc=0=[]) -7963> 2014-11-03 11:37:49.165689 7f28fe818700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.165688, event: reached_pg, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7962> 2014-11-03 11:37:49.165727 7f28fe818700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.165727, event: started, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7961> 2014-11-03 11:37:49.165852 7f28fe818700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.165852, event: started, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7960> 2014-11-03 11:37:49.165904 7f28fe818700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.165904, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7959> 2014-11-03 11:37:49.166023 7f2914d0c700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.166023, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7958> 2014-11-03 11:37:49.166247 7f291450b700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.166247, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7957> 2014-11-03 11:37:49.166291 7f2911d06700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.166291, event: commit_sent, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7956> 2014-11-03 11:37:49.166328 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23144000 con 0x230c2000 -7955> 2014-11-03 11:37:49.166736 7f2912507700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.166736, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7954> 2014-11-03 11:37:49.166752 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 70 ==== osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2177299988 0 3511520646) 0xd55d000 con 0x230c2000 -7953> 2014-11-03 11:37:49.166786 7f28ebc48700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.166595, event: header_read, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7952> 2014-11-03 11:37:49.166872 7f28ebc48700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.166600, event: throttled, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7951> 2014-11-03 11:37:49.166781 7f2912507700 5 -- op tracker -- seq: 2857, time: 2014-11-03 11:37:49.166780, event: done, op: osd_sub_op(client.7914488.0:1855187 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166711 snapset=0=[]:[] snapc=0=[]) -7950> 2014-11-03 11:37:49.166897 7f28ebc48700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.166738, event: all_read, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7949> 2014-11-03 11:37:49.166930 7f28ebc48700 5 -- op tracker -- seq: 2858, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7948> 2014-11-03 11:37:49.166993 7f290101d700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.166993, event: reached_pg, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7947> 2014-11-03 11:37:49.167017 7f290101d700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167017, event: started, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7946> 2014-11-03 11:37:49.167144 7f290101d700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167144, event: started, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7945> 2014-11-03 11:37:49.167178 7f290101d700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167177, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7944> 2014-11-03 11:37:49.167273 7f2914d0c700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167272, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7943> 2014-11-03 11:37:49.167475 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 71 ==== osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1089445848 0 3511520646) 0x2d575800 con 0x230c2000 -7942> 2014-11-03 11:37:49.167492 7f291450b700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167491, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7941> 2014-11-03 11:37:49.167515 7f28ebc48700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167390, event: header_read, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7940> 2014-11-03 11:37:49.167535 7f28ebc48700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167391, event: throttled, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7939> 2014-11-03 11:37:49.167539 7f2911d06700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167539, event: commit_sent, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7938> 2014-11-03 11:37:49.167553 7f28ebc48700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167465, event: all_read, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7937> 2014-11-03 11:37:49.167563 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x248b62c0 con 0x230c2000 -7936> 2014-11-03 11:37:49.167569 7f28ebc48700 5 -- op tracker -- seq: 2859, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7935> 2014-11-03 11:37:49.167625 7f28fe818700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167625, event: reached_pg, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7934> 2014-11-03 11:37:49.167650 7f28fe818700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167650, event: started, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7933> 2014-11-03 11:37:49.167780 7f28fe818700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167779, event: started, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7932> 2014-11-03 11:37:49.167820 7f28fe818700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167820, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7931> 2014-11-03 11:37:49.167926 7f2914d0c700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.167926, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7930> 2014-11-03 11:37:49.167907 7f2912507700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167907, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7929> 2014-11-03 11:37:49.167994 7f2912507700 5 -- op tracker -- seq: 2858, time: 2014-11-03 11:37:49.167994, event: done, op: osd_sub_op(client.7914488.0:1855188 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166712 snapset=0=[]:[] snapc=0=[]) -7928> 2014-11-03 11:37:49.168144 7f291450b700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.168144, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7927> 2014-11-03 11:37:49.168187 7f2911d06700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.168187, event: commit_sent, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7926> 2014-11-03 11:37:49.168218 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x248b7340 con 0x230c2000 -7925> 2014-11-03 11:37:49.168507 7f2912507700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.168507, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7924> 2014-11-03 11:37:49.168542 7f2912507700 5 -- op tracker -- seq: 2859, time: 2014-11-03 11:37:49.168542, event: done, op: osd_sub_op(client.7914488.0:1855189 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166713 snapset=0=[]:[] snapc=0=[]) -7923> 2014-11-03 11:37:49.169755 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 72 ==== osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1906263485 0 3511520646) 0x27000000 con 0x230c2000 -7922> 2014-11-03 11:37:49.169789 7f28ebc48700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.169665, event: header_read, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7921> 2014-11-03 11:37:49.169824 7f28ebc48700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.169667, event: throttled, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7920> 2014-11-03 11:37:49.169840 7f28ebc48700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.169744, event: all_read, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7919> 2014-11-03 11:37:49.169855 7f28ebc48700 5 -- op tracker -- seq: 2860, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7918> 2014-11-03 11:37:49.169902 7f290101d700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.169902, event: reached_pg, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7917> 2014-11-03 11:37:49.169949 7f290101d700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.169949, event: started, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7916> 2014-11-03 11:37:49.170089 7f290101d700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.170089, event: started, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7915> 2014-11-03 11:37:49.170177 7f290101d700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.170177, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7914> 2014-11-03 11:37:49.170293 7f2914d0c700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.170293, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7913> 2014-11-03 11:37:49.170524 7f291450b700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.170524, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7912> 2014-11-03 11:37:49.170565 7f2911d06700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.170565, event: commit_sent, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7911> 2014-11-03 11:37:49.170596 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d6ea580 con 0x230c2000 -7910> 2014-11-03 11:37:49.170861 7f2912507700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.170861, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7909> 2014-11-03 11:37:49.170867 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 73 ==== osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (811649469 0 3511520646) 0x24662800 con 0x230c2000 -7908> 2014-11-03 11:37:49.170896 7f28ebc48700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.170768, event: header_read, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7907> 2014-11-03 11:37:49.170949 7f28ebc48700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.170770, event: throttled, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7906> 2014-11-03 11:37:49.170902 7f2912507700 5 -- op tracker -- seq: 2860, time: 2014-11-03 11:37:49.170902, event: done, op: osd_sub_op(client.7914488.0:1855190 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166714 snapset=0=[]:[] snapc=0=[]) -7905> 2014-11-03 11:37:49.170982 7f28ebc48700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.170856, event: all_read, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7904> 2014-11-03 11:37:49.171024 7f28ebc48700 5 -- op tracker -- seq: 2861, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7903> 2014-11-03 11:37:49.171097 7f28fe818700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.171097, event: reached_pg, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7902> 2014-11-03 11:37:49.171128 7f28fe818700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.171128, event: started, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7901> 2014-11-03 11:37:49.171237 7f28fe818700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.171237, event: started, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7900> 2014-11-03 11:37:49.171271 7f28fe818700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.171271, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7899> 2014-11-03 11:37:49.171340 7f2914d0c700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.171339, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7898> 2014-11-03 11:37:49.171567 7f291450b700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.171567, event: journaled_completion_queued, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7897> 2014-11-03 11:37:49.171636 7f2911d06700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.171636, event: commit_sent, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7896> 2014-11-03 11:37:49.171672 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d6ea000 con 0x230c2000 -7895> 2014-11-03 11:37:49.172052 7f2912507700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.172051, event: sub_op_applied, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7894> 2014-11-03 11:37:49.172095 7f2912507700 5 -- op tracker -- seq: 2861, time: 2014-11-03 11:37:49.172095, event: done, op: osd_sub_op(client.7914488.0:1855191 6.773 d2be0773/rbd_data.49eaa02ae8944a.0000000000001039/head//6 [] v 105950'12166715 snapset=0=[]:[] snapc=0=[]) -7893> 2014-11-03 11:37:49.205160 7f28f45d1700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 45 ==== osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2514487133 0 3511520646) 0x24d6c000 con 0x22fef580 -7892> 2014-11-03 11:37:49.205215 7f28f45d1700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.204952, event: header_read, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7891> 2014-11-03 11:37:49.205232 7f28f45d1700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.204956, event: throttled, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7890> 2014-11-03 11:37:49.205241 7f28f45d1700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.205140, event: all_read, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7889> 2014-11-03 11:37:49.205282 7f28f45d1700 5 -- op tracker -- seq: 2862, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7888> 2014-11-03 11:37:49.205355 7f28ff019700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.205355, event: reached_pg, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7887> 2014-11-03 11:37:49.205633 7f28ff019700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.205633, event: started, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7886> 2014-11-03 11:37:49.205844 7f28ff019700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.205844, event: started, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7885> 2014-11-03 11:37:49.205917 7f28ff019700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.205917, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7884> 2014-11-03 11:37:49.206028 7f2914d0c700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.206028, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7883> 2014-11-03 11:37:49.206322 7f291450b700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.206321, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7882> 2014-11-03 11:37:49.206453 7f2911d06700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.206453, event: commit_sent, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7881> 2014-11-03 11:37:49.206495 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- osd_sub_op_reply(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d6eab00 con 0x22fef580 -7880> 2014-11-03 11:37:49.206865 7f2912507700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.206865, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7879> 2014-11-03 11:37:49.206891 7f28f45d1700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 46 ==== osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2574048956 0 3511520646) 0x2d47b000 con 0x22fef580 -7878> 2014-11-03 11:37:49.206929 7f28f45d1700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.206757, event: header_read, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7877> 2014-11-03 11:37:49.206936 7f2912507700 5 -- op tracker -- seq: 2862, time: 2014-11-03 11:37:49.206936, event: done, op: osd_sub_op(client.7868288.0:25603289 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320277 snapset=0=[]:[] snapc=0=[]) -7876> 2014-11-03 11:37:49.206970 7f28f45d1700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.206759, event: throttled, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7875> 2014-11-03 11:37:49.206987 7f28f45d1700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.206878, event: all_read, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7874> 2014-11-03 11:37:49.207013 7f28f45d1700 5 -- op tracker -- seq: 2863, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7873> 2014-11-03 11:37:49.207076 7f290181e700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.207075, event: reached_pg, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7872> 2014-11-03 11:37:49.207114 7f290181e700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.207113, event: started, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7871> 2014-11-03 11:37:49.207260 7f290181e700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.207259, event: started, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7870> 2014-11-03 11:37:49.207307 7f290181e700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.207306, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7869> 2014-11-03 11:37:49.207397 7f2914d0c700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.207397, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7868> 2014-11-03 11:37:49.207775 7f291450b700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.207775, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7867> 2014-11-03 11:37:49.207884 7f2911d06700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.207883, event: commit_sent, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7866> 2014-11-03 11:37:49.207919 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- osd_sub_op_reply(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] ondisk, result = 0) v2 -- ?+0 0x27417b00 con 0x22fef580 -7865> 2014-11-03 11:37:49.208189 7f2912507700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.208189, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7864> 2014-11-03 11:37:49.208275 7f2912507700 5 -- op tracker -- seq: 2863, time: 2014-11-03 11:37:49.208275, event: done, op: osd_sub_op(client.7868288.0:25603290 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320278 snapset=0=[]:[] snapc=0=[]) -7863> 2014-11-03 11:37:49.208656 7f28f45d1700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 47 ==== osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1963869252 0 3511520646) 0x2d47d400 con 0x22fef580 -7862> 2014-11-03 11:37:49.208687 7f28f45d1700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.208573, event: header_read, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7861> 2014-11-03 11:37:49.208704 7f28f45d1700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.208574, event: throttled, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7860> 2014-11-03 11:37:49.208714 7f28f45d1700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.208642, event: all_read, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7859> 2014-11-03 11:37:49.208727 7f28f45d1700 5 -- op tracker -- seq: 2864, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7858> 2014-11-03 11:37:49.208866 7f28ff019700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.208865, event: reached_pg, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7857> 2014-11-03 11:37:49.208899 7f28ff019700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.208898, event: started, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7856> 2014-11-03 11:37:49.209010 7f28ff019700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.209009, event: started, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7855> 2014-11-03 11:37:49.209056 7f28ff019700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.209056, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7854> 2014-11-03 11:37:49.209130 7f2914d0c700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.209130, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7853> 2014-11-03 11:37:49.209604 7f291450b700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.209604, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7852> 2014-11-03 11:37:49.209734 7f2911d06700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.209734, event: commit_sent, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7851> 2014-11-03 11:37:49.209770 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- osd_sub_op_reply(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3110d2c0 con 0x22fef580 -7850> 2014-11-03 11:37:49.209964 7f2912507700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.209963, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7849> 2014-11-03 11:37:49.210014 7f2912507700 5 -- op tracker -- seq: 2864, time: 2014-11-03 11:37:49.210014, event: done, op: osd_sub_op(client.7868288.0:25603291 6.54c 1b69f54c/rbd_data.52354a3c982097.0000000000000330/head//6 [] v 105950'16320279 snapset=0=[]:[] snapc=0=[]) -7848> 2014-11-03 11:37:49.220460 7f2918d14700 5 osd.21 105950 tick -7847> 2014-11-03 11:37:49.220510 7f2918d14700 10 monclient: _send_mon_message to mon.a at 192.168.5.201:6789/0 -7846> 2014-11-03 11:37:49.220524 7f2918d14700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:6789/0 -- pg_stats(4 pgs tid 5 v 0) v1 -- ?+0 0x314a0600 con 0x22c29160 -7845> 2014-11-03 11:37:49.222856 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 84 ==== osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3386528902 0 3511520646) 0x3348c00 con 0x22fef420 -7844> 2014-11-03 11:37:49.222895 7f28f41cd700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.222742, event: header_read, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7843> 2014-11-03 11:37:49.222920 7f28f41cd700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.222745, event: throttled, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7842> 2014-11-03 11:37:49.222940 7f28f41cd700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.222840, event: all_read, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7841> 2014-11-03 11:37:49.222958 7f28f41cd700 5 -- op tracker -- seq: 2865, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7840> 2014-11-03 11:37:49.223028 7f290101d700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223028, event: reached_pg, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7839> 2014-11-03 11:37:49.223063 7f290101d700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223063, event: started, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7838> 2014-11-03 11:37:49.223162 7f290101d700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223162, event: started, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7837> 2014-11-03 11:37:49.223194 7f290101d700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223194, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7836> 2014-11-03 11:37:49.223263 7f2914d0c700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223263, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7835> 2014-11-03 11:37:49.223465 7f291450b700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223464, event: journaled_completion_queued, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7834> 2014-11-03 11:37:49.223538 7f2911d06700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223538, event: commit_sent, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7833> 2014-11-03 11:37:49.223569 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3110e340 con 0x22fef420 -7832> 2014-11-03 11:37:49.223796 7f2912507700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223796, event: sub_op_applied, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7831> 2014-11-03 11:37:49.223833 7f2912507700 5 -- op tracker -- seq: 2865, time: 2014-11-03 11:37:49.223833, event: done, op: osd_sub_op(client.7868641.0:2509826 6.755 c5de3f55/rbd_data.59709c2ae8944a.0000000000000440/head//6 [] v 105950'16057564 snapset=0=[]:[] snapc=0=[]) -7830> 2014-11-03 11:37:49.280273 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.97 192.168.5.7:0/21285 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.277874) v2 ==== 47+0+0 (2218022021 0 0) 0x23305c00 con 0x24537c60 -7829> 2014-11-03 11:37:49.280317 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.97 192.168.5.7:0/21285 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.277874) v2 ==== 47+0+0 (2218022021 0 0) 0x6352000 con 0x2484a2c0 -7828> 2014-11-03 11:37:49.280369 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/21285 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.277874) v2 -- ?+0 0x23482800 con 0x24537c60 -7827> 2014-11-03 11:37:49.280521 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/21285 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.277874) v2 -- ?+0 0x25835800 con 0x2484a2c0 -7826> 2014-11-03 11:37:49.283985 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.114 192.168.5.9:0/17408 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.282328) v2 ==== 47+0+0 (3225746380 0 0) 0x2295ae00 con 0x2469e580 -7825> 2014-11-03 11:37:49.284057 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/17408 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.282328) v2 -- ?+0 0x23305c00 con 0x2469e580 -7824> 2014-11-03 11:37:49.284277 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.114 192.168.5.9:0/17408 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.282328) v2 ==== 47+0+0 (3225746380 0 0) 0x80ff800 con 0x2484a000 -7823> 2014-11-03 11:37:49.284360 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/17408 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.282328) v2 -- ?+0 0x6352000 con 0x2484a000 -7822> 2014-11-03 11:37:49.304839 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.68 192.168.5.5:0/8133 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.302865) v2 ==== 47+0+0 (1381397686 0 0) 0x2b1ff800 con 0x24292420 -7821> 2014-11-03 11:37:49.304852 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.68 192.168.5.5:0/8133 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.302865) v2 ==== 47+0+0 (1381397686 0 0) 0x17ae8800 con 0x24321420 -7820> 2014-11-03 11:37:49.304966 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/8133 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.302865) v2 -- ?+0 0x80ff800 con 0x24292420 -7819> 2014-11-03 11:37:49.305056 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/8133 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.302865) v2 -- ?+0 0x2295ae00 con 0x24321420 -7818> 2014-11-03 11:37:49.316699 7f28ef783700 1 -- 192.168.5.2:6803/7606 <== osd.74 192.168.5.6:6826/9268 40 ==== osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (123521458 0 3511520646) 0x29e11000 con 0x2308bdc0 -7817> 2014-11-03 11:37:49.316775 7f28ef783700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.316515, event: header_read, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7816> 2014-11-03 11:37:49.316825 7f28ef783700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.316531, event: throttled, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7815> 2014-11-03 11:37:49.316839 7f28ef783700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.316672, event: all_read, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7814> 2014-11-03 11:37:49.316852 7f28ef783700 5 -- op tracker -- seq: 2866, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7813> 2014-11-03 11:37:49.316987 7f28ff81a700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.316986, event: reached_pg, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7812> 2014-11-03 11:37:49.317033 7f28ff81a700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.317033, event: started, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7811> 2014-11-03 11:37:49.317133 7f28ff81a700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.317133, event: started, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7810> 2014-11-03 11:37:49.317167 7f28ff81a700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.317167, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7809> 2014-11-03 11:37:49.317230 7f2914d0c700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.317230, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7808> 2014-11-03 11:37:49.317519 7f291450b700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.317519, event: journaled_completion_queued, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7807> 2014-11-03 11:37:49.317699 7f2911d06700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.317699, event: commit_sent, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7806> 2014-11-03 11:37:49.317740 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6826/9268 -- osd_sub_op_reply(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3110e080 con 0x2308bdc0 -7805> 2014-11-03 11:37:49.318096 7f2912507700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.318095, event: sub_op_applied, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7804> 2014-11-03 11:37:49.318186 7f2912507700 5 -- op tracker -- seq: 2866, time: 2014-11-03 11:37:49.318186, event: done, op: osd_sub_op(client.7869197.0:1605876 6.749 78a55749/rbd_data.5438f97ef36e60.0000000000000441/head//6 [] v 105950'7802352 snapset=0=[]:[] snapc=0=[]) -7803> 2014-11-03 11:37:49.343045 7f28c4e00700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x23bd8dc0 sd=67 :6802 s=0 pgs=0 cs=0 l=0 c=0x245379a0).accept sd=67 192.168.9.3:49685/0 -7802> 2014-11-03 11:37:49.344333 7f28c4e00700 1 -- 192.168.5.2:6802/7606 <== client.9545673 192.168.9.3:0/1015168 1 ==== osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) v4 ==== 253+0+12288 (2107326395 0 1421846735) 0x235dfc80 con 0x245379a0 -7801> 2014-11-03 11:37:49.344400 7f28c4e00700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.344144, event: header_read, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7800> 2014-11-03 11:37:49.344444 7f28c4e00700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.344174, event: throttled, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7799> 2014-11-03 11:37:49.344474 7f28c4e00700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.344298, event: all_read, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7798> 2014-11-03 11:37:49.344500 7f28c4e00700 5 -- op tracker -- seq: 2867, time: 0.000000, event: dispatched, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7797> 2014-11-03 11:37:49.344668 7f290001b700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.344667, event: reached_pg, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7796> 2014-11-03 11:37:49.344720 7f28c4e00700 1 -- 192.168.5.2:6802/7606 <== client.9545673 192.168.9.3:0/1015168 2 ==== osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) v4 ==== 253+0+4096 (1937646797 0 2160843485) 0x5fee280 con 0x245379a0 -7795> 2014-11-03 11:37:49.344761 7f28c4e00700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.344622, event: header_read, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7794> 2014-11-03 11:37:49.344792 7f28c4e00700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.344624, event: throttled, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7793> 2014-11-03 11:37:49.344813 7f28c4e00700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.344708, event: all_read, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7792> 2014-11-03 11:37:49.344859 7f28c4e00700 5 -- op tracker -- seq: 2868, time: 0.000000, event: dispatched, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7791> 2014-11-03 11:37:49.344951 7f290001b700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.344951, event: started, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7790> 2014-11-03 11:37:49.345004 7f290001b700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.345004, event: started, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7789> 2014-11-03 11:37:49.345152 7f290001b700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.345152, event: waiting for subops from 1,70, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7788> 2014-11-03 11:37:49.345208 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- osd_sub_op(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] v 105950'819291 snapset=0=[]:[] snapc=0=[]) v11 -- ?+14909 0x2d574c00 con 0x22fbc160 -7787> 2014-11-03 11:37:49.345285 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- osd_sub_op(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] v 105950'819291 snapset=0=[]:[] snapc=0=[]) v11 -- ?+14909 0x3348c00 con 0x22fe0000 -7786> 2014-11-03 11:37:49.345529 7f290001b700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.345529, event: commit_queued_for_journal_write, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7785> 2014-11-03 11:37:49.345613 7f28fd816700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.345613, event: reached_pg, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7784> 2014-11-03 11:37:49.345650 7f2914d0c700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.345650, event: write_thread_in_journal_buffer, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7783> 2014-11-03 11:37:49.345716 7f28fd816700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.345715, event: started, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7782> 2014-11-03 11:37:49.345766 7f28fd816700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.345766, event: started, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7781> 2014-11-03 11:37:49.345851 7f28fd816700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.345851, event: waiting for subops from 1,70, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7780> 2014-11-03 11:37:49.345865 7f291450b700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.345865, event: journaled_completion_queued, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7779> 2014-11-03 11:37:49.345899 7f28fd816700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- osd_sub_op(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] v 105950'819292 snapset=0=[]:[] snapc=0=[]) v11 -- ?+6717 0x28fef400 con 0x22fbc160 -7778> 2014-11-03 11:37:49.345959 7f28fd816700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- osd_sub_op(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] v 105950'819292 snapset=0=[]:[] snapc=0=[]) v11 -- ?+6717 0x27ed9800 con 0x22fe0000 -7777> 2014-11-03 11:37:49.346074 7f28fd816700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.346074, event: commit_queued_for_journal_write, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7776> 2014-11-03 11:37:49.346179 7f2914d0c700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.346179, event: write_thread_in_journal_buffer, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7775> 2014-11-03 11:37:49.346169 7f2911d06700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.346168, event: op_commit, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7774> 2014-11-03 11:37:49.346502 7f291450b700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.346501, event: journaled_completion_queued, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7773> 2014-11-03 11:37:49.346573 7f2911d06700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.346572, event: op_commit, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7772> 2014-11-03 11:37:49.346704 7f2912507700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.346704, event: op_applied, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7771> 2014-11-03 11:37:49.347329 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 46 ==== osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (3759804665 0 0) 0x2fcac000 con 0x22fbc160 -7770> 2014-11-03 11:37:49.347357 7f28f6df9700 5 -- op tracker -- seq: 2869, time: 2014-11-03 11:37:49.347259, event: header_read, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7769> 2014-11-03 11:37:49.347371 7f28f6df9700 5 -- op tracker -- seq: 2869, time: 2014-11-03 11:37:49.347261, event: throttled, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7768> 2014-11-03 11:37:49.347379 7f28f6df9700 5 -- op tracker -- seq: 2869, time: 2014-11-03 11:37:49.347320, event: all_read, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7767> 2014-11-03 11:37:49.347390 7f28f6df9700 5 -- op tracker -- seq: 2869, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7766> 2014-11-03 11:37:49.347436 7f290001b700 5 -- op tracker -- seq: 2869, time: 2014-11-03 11:37:49.347435, event: reached_pg, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7765> 2014-11-03 11:37:49.347457 7f290001b700 5 -- op tracker -- seq: 2869, time: 2014-11-03 11:37:49.347457, event: started, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7764> 2014-11-03 11:37:49.347468 7f290001b700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:49.347468, event: sub_op_commit_rec, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7763> 2014-11-03 11:37:49.347487 7f290001b700 5 -- op tracker -- seq: 2869, time: 2014-11-03 11:37:49.347487, event: done, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7762> 2014-11-03 11:37:49.347644 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 47 ==== osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (735211585 0 0) 0x28b16840 con 0x22fbc160 -7761> 2014-11-03 11:37:49.347706 7f28f6df9700 5 -- op tracker -- seq: 2870, time: 2014-11-03 11:37:49.347578, event: header_read, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7760> 2014-11-03 11:37:49.347725 7f28f6df9700 5 -- op tracker -- seq: 2870, time: 2014-11-03 11:37:49.347579, event: throttled, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7759> 2014-11-03 11:37:49.347741 7f28f6df9700 5 -- op tracker -- seq: 2870, time: 2014-11-03 11:37:49.347636, event: all_read, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7758> 2014-11-03 11:37:49.347756 7f28f6df9700 5 -- op tracker -- seq: 2870, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7757> 2014-11-03 11:37:49.347816 7f28fd816700 5 -- op tracker -- seq: 2870, time: 2014-11-03 11:37:49.347816, event: reached_pg, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7756> 2014-11-03 11:37:49.347844 7f28fd816700 5 -- op tracker -- seq: 2870, time: 2014-11-03 11:37:49.347844, event: started, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7755> 2014-11-03 11:37:49.347859 7f28fd816700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.347859, event: sub_op_commit_rec, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7754> 2014-11-03 11:37:49.347885 7f28fd816700 5 -- op tracker -- seq: 2870, time: 2014-11-03 11:37:49.347885, event: done, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7753> 2014-11-03 11:37:49.347923 7f2912507700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:49.347922, event: op_applied, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7752> 2014-11-03 11:37:49.382425 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.89 192.168.5.7:0/24271 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.380249) v2 ==== 47+0+0 (1501302579 0 0) 0x2369d200 con 0x24925580 -7751> 2014-11-03 11:37:49.382496 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/24271 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.380249) v2 -- ?+0 0x2b1ff800 con 0x24925580 -7750> 2014-11-03 11:37:49.382525 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.89 192.168.5.7:0/24271 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.380249) v2 ==== 47+0+0 (1501302579 0 0) 0x147bec00 con 0x242292c0 -7749> 2014-11-03 11:37:49.382587 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/24271 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.380249) v2 -- ?+0 0x17ae8800 con 0x242292c0 -7748> 2014-11-03 11:37:49.386267 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.49 192.168.5.4:0/21769 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.384584) v2 ==== 47+0+0 (1188851381 0 0) 0x17ae8800 con 0x242296e0 -7747> 2014-11-03 11:37:49.386327 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/21769 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.384584) v2 -- ?+0 0x147bec00 con 0x242296e0 -7746> 2014-11-03 11:37:49.386327 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.49 192.168.5.4:0/21769 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.384584) v2 ==== 47+0+0 (1188851381 0 0) 0x2b1ff800 con 0x249259a0 -7745> 2014-11-03 11:37:49.386405 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/21769 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.384584) v2 -- ?+0 0x2369d200 con 0x249259a0 -7744> 2014-11-03 11:37:49.391916 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 38 ==== osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2624214911 0 3511520646) 0x3125c800 con 0x231ab580 -7743> 2014-11-03 11:37:49.391972 7f28ea430700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.391741, event: header_read, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7742> 2014-11-03 11:37:49.392011 7f28ea430700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.391744, event: throttled, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7741> 2014-11-03 11:37:49.392025 7f28ea430700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.391906, event: all_read, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7740> 2014-11-03 11:37:49.392058 7f28ea430700 5 -- op tracker -- seq: 2871, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7739> 2014-11-03 11:37:49.392142 7f290181e700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.392141, event: reached_pg, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7738> 2014-11-03 11:37:49.392195 7f290181e700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.392195, event: started, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7737> 2014-11-03 11:37:49.392346 7f290181e700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.392346, event: started, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7736> 2014-11-03 11:37:49.392390 7f290181e700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.392390, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7735> 2014-11-03 11:37:49.392479 7f2914d0c700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.392478, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7734> 2014-11-03 11:37:49.392705 7f291450b700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.392705, event: journaled_completion_queued, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7733> 2014-11-03 11:37:49.393042 7f2911d06700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.393041, event: commit_sent, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7732> 2014-11-03 11:37:49.393111 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3110db00 con 0x231ab580 -7731> 2014-11-03 11:37:49.393603 7f2912507700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.393603, event: sub_op_applied, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7730> 2014-11-03 11:37:49.393721 7f2912507700 5 -- op tracker -- seq: 2871, time: 2014-11-03 11:37:49.393721, event: done, op: osd_sub_op(client.7869095.0:16207356 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629960 snapset=0=[]:[] snapc=0=[]) -7729> 2014-11-03 11:37:49.395120 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 85 ==== MOSDPGPush(6.58e 105950 [PushOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, version: 105945'11622058, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1847717271 0 0) 0x29fcd000 con 0x22fbc420 -7728> 2014-11-03 11:37:49.395164 7f28f72fe700 5 -- op tracker -- seq: 2872, time: 2014-11-03 11:37:49.383354, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, version: 105945'11622058, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7727> 2014-11-03 11:37:49.395184 7f28f72fe700 5 -- op tracker -- seq: 2872, time: 2014-11-03 11:37:49.383357, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, version: 105945'11622058, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7726> 2014-11-03 11:37:49.395224 7f28f72fe700 5 -- op tracker -- seq: 2872, time: 2014-11-03 11:37:49.395104, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, version: 105945'11622058, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7725> 2014-11-03 11:37:49.395240 7f28f72fe700 5 -- op tracker -- seq: 2872, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, version: 105945'11622058, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7724> 2014-11-03 11:37:49.395316 7f28fe818700 5 -- op tracker -- seq: 2872, time: 2014-11-03 11:37:49.395316, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, version: 105945'11622058, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7723> 2014-11-03 11:37:49.395540 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 39 ==== osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2064344826 0 3511520646) 0x2c691800 con 0x231ab580 -7722> 2014-11-03 11:37:49.395581 7f28ea430700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.395412, event: header_read, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7721> 2014-11-03 11:37:49.395610 7f28ea430700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.395414, event: throttled, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7720> 2014-11-03 11:37:49.395628 7f28ea430700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.395518, event: all_read, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7719> 2014-11-03 11:37:49.395644 7f28ea430700 5 -- op tracker -- seq: 2873, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7718> 2014-11-03 11:37:49.395681 7f28fe818700 5 -- op tracker -- seq: 2872, time: 2014-11-03 11:37:49.395681, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6, version: 105945'11622058, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35dd3d8e/rbd_data.74ae9c3be03aff.0000000000001750/head//6@105945'11622058, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7717> 2014-11-03 11:37:49.395737 7f28ff019700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.395736, event: reached_pg, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7716> 2014-11-03 11:37:49.395770 7f28ff019700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.395770, event: started, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7715> 2014-11-03 11:37:49.395895 7f28ff019700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.395895, event: started, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7714> 2014-11-03 11:37:49.395934 7f28ff019700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.395934, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7713> 2014-11-03 11:37:49.396209 7f2914d0c700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.396209, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7712> 2014-11-03 11:37:49.397267 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.95 192.168.5.7:0/14919 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.394881) v2 ==== 47+0+0 (308722831 0 0) 0x6425400 con 0x2484a580 -7711> 2014-11-03 11:37:49.397286 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 40 ==== osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2415833529 0 3511520646) 0x2c692400 con 0x231ab580 -7710> 2014-11-03 11:37:49.397267 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.95 192.168.5.7:0/14919 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.394881) v2 ==== 47+0+0 (308722831 0 0) 0x80d2600 con 0x24321000 -7709> 2014-11-03 11:37:49.397327 7f28ea430700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.397176, event: header_read, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7708> 2014-11-03 11:37:49.397325 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/14919 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.394881) v2 -- ?+0 0x2b1ff800 con 0x2484a580 -7707> 2014-11-03 11:37:49.397367 7f28ea430700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.397178, event: throttled, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7706> 2014-11-03 11:37:49.397387 7f28ea430700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.397271, event: all_read, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7705> 2014-11-03 11:37:49.397407 7f28ea430700 5 -- op tracker -- seq: 2874, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7704> 2014-11-03 11:37:49.397431 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/14919 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.394881) v2 -- ?+0 0x17ae8800 con 0x24321000 -7703> 2014-11-03 11:37:49.397475 7f290181e700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.397475, event: reached_pg, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7702> 2014-11-03 11:37:49.397504 7f290181e700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.397503, event: started, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7701> 2014-11-03 11:37:49.397653 7f290181e700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.397652, event: started, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7700> 2014-11-03 11:37:49.397697 7f290181e700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.397697, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7699> 2014-11-03 11:37:49.400839 7f291450b700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.400838, event: journaled_completion_queued, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7698> 2014-11-03 11:37:49.400905 7f2911d06700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.400905, event: commit_sent, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7697> 2014-11-03 11:37:49.400921 7f2914d0c700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.400921, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7696> 2014-11-03 11:37:49.400954 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] ondisk, result = 0) v2 -- ?+0 0x279fb840 con 0x231ab580 -7695> 2014-11-03 11:37:49.401131 7f291450b700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.401131, event: journaled_completion_queued, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7694> 2014-11-03 11:37:49.401191 7f2911d06700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.401191, event: commit_sent, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7693> 2014-11-03 11:37:49.401226 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] ondisk, result = 0) v2 -- ?+0 0x279fb580 con 0x231ab580 -7692> 2014-11-03 11:37:49.401401 7f2912507700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.401401, event: sub_op_applied, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7691> 2014-11-03 11:37:49.401478 7f2912507700 5 -- op tracker -- seq: 2873, time: 2014-11-03 11:37:49.401478, event: done, op: osd_sub_op(client.7869095.0:16207357 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629961 snapset=0=[]:[] snapc=0=[]) -7690> 2014-11-03 11:37:49.401626 7f2912507700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.401626, event: sub_op_applied, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7689> 2014-11-03 11:37:49.401676 7f2912507700 5 -- op tracker -- seq: 2874, time: 2014-11-03 11:37:49.401676, event: done, op: osd_sub_op(client.7869095.0:16207358 6.a79 60cfba79/rbd_data.4f51ee466e8d14.00000000000000e1/head//6 [] v 105950'8629962 snapset=0=[]:[] snapc=0=[]) -7688> 2014-11-03 11:37:49.409660 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 152 ==== osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1484062653 0 3511520646) 0x29e1f400 con 0x22fefdc0 -7687> 2014-11-03 11:37:49.409716 7f28f36c2700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.409314, event: header_read, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7686> 2014-11-03 11:37:49.409744 7f28f36c2700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.409315, event: throttled, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7685> 2014-11-03 11:37:49.409766 7f28f36c2700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.409636, event: all_read, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7684> 2014-11-03 11:37:49.409806 7f28f36c2700 5 -- op tracker -- seq: 2875, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7683> 2014-11-03 11:37:49.409997 7f290101d700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.409997, event: reached_pg, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7682> 2014-11-03 11:37:49.410084 7f290101d700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.410084, event: started, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7681> 2014-11-03 11:37:49.410300 7f290101d700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.410299, event: started, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7680> 2014-11-03 11:37:49.410358 7f290101d700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.410358, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7679> 2014-11-03 11:37:49.410478 7f2914d0c700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.410478, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7678> 2014-11-03 11:37:49.410710 7f291450b700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.410710, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7677> 2014-11-03 11:37:49.410774 7f2911d06700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.410774, event: commit_sent, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7676> 2014-11-03 11:37:49.410828 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x279fc080 con 0x22fefdc0 -7675> 2014-11-03 11:37:49.411135 7f2912507700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.411135, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7674> 2014-11-03 11:37:49.411187 7f2912507700 5 -- op tracker -- seq: 2875, time: 2014-11-03 11:37:49.411187, event: done, op: osd_sub_op(client.8703713.0:32254358 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953134 snapset=0=[]:[] snapc=0=[]) -7673> 2014-11-03 11:37:49.427184 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.41 192.168.5.3:0/28523 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.426304) v2 ==== 47+0+0 (3191661746 0 0) 0x3501400 con 0x23b1c840 -7672> 2014-11-03 11:37:49.427227 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.41 192.168.5.3:0/28523 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.426304) v2 ==== 47+0+0 (3191661746 0 0) 0x23e31000 con 0x242c09a0 -7671> 2014-11-03 11:37:49.427260 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/28523 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.426304) v2 -- ?+0 0x6425400 con 0x23b1c840 -7670> 2014-11-03 11:37:49.427347 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/28523 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.426304) v2 -- ?+0 0x80d2600 con 0x242c09a0 -7669> 2014-11-03 11:37:49.429254 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 153 ==== osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2843999378 0 3511520646) 0x28ea3c00 con 0x22fefdc0 -7668> 2014-11-03 11:37:49.429289 7f28f36c2700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429153, event: header_read, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7667> 2014-11-03 11:37:49.429302 7f28f36c2700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429155, event: throttled, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7666> 2014-11-03 11:37:49.429311 7f28f36c2700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429242, event: all_read, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7665> 2014-11-03 11:37:49.429319 7f28f36c2700 5 -- op tracker -- seq: 2876, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7664> 2014-11-03 11:37:49.429405 7f28fe818700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429404, event: reached_pg, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7663> 2014-11-03 11:37:49.429456 7f28fe818700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429456, event: started, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7662> 2014-11-03 11:37:49.429604 7f28fe818700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429604, event: started, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7661> 2014-11-03 11:37:49.429653 7f28fe818700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429653, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7660> 2014-11-03 11:37:49.429740 7f2914d0c700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.429739, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7659> 2014-11-03 11:37:49.430094 7f291450b700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.430094, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7658> 2014-11-03 11:37:49.430173 7f2911d06700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.430173, event: commit_sent, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7657> 2014-11-03 11:37:49.430200 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x922db80 con 0x22fefdc0 -7656> 2014-11-03 11:37:49.430418 7f2912507700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.430418, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7655> 2014-11-03 11:37:49.430474 7f2912507700 5 -- op tracker -- seq: 2876, time: 2014-11-03 11:37:49.430474, event: done, op: osd_sub_op(client.8703713.0:32254363 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953135 snapset=0=[]:[] snapc=0=[]) -7654> 2014-11-03 11:37:49.433764 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 154 ==== osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (828612278 0 3511520646) 0x2605a800 con 0x22fefdc0 -7653> 2014-11-03 11:37:49.433805 7f28f36c2700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.433682, event: header_read, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7652> 2014-11-03 11:37:49.433833 7f28f36c2700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.433683, event: throttled, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7651> 2014-11-03 11:37:49.433852 7f28f36c2700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.433756, event: all_read, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7650> 2014-11-03 11:37:49.433865 7f28f36c2700 5 -- op tracker -- seq: 2877, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7649> 2014-11-03 11:37:49.434002 7f290101d700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434002, event: reached_pg, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7648> 2014-11-03 11:37:49.434068 7f290101d700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434068, event: started, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7647> 2014-11-03 11:37:49.434253 7f290101d700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434252, event: started, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7646> 2014-11-03 11:37:49.434305 7f290101d700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434305, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7645> 2014-11-03 11:37:49.434385 7f2914d0c700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434385, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7644> 2014-11-03 11:37:49.434620 7f291450b700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434620, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7643> 2014-11-03 11:37:49.434672 7f2911d06700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434672, event: commit_sent, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7642> 2014-11-03 11:37:49.434692 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x922c580 con 0x22fefdc0 -7641> 2014-11-03 11:37:49.434949 7f2912507700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434949, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7640> 2014-11-03 11:37:49.434989 7f2912507700 5 -- op tracker -- seq: 2877, time: 2014-11-03 11:37:49.434989, event: done, op: osd_sub_op(client.8703713.0:32254365 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953136 snapset=0=[]:[] snapc=0=[]) -7639> 2014-11-03 11:37:49.444622 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 155 ==== osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3490091919 0 3511520646) 0x2cff5400 con 0x22fefdc0 -7638> 2014-11-03 11:37:49.444691 7f28f36c2700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.444463, event: header_read, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7637> 2014-11-03 11:37:49.444717 7f28f36c2700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.444465, event: throttled, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7636> 2014-11-03 11:37:49.444739 7f28f36c2700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.444600, event: all_read, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7635> 2014-11-03 11:37:49.444754 7f28f36c2700 5 -- op tracker -- seq: 2878, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7634> 2014-11-03 11:37:49.444912 7f28fe818700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.444911, event: reached_pg, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7633> 2014-11-03 11:37:49.444967 7f28fe818700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.444967, event: started, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7632> 2014-11-03 11:37:49.445114 7f28fe818700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.445114, event: started, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7631> 2014-11-03 11:37:49.445163 7f28fe818700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.445162, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7630> 2014-11-03 11:37:49.445294 7f2914d0c700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.445294, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7629> 2014-11-03 11:37:49.445601 7f291450b700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.445601, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7628> 2014-11-03 11:37:49.445667 7f2911d06700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.445667, event: commit_sent, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7627> 2014-11-03 11:37:49.445705 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 156 ==== osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (604852347 0 3511520646) 0x96ea000 con 0x22fefdc0 -7626> 2014-11-03 11:37:49.445712 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x922cdc0 con 0x22fefdc0 -7625> 2014-11-03 11:37:49.445748 7f28f36c2700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.445485, event: header_read, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7624> 2014-11-03 11:37:49.445762 7f28f36c2700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.445487, event: throttled, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7623> 2014-11-03 11:37:49.445771 7f28f36c2700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.445686, event: all_read, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7622> 2014-11-03 11:37:49.445801 7f28f36c2700 5 -- op tracker -- seq: 2879, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7621> 2014-11-03 11:37:49.445913 7f290101d700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.445913, event: reached_pg, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7620> 2014-11-03 11:37:49.445968 7f290101d700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.445968, event: started, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7619> 2014-11-03 11:37:49.446130 7f290101d700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.446130, event: started, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7618> 2014-11-03 11:37:49.446192 7f290101d700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.446192, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7617> 2014-11-03 11:37:49.446297 7f2914d0c700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.446297, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7616> 2014-11-03 11:37:49.446306 7f2912507700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.446306, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7615> 2014-11-03 11:37:49.446358 7f2912507700 5 -- op tracker -- seq: 2878, time: 2014-11-03 11:37:49.446358, event: done, op: osd_sub_op(client.8703713.0:32254366 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953137 snapset=0=[]:[] snapc=0=[]) -7614> 2014-11-03 11:37:49.446481 7f291450b700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.446481, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7613> 2014-11-03 11:37:49.446563 7f2911d06700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.446563, event: commit_sent, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7612> 2014-11-03 11:37:49.446589 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x240d3600 con 0x22fefdc0 -7611> 2014-11-03 11:37:49.446793 7f2912507700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.446793, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7610> 2014-11-03 11:37:49.446834 7f2912507700 5 -- op tracker -- seq: 2879, time: 2014-11-03 11:37:49.446834, event: done, op: osd_sub_op(client.8703713.0:32254367 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953138 snapset=0=[]:[] snapc=0=[]) -7609> 2014-11-03 11:37:49.449799 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 157 ==== osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2895872607 0 3511520646) 0x29e1e800 con 0x22fefdc0 -7608> 2014-11-03 11:37:49.449827 7f28f36c2700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.449684, event: header_read, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7607> 2014-11-03 11:37:49.449840 7f28f36c2700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.449686, event: throttled, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7606> 2014-11-03 11:37:49.449848 7f28f36c2700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.449788, event: all_read, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7605> 2014-11-03 11:37:49.449856 7f28f36c2700 5 -- op tracker -- seq: 2880, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7604> 2014-11-03 11:37:49.449976 7f28fe818700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.449976, event: reached_pg, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7603> 2014-11-03 11:37:49.450030 7f28fe818700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.450030, event: started, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7602> 2014-11-03 11:37:49.450181 7f28fe818700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.450181, event: started, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7601> 2014-11-03 11:37:49.450227 7f28fe818700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.450227, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7600> 2014-11-03 11:37:49.450336 7f2914d0c700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.450336, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7599> 2014-11-03 11:37:49.450574 7f291450b700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.450573, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7598> 2014-11-03 11:37:49.450636 7f2911d06700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.450636, event: commit_sent, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7597> 2014-11-03 11:37:49.450690 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x240d3340 con 0x22fefdc0 -7596> 2014-11-03 11:37:49.450998 7f2912507700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.450998, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7595> 2014-11-03 11:37:49.451050 7f2912507700 5 -- op tracker -- seq: 2880, time: 2014-11-03 11:37:49.451050, event: done, op: osd_sub_op(client.8703713.0:32254371 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953139 snapset=0=[]:[] snapc=0=[]) -7594> 2014-11-03 11:37:49.467160 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 158 ==== osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3311293909 0 3511520646) 0x28ea4800 con 0x22fefdc0 -7593> 2014-11-03 11:37:49.467224 7f28f36c2700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467027, event: header_read, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7592> 2014-11-03 11:37:49.467298 7f28f36c2700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467029, event: throttled, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7591> 2014-11-03 11:37:49.467321 7f28f36c2700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467141, event: all_read, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7590> 2014-11-03 11:37:49.467340 7f28f36c2700 5 -- op tracker -- seq: 2881, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7589> 2014-11-03 11:37:49.467464 7f290101d700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467463, event: reached_pg, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7588> 2014-11-03 11:37:49.467523 7f290101d700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467522, event: started, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7587> 2014-11-03 11:37:49.467670 7f290101d700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467669, event: started, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7586> 2014-11-03 11:37:49.467722 7f290101d700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467722, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7585> 2014-11-03 11:37:49.467834 7f2914d0c700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.467834, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7584> 2014-11-03 11:37:49.468105 7f291450b700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.468105, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7583> 2014-11-03 11:37:49.468203 7f2911d06700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.468203, event: commit_sent, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7582> 2014-11-03 11:37:49.468231 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x240d22c0 con 0x22fefdc0 -7581> 2014-11-03 11:37:49.468611 7f2912507700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.468611, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7580> 2014-11-03 11:37:49.468694 7f2912507700 5 -- op tracker -- seq: 2881, time: 2014-11-03 11:37:49.468694, event: done, op: osd_sub_op(client.8703713.0:32254373 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953140 snapset=0=[]:[] snapc=0=[]) -7579> 2014-11-03 11:37:49.471903 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 159 ==== osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1806211782 0 3511520646) 0x2cff3000 con 0x22fefdc0 -7578> 2014-11-03 11:37:49.471934 7f28f36c2700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.471762, event: header_read, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7577> 2014-11-03 11:37:49.471948 7f28f36c2700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.471764, event: throttled, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7576> 2014-11-03 11:37:49.471956 7f28f36c2700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.471894, event: all_read, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7575> 2014-11-03 11:37:49.471998 7f28f36c2700 5 -- op tracker -- seq: 2882, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7574> 2014-11-03 11:37:49.472099 7f28fe818700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.472099, event: reached_pg, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7573> 2014-11-03 11:37:49.472158 7f28fe818700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.472157, event: started, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7572> 2014-11-03 11:37:49.472341 7f28fe818700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.472340, event: started, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7571> 2014-11-03 11:37:49.472413 7f28fe818700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.472413, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7570> 2014-11-03 11:37:49.472519 7f2914d0c700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.472518, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7569> 2014-11-03 11:37:49.491362 7f291450b700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.491361, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7568> 2014-11-03 11:37:49.491404 7f2911d06700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.491403, event: commit_sent, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7567> 2014-11-03 11:37:49.491438 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x240d2840 con 0x22fefdc0 -7566> 2014-11-03 11:37:49.491678 7f2912507700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.491678, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7565> 2014-11-03 11:37:49.491719 7f2912507700 5 -- op tracker -- seq: 2882, time: 2014-11-03 11:37:49.491719, event: done, op: osd_sub_op(client.8703713.0:32254375 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953141 snapset=0=[]:[] snapc=0=[]) -7564> 2014-11-03 11:37:49.503129 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 160 ==== osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2433215539 0 3511520646) 0x2398e400 con 0x22fefdc0 -7563> 2014-11-03 11:37:49.503168 7f28f36c2700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.502994, event: header_read, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7562> 2014-11-03 11:37:49.503203 7f28f36c2700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.502996, event: throttled, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7561> 2014-11-03 11:37:49.503219 7f28f36c2700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.503116, event: all_read, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7560> 2014-11-03 11:37:49.503234 7f28f36c2700 5 -- op tracker -- seq: 2883, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7559> 2014-11-03 11:37:49.503357 7f290101d700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.503357, event: reached_pg, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7558> 2014-11-03 11:37:49.503416 7f290101d700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.503416, event: started, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7557> 2014-11-03 11:37:49.503570 7f290101d700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.503570, event: started, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7556> 2014-11-03 11:37:49.503617 7f290101d700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.503617, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7555> 2014-11-03 11:37:49.503753 7f2914d0c700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.503752, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7554> 2014-11-03 11:37:49.504198 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 161 ==== osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1026327 0 3511520646) 0x96ea000 con 0x22fefdc0 -7553> 2014-11-03 11:37:49.504229 7f28f36c2700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.504109, event: header_read, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7552> 2014-11-03 11:37:49.504243 7f28f36c2700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.504111, event: throttled, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7551> 2014-11-03 11:37:49.504254 7f28f36c2700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.504189, event: all_read, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7550> 2014-11-03 11:37:49.504262 7f28f36c2700 5 -- op tracker -- seq: 2884, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7549> 2014-11-03 11:37:49.504364 7f28fe818700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.504364, event: reached_pg, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7548> 2014-11-03 11:37:49.504415 7f28fe818700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.504415, event: started, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7547> 2014-11-03 11:37:49.504546 7f28fe818700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.504546, event: started, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7546> 2014-11-03 11:37:49.504592 7f28fe818700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.504592, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7545> 2014-11-03 11:37:49.506633 7f291450b700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.506632, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7544> 2014-11-03 11:37:49.506706 7f2911d06700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.506705, event: commit_sent, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7543> 2014-11-03 11:37:49.506710 7f2914d0c700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.506710, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7542> 2014-11-03 11:37:49.506752 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2fcf9000 con 0x22fefdc0 -7541> 2014-11-03 11:37:49.506914 7f2912507700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.506914, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7540> 2014-11-03 11:37:49.506955 7f2912507700 5 -- op tracker -- seq: 2883, time: 2014-11-03 11:37:49.506955, event: done, op: osd_sub_op(client.8703713.0:32254376 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953142 snapset=0=[]:[] snapc=0=[]) -7539> 2014-11-03 11:37:49.507795 7f291450b700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.507795, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7538> 2014-11-03 11:37:49.507891 7f2911d06700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.507891, event: commit_sent, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7537> 2014-11-03 11:37:49.507926 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2fcf9dc0 con 0x22fefdc0 -7536> 2014-11-03 11:37:49.508114 7f2912507700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.508114, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7535> 2014-11-03 11:37:49.508160 7f2912507700 5 -- op tracker -- seq: 2884, time: 2014-11-03 11:37:49.508160, event: done, op: osd_sub_op(client.8703713.0:32254377 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953143 snapset=0=[]:[] snapc=0=[]) -7534> 2014-11-03 11:37:49.538265 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x24139200 con 0x231abb00 -7533> 2014-11-03 11:37:49.545587 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 44 ==== osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (11148021 0 3511520646) 0x24e87400 con 0x230a9b00 -7532> 2014-11-03 11:37:49.545663 7f28f038f700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.545375, event: header_read, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7531> 2014-11-03 11:37:49.545704 7f28f038f700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.545377, event: throttled, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7530> 2014-11-03 11:37:49.545734 7f28f038f700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.545558, event: all_read, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7529> 2014-11-03 11:37:49.545762 7f28f038f700 5 -- op tracker -- seq: 2885, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7528> 2014-11-03 11:37:49.545862 7f290101d700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.545862, event: reached_pg, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7527> 2014-11-03 11:37:49.545912 7f290101d700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.545911, event: started, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7526> 2014-11-03 11:37:49.546059 7f290101d700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.546059, event: started, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7525> 2014-11-03 11:37:49.546122 7f290101d700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.546122, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7524> 2014-11-03 11:37:49.546220 7f2914d0c700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.546220, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7523> 2014-11-03 11:37:49.546449 7f291450b700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.546449, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7522> 2014-11-03 11:37:49.546513 7f2911d06700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.546513, event: commit_sent, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7521> 2014-11-03 11:37:49.546553 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcfa340 con 0x230a9b00 -7520> 2014-11-03 11:37:49.546845 7f2912507700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.546844, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7519> 2014-11-03 11:37:49.546887 7f2912507700 5 -- op tracker -- seq: 2885, time: 2014-11-03 11:37:49.546886, event: done, op: osd_sub_op(client.8236383.0:16630681 6.458 2c5ccc58/rbd_data.7e655d2ae8944a.0000000000004440/head//6 [] v 105950'17850147 snapset=0=[]:[] snapc=0=[]) -7518> 2014-11-03 11:37:49.556667 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.47 192.168.5.4:0/12077 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.555133) v2 ==== 47+0+0 (2721932241 0 0) 0x259f3600 con 0x242926e0 -7517> 2014-11-03 11:37:49.556742 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.555133) v2 -- ?+0 0x3501400 con 0x242926e0 -7516> 2014-11-03 11:37:49.556850 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.47 192.168.5.4:0/12077 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.555133) v2 ==== 47+0+0 (2721932241 0 0) 0x2ba4d400 con 0x243216e0 -7515> 2014-11-03 11:37:49.556908 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.555133) v2 -- ?+0 0x23e31000 con 0x243216e0 -7514> 2014-11-03 11:37:49.567173 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.112 192.168.5.9:0/24552 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.565916) v2 ==== 47+0+0 (2713555238 0 0) 0x6a83400 con 0x24537b00 -7513> 2014-11-03 11:37:49.567189 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.112 192.168.5.9:0/24552 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.565916) v2 ==== 47+0+0 (2713555238 0 0) 0x6a83000 con 0x2484a160 -7512> 2014-11-03 11:37:49.567241 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/24552 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.565916) v2 -- ?+0 0x2ba4d400 con 0x24537b00 -7511> 2014-11-03 11:37:49.567342 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/24552 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.565916) v2 -- ?+0 0x259f3600 con 0x2484a160 -7510> 2014-11-03 11:37:49.577178 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.101 192.168.5.8:0/13676 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.575623) v2 ==== 47+0+0 (3483051445 0 0) 0x23305000 con 0x239f86e0 -7509> 2014-11-03 11:37:49.577173 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.101 192.168.5.8:0/13676 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.575623) v2 ==== 47+0+0 (3483051445 0 0) 0x24dce000 con 0x238b8840 -7508> 2014-11-03 11:37:49.577242 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/13676 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.575623) v2 -- ?+0 0x6a83000 con 0x239f86e0 -7507> 2014-11-03 11:37:49.577304 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/13676 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.575623) v2 -- ?+0 0x6a83400 con 0x238b8840 -7506> 2014-11-03 11:37:49.601609 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.76 192.168.5.6:0/22493 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.601097) v2 ==== 47+0+0 (2751997624 0 0) 0x852f800 con 0x24229dc0 -7505> 2014-11-03 11:37:49.601676 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.601097) v2 -- ?+0 0x24dce000 con 0x24229dc0 -7504> 2014-11-03 11:37:49.601730 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.76 192.168.5.6:0/22493 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.601097) v2 ==== 47+0+0 (2751997624 0 0) 0x8025200 con 0x23b1c160 -7503> 2014-11-03 11:37:49.601831 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.601097) v2 -- ?+0 0x23305000 con 0x23b1c160 -7502> 2014-11-03 11:37:49.606889 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.9 192.168.5.1:0/17062 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.606414) v2 ==== 47+0+0 (4291719363 0 0) 0x30db400 con 0x24292580 -7501> 2014-11-03 11:37:49.606955 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/17062 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.606414) v2 -- ?+0 0x8025200 con 0x24292580 -7500> 2014-11-03 11:37:49.606909 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.9 192.168.5.1:0/17062 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.606414) v2 ==== 47+0+0 (4291719363 0 0) 0x1adc4a00 con 0x24321580 -7499> 2014-11-03 11:37:49.607011 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/17062 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.606414) v2 -- ?+0 0x852f800 con 0x24321580 -7498> 2014-11-03 11:37:49.633878 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 83 ==== osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4925 (650602801 0 2967796223) 0x2448d000 con 0x22fbcb00 -7497> 2014-11-03 11:37:49.633918 7f28f6cf8700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.633626, event: header_read, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7496> 2014-11-03 11:37:49.633939 7f28f6cf8700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.633630, event: throttled, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7495> 2014-11-03 11:37:49.633956 7f28f6cf8700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.633854, event: all_read, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7494> 2014-11-03 11:37:49.633965 7f28f6cf8700 5 -- op tracker -- seq: 2886, time: 0.000000, event: dispatched, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7493> 2014-11-03 11:37:49.634028 7f290201f700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.634028, event: reached_pg, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7492> 2014-11-03 11:37:49.634205 7f290201f700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.634205, event: started, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7491> 2014-11-03 11:37:49.634417 7f290201f700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.634417, event: started, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7490> 2014-11-03 11:37:49.634469 7f290201f700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.634469, event: commit_queued_for_journal_write, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7489> 2014-11-03 11:37:49.634568 7f2914d0c700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.634568, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7488> 2014-11-03 11:37:49.634917 7f291450b700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.634916, event: journaled_completion_queued, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7487> 2014-11-03 11:37:49.634962 7f2911d06700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.634962, event: commit_sent, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7486> 2014-11-03 11:37:49.635024 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2fcf9840 con 0x22fbcb00 -7485> 2014-11-03 11:37:49.635630 7f2912507700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.635629, event: sub_op_applied, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7484> 2014-11-03 11:37:49.635694 7f2912507700 5 -- op tracker -- seq: 2886, time: 2014-11-03 11:37:49.635693, event: done, op: osd_sub_op(client.7871531.0:413975 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114067 snapset=0=[]:[] snapc=0=[]) -7483> 2014-11-03 11:37:49.647508 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 26 ==== MOSDPGPush(6.58e 105950 [PushOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, version: 105945'11622060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (15230627 0 0) 0x24139200 con 0x231abb00 -7482> 2014-11-03 11:37:49.647551 7f28eb945700 5 -- op tracker -- seq: 2887, time: 2014-11-03 11:37:49.630799, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, version: 105945'11622060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7481> 2014-11-03 11:37:49.647575 7f28eb945700 5 -- op tracker -- seq: 2887, time: 2014-11-03 11:37:49.630800, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, version: 105945'11622060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7480> 2014-11-03 11:37:49.647588 7f28eb945700 5 -- op tracker -- seq: 2887, time: 2014-11-03 11:37:49.647494, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, version: 105945'11622060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7479> 2014-11-03 11:37:49.647604 7f28eb945700 5 -- op tracker -- seq: 2887, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, version: 105945'11622060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7478> 2014-11-03 11:37:49.647672 7f28fe818700 5 -- op tracker -- seq: 2887, time: 2014-11-03 11:37:49.647672, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, version: 105945'11622060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7477> 2014-11-03 11:37:49.647897 7f28fe818700 5 -- op tracker -- seq: 2887, time: 2014-11-03 11:37:49.647897, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6, version: 105945'11622060, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11ba158e/rbd_data.3e5d6a2ae8944a.0000000000001835/head//6@105945'11622060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7476> 2014-11-03 11:37:49.702467 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.121 192.168.5.9:0/12205 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.701466) v2 ==== 47+0+0 (2582583835 0 0) 0x23cbe400 con 0x24229b00 -7475> 2014-11-03 11:37:49.702520 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/12205 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.701466) v2 -- ?+0 0x1adc4a00 con 0x24229b00 -7474> 2014-11-03 11:37:49.702503 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.121 192.168.5.9:0/12205 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.701466) v2 ==== 47+0+0 (2582583835 0 0) 0x23cbe200 con 0x24925dc0 -7473> 2014-11-03 11:37:49.702591 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/12205 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.701466) v2 -- ?+0 0x30db400 con 0x24925dc0 -7472> 2014-11-03 11:37:49.723118 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x24139000 con 0x231abb00 -7471> 2014-11-03 11:37:49.744310 7f28c4099700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x258beb80 sd=203 :6802 s=0 pgs=0 cs=0 l=0 c=0x24537dc0).accept sd=203 192.168.5.201:58643/0 -7470> 2014-11-03 11:37:49.745427 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419883 192.168.5.201:0/1028591 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x91aea00 con 0x24537dc0 -7469> 2014-11-03 11:37:49.745978 7f28fb812700 0 osd.21 105950 do_command r=0 -7468> 2014-11-03 11:37:49.746002 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028591 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xb0a5380 con 0x24537dc0 -7467> 2014-11-03 11:37:49.759583 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419883 192.168.5.201:0/1028591 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2a6e8400 con 0x24537dc0 -7466> 2014-11-03 11:37:49.760383 7f28fb812700 0 osd.21 105950 do_command r=0 -7465> 2014-11-03 11:37:49.760401 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028591 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0xa5f6e00 con 0x24537dc0 -7464> 2014-11-03 11:37:49.761781 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 84 ==== osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (1124680636 0 3511520646) 0x27806c00 con 0x22fbcb00 -7463> 2014-11-03 11:37:49.761843 7f28f6cf8700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.761529, event: header_read, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7462> 2014-11-03 11:37:49.761862 7f28f6cf8700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.761532, event: throttled, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7461> 2014-11-03 11:37:49.761874 7f28f6cf8700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.761756, event: all_read, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7460> 2014-11-03 11:37:49.761887 7f28f6cf8700 5 -- op tracker -- seq: 2888, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7459> 2014-11-03 11:37:49.761941 7f28fe017700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.761941, event: reached_pg, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7458> 2014-11-03 11:37:49.761971 7f28fe017700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.761971, event: started, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7457> 2014-11-03 11:37:49.762079 7f28fe017700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.762079, event: started, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7456> 2014-11-03 11:37:49.762143 7f28fe017700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.762143, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7455> 2014-11-03 11:37:49.762211 7f2914d0c700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.762211, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7454> 2014-11-03 11:37:49.762415 7f291450b700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.762415, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7453> 2014-11-03 11:37:49.762459 7f2911d06700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.762459, event: commit_sent, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7452> 2014-11-03 11:37:49.762497 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfce000 con 0x22fbcb00 -7451> 2014-11-03 11:37:49.763201 7f2912507700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.763201, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7450> 2014-11-03 11:37:49.763349 7f2912507700 5 -- op tracker -- seq: 2888, time: 2014-11-03 11:37:49.763349, event: done, op: osd_sub_op(client.7869068.0:21687356 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134360 snapset=0=[]:[] snapc=0=[]) -7449> 2014-11-03 11:37:49.763929 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 85 ==== osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (1839453496 0 3511520646) 0x27072400 con 0x22fbcb00 -7448> 2014-11-03 11:37:49.763979 7f28f6cf8700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.763804, event: header_read, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7447> 2014-11-03 11:37:49.764071 7f28f6cf8700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.763806, event: throttled, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7446> 2014-11-03 11:37:49.764189 7f28f6cf8700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.763908, event: all_read, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7445> 2014-11-03 11:37:49.764224 7f28f6cf8700 5 -- op tracker -- seq: 2889, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7444> 2014-11-03 11:37:49.764420 7f290081c700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.764419, event: reached_pg, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7443> 2014-11-03 11:37:49.764475 7f290081c700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.764475, event: started, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7442> 2014-11-03 11:37:49.764712 7f290081c700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.764712, event: started, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7441> 2014-11-03 11:37:49.764776 7f290081c700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.764776, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7440> 2014-11-03 11:37:49.764904 7f2914d0c700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.764904, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7439> 2014-11-03 11:37:49.766155 7f291450b700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.766155, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7438> 2014-11-03 11:37:49.766169 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028591 pipe(0x258beb80 sd=203 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537dc0).reader couldn't read tag, (0) Success -7437> 2014-11-03 11:37:49.766217 7f2911d06700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.766217, event: commit_sent, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7436> 2014-11-03 11:37:49.766261 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028591 pipe(0x258beb80 sd=203 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537dc0).fault (0) Success -7435> 2014-11-03 11:37:49.766275 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfcf8c0 con 0x22fbcb00 -7434> 2014-11-03 11:37:49.766673 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x24537dc0 session 0x78e6000 -7433> 2014-11-03 11:37:49.766640 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 86 ==== osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (580952662 0 3511520646) 0x26942c00 con 0x22fbcb00 -7432> 2014-11-03 11:37:49.766737 7f28f6cf8700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.766462, event: header_read, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7431> 2014-11-03 11:37:49.766769 7f28f6cf8700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.766464, event: throttled, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7430> 2014-11-03 11:37:49.766782 7f28f6cf8700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.766619, event: all_read, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7429> 2014-11-03 11:37:49.766794 7f28f6cf8700 5 -- op tracker -- seq: 2890, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7428> 2014-11-03 11:37:49.766857 7f28fe017700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.766857, event: reached_pg, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7427> 2014-11-03 11:37:49.766891 7f28fe017700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.766891, event: started, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7426> 2014-11-03 11:37:49.766980 7f28fe017700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.766979, event: started, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7425> 2014-11-03 11:37:49.767020 7f28fe017700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.767020, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7424> 2014-11-03 11:37:49.767192 7f2914d0c700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.767191, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7423> 2014-11-03 11:37:49.767813 7f291450b700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.767812, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7422> 2014-11-03 11:37:49.767794 7f2912507700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.767793, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7421> 2014-11-03 11:37:49.767900 7f2912507700 5 -- op tracker -- seq: 2889, time: 2014-11-03 11:37:49.767900, event: done, op: osd_sub_op(client.7869068.0:21687357 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134361 snapset=0=[]:[] snapc=0=[]) -7420> 2014-11-03 11:37:49.768414 7f2911d06700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.768414, event: commit_sent, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7419> 2014-11-03 11:37:49.768451 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfceb00 con 0x22fbcb00 -7418> 2014-11-03 11:37:49.769174 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 87 ==== osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (2515905453 0 3511520646) 0x26942000 con 0x22fbcb00 -7417> 2014-11-03 11:37:49.769219 7f28f6cf8700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769028, event: header_read, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7416> 2014-11-03 11:37:49.769241 7f28f6cf8700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769030, event: throttled, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7415> 2014-11-03 11:37:49.769257 7f28f6cf8700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769155, event: all_read, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7414> 2014-11-03 11:37:49.769273 7f28f6cf8700 5 -- op tracker -- seq: 2891, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7413> 2014-11-03 11:37:49.769450 7f290081c700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769450, event: reached_pg, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7412> 2014-11-03 11:37:49.769493 7f290081c700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769493, event: started, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7411> 2014-11-03 11:37:49.769698 7f290081c700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769698, event: started, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7410> 2014-11-03 11:37:49.769753 7f290081c700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769753, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7409> 2014-11-03 11:37:49.769853 7f2912507700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.769852, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7408> 2014-11-03 11:37:49.769858 7f2914d0c700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.769858, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7407> 2014-11-03 11:37:49.769927 7f2912507700 5 -- op tracker -- seq: 2890, time: 2014-11-03 11:37:49.769927, event: done, op: osd_sub_op(client.7869068.0:21687358 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134362 snapset=0=[]:[] snapc=0=[]) -7406> 2014-11-03 11:37:49.770151 7f291450b700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.770150, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7405> 2014-11-03 11:37:49.770219 7f2911d06700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.770219, event: commit_sent, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7404> 2014-11-03 11:37:49.770283 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfcf340 con 0x22fbcb00 -7403> 2014-11-03 11:37:49.770878 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 88 ==== osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (537532960 0 3511520646) 0x2cff3000 con 0x22fbcb00 -7402> 2014-11-03 11:37:49.770930 7f28f6cf8700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.770729, event: header_read, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7401> 2014-11-03 11:37:49.770957 7f28f6cf8700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.770732, event: throttled, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7400> 2014-11-03 11:37:49.770977 7f28f6cf8700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.770874, event: all_read, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7399> 2014-11-03 11:37:49.770995 7f28f6cf8700 5 -- op tracker -- seq: 2892, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7398> 2014-11-03 11:37:49.771080 7f28fe017700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.771080, event: reached_pg, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7397> 2014-11-03 11:37:49.771117 7f28fe017700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.771117, event: started, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7396> 2014-11-03 11:37:49.771218 7f28fe017700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.771218, event: started, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7395> 2014-11-03 11:37:49.771252 7f28fe017700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.771252, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7394> 2014-11-03 11:37:49.771339 7f2914d0c700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.771339, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7393> 2014-11-03 11:37:49.771350 7f2912507700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.771350, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7392> 2014-11-03 11:37:49.771420 7f2912507700 5 -- op tracker -- seq: 2891, time: 2014-11-03 11:37:49.771420, event: done, op: osd_sub_op(client.7869068.0:21687359 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134363 snapset=0=[]:[] snapc=0=[]) -7391> 2014-11-03 11:37:49.771610 7f291450b700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.771609, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7390> 2014-11-03 11:37:49.771719 7f2911d06700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.771719, event: commit_sent, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7389> 2014-11-03 11:37:49.771777 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfcedc0 con 0x22fbcb00 -7388> 2014-11-03 11:37:49.772086 7f2912507700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.772085, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7387> 2014-11-03 11:37:49.772146 7f2912507700 5 -- op tracker -- seq: 2892, time: 2014-11-03 11:37:49.772146, event: done, op: osd_sub_op(client.7869068.0:21687360 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134364 snapset=0=[]:[] snapc=0=[]) -7386> 2014-11-03 11:37:49.772443 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 89 ==== osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (1823187006 0 3511520646) 0x96ea000 con 0x22fbcb00 -7385> 2014-11-03 11:37:49.772489 7f28f6cf8700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.772332, event: header_read, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7384> 2014-11-03 11:37:49.772519 7f28f6cf8700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.772333, event: throttled, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7383> 2014-11-03 11:37:49.772538 7f28f6cf8700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.772429, event: all_read, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7382> 2014-11-03 11:37:49.772557 7f28f6cf8700 5 -- op tracker -- seq: 2893, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7381> 2014-11-03 11:37:49.772636 7f290081c700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.772635, event: reached_pg, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7380> 2014-11-03 11:37:49.772687 7f290081c700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.772686, event: started, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7379> 2014-11-03 11:37:49.772856 7f290081c700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.772856, event: started, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7378> 2014-11-03 11:37:49.772911 7f290081c700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.772911, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7377> 2014-11-03 11:37:49.773002 7f2914d0c700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.773001, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7376> 2014-11-03 11:37:49.773231 7f291450b700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.773231, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7375> 2014-11-03 11:37:49.773289 7f2911d06700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.773289, event: commit_sent, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7374> 2014-11-03 11:37:49.773325 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfce580 con 0x22fbcb00 -7373> 2014-11-03 11:37:49.773615 7f2912507700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.773615, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7372> 2014-11-03 11:37:49.773687 7f2912507700 5 -- op tracker -- seq: 2893, time: 2014-11-03 11:37:49.773686, event: done, op: osd_sub_op(client.7869068.0:21687361 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134365 snapset=0=[]:[] snapc=0=[]) -7371> 2014-11-03 11:37:49.812440 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 27 ==== MOSDPGPush(6.58e 105950 [PushOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, version: 105945'11622074, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195446+0+0 (1296912836 0 0) 0x24139000 con 0x231abb00 -7370> 2014-11-03 11:37:49.812487 7f28eb945700 5 -- op tracker -- seq: 2894, time: 2014-11-03 11:37:49.795956, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, version: 105945'11622074, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7369> 2014-11-03 11:37:49.812513 7f28eb945700 5 -- op tracker -- seq: 2894, time: 2014-11-03 11:37:49.795959, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, version: 105945'11622074, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7368> 2014-11-03 11:37:49.812526 7f28eb945700 5 -- op tracker -- seq: 2894, time: 2014-11-03 11:37:49.812425, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, version: 105945'11622074, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7367> 2014-11-03 11:37:49.812551 7f28eb945700 5 -- op tracker -- seq: 2894, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, version: 105945'11622074, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7366> 2014-11-03 11:37:49.812661 7f290101d700 5 -- op tracker -- seq: 2894, time: 2014-11-03 11:37:49.812661, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, version: 105945'11622074, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7365> 2014-11-03 11:37:49.813066 7f290101d700 5 -- op tracker -- seq: 2894, time: 2014-11-03 11:37:49.813065, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6, version: 105945'11622074, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6b862d8e/rbd_data.3c2c4e2ae8944a.0000000000000622/head//6@105945'11622074, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7364> 2014-11-03 11:37:49.833921 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.13 192.168.5.1:0/22281 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.833130) v2 ==== 47+0+0 (2034654833 0 0) 0x2342e800 con 0x2445c420 -7363> 2014-11-03 11:37:49.833966 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/22281 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.833130) v2 -- ?+0 0x23cbe200 con 0x2445c420 -7362> 2014-11-03 11:37:49.833966 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.13 192.168.5.1:0/22281 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.833130) v2 ==== 47+0+0 (2034654833 0 0) 0x2a7c4400 con 0x23ea4580 -7361> 2014-11-03 11:37:49.834014 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/22281 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.833130) v2 -- ?+0 0x23cbe400 con 0x23ea4580 -7360> 2014-11-03 11:37:49.839460 7f28c4099700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x3110ddc0 sd=178 :6802 s=0 pgs=0 cs=0 l=0 c=0x24537000).accept sd=178 192.168.5.201:58646/0 -7359> 2014-11-03 11:37:49.840313 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450808 192.168.5.201:0/1028680 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "24.7d7"}) v1 ==== 80+0+0 (3756934197 0 0) 0x252d7800 con 0x24537000 -7358> 2014-11-03 11:37:49.840804 7f28fb812700 0 osd.21 105950 do_command r=0 -7357> 2014-11-03 11:37:49.840829 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028680 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f1c0 con 0x24537000 -7356> 2014-11-03 11:37:49.850690 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450808 192.168.5.201:0/1028680 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "24.7d7"}) v1 ==== 74+0+0 (774687807 0 0) 0x25e25800 con 0x24537000 -7355> 2014-11-03 11:37:49.851379 7f28fb812700 0 osd.21 105950 do_command r=0 -7354> 2014-11-03 11:37:49.851394 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028680 -- command_reply(tid 2: 0 ) v1 -- ?+13413 0x17c608c0 con 0x24537000 -7353> 2014-11-03 11:37:49.856412 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028680 pipe(0x3110ddc0 sd=178 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537000).reader couldn't read tag, (0) Success -7352> 2014-11-03 11:37:49.856452 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028680 pipe(0x3110ddc0 sd=178 :6802 s=2 pgs=2 cs=1 l=1 c=0x24537000).fault (0) Success -7351> 2014-11-03 11:37:49.856516 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x24537000 session 0x78e6000 -7350> 2014-11-03 11:37:49.862055 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 90 ==== osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3893389479 0 3511520646) 0x27806c00 con 0x22fbcb00 -7349> 2014-11-03 11:37:49.862167 7f28f6cf8700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.861777, event: header_read, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7348> 2014-11-03 11:37:49.862203 7f28f6cf8700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.861781, event: throttled, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7347> 2014-11-03 11:37:49.862238 7f28f6cf8700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.862031, event: all_read, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7346> 2014-11-03 11:37:49.862266 7f28f6cf8700 5 -- op tracker -- seq: 2895, time: 0.000000, event: dispatched, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7345> 2014-11-03 11:37:49.862362 7f28fe017700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.862362, event: reached_pg, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7344> 2014-11-03 11:37:49.862398 7f28fe017700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.862398, event: started, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7343> 2014-11-03 11:37:49.862506 7f28fe017700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.862506, event: started, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7342> 2014-11-03 11:37:49.862538 7f28fe017700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.862538, event: commit_queued_for_journal_write, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7341> 2014-11-03 11:37:49.862631 7f2914d0c700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.862630, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7340> 2014-11-03 11:37:49.862938 7f291450b700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.862938, event: journaled_completion_queued, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7339> 2014-11-03 11:37:49.863050 7f2911d06700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.863050, event: commit_sent, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7338> 2014-11-03 11:37:49.863096 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfce840 con 0x22fbcb00 -7337> 2014-11-03 11:37:49.863404 7f2912507700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.863404, event: sub_op_applied, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7336> 2014-11-03 11:37:49.863476 7f2912507700 5 -- op tracker -- seq: 2895, time: 2014-11-03 11:37:49.863475, event: done, op: osd_sub_op(client.9561337.0:516030 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134366 snapset=0=[]:[] snapc=0=[]) -7335> 2014-11-03 11:37:49.863876 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 103 ==== osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (732823528 0 3511520646) 0x2e169400 con 0x22fe0c60 -7334> 2014-11-03 11:37:49.863924 7f28f4eda700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.863743, event: header_read, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7333> 2014-11-03 11:37:49.863956 7f28f4eda700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.863745, event: throttled, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7332> 2014-11-03 11:37:49.863983 7f28f4eda700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.863860, event: all_read, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7331> 2014-11-03 11:37:49.864001 7f28f4eda700 5 -- op tracker -- seq: 2896, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7330> 2014-11-03 11:37:49.864064 7f290081c700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.864064, event: reached_pg, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7329> 2014-11-03 11:37:49.864092 7f290081c700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.864092, event: started, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7328> 2014-11-03 11:37:49.864216 7f290081c700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.864215, event: started, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7327> 2014-11-03 11:37:49.864260 7f290081c700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.864260, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7326> 2014-11-03 11:37:49.864407 7f2914d0c700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.864407, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7325> 2014-11-03 11:37:49.864714 7f291450b700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.864713, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7324> 2014-11-03 11:37:49.864809 7f2911d06700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.864808, event: commit_sent, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7323> 2014-11-03 11:37:49.864861 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28229340 con 0x22fe0c60 -7322> 2014-11-03 11:37:49.864937 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 104 ==== osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1048920872 0 3511520646) 0x2c338400 con 0x22fe0c60 -7321> 2014-11-03 11:37:49.865020 7f28f4eda700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.864792, event: header_read, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7320> 2014-11-03 11:37:49.865051 7f28f4eda700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.864793, event: throttled, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7319> 2014-11-03 11:37:49.865088 7f28f4eda700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.864912, event: all_read, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7318> 2014-11-03 11:37:49.865118 7f28f4eda700 5 -- op tracker -- seq: 2897, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7317> 2014-11-03 11:37:49.865214 7f28fe017700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.865213, event: reached_pg, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7316> 2014-11-03 11:37:49.865257 7f28fe017700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.865257, event: started, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7315> 2014-11-03 11:37:49.865406 7f28fe017700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.865406, event: started, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7314> 2014-11-03 11:37:49.865459 7f28fe017700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.865459, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7313> 2014-11-03 11:37:49.865578 7f2912507700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.865578, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7312> 2014-11-03 11:37:49.865580 7f2914d0c700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.865580, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7311> 2014-11-03 11:37:49.865642 7f2912507700 5 -- op tracker -- seq: 2896, time: 2014-11-03 11:37:49.865642, event: done, op: osd_sub_op(client.7867900.0:14923924 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219452 snapset=0=[]:[] snapc=0=[]) -7310> 2014-11-03 11:37:49.865833 7f291450b700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.865833, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7309> 2014-11-03 11:37:49.865841 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 105 ==== osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3402307307 0 3511520646) 0x2c336000 con 0x22fe0c60 -7308> 2014-11-03 11:37:49.865878 7f28f4eda700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.865712, event: header_read, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7307> 2014-11-03 11:37:49.865898 7f28f4eda700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.865714, event: throttled, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7306> 2014-11-03 11:37:49.865897 7f2911d06700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.865897, event: commit_sent, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7305> 2014-11-03 11:37:49.865950 7f28f4eda700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.865820, event: all_read, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7304> 2014-11-03 11:37:49.865976 7f28f4eda700 5 -- op tracker -- seq: 2898, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7303> 2014-11-03 11:37:49.865982 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28228580 con 0x22fe0c60 -7302> 2014-11-03 11:37:49.866138 7f290081c700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.866138, event: reached_pg, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7301> 2014-11-03 11:37:49.866187 7f290081c700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.866187, event: started, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7300> 2014-11-03 11:37:49.866316 7f290081c700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.866316, event: started, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7299> 2014-11-03 11:37:49.866369 7f290081c700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.866368, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7298> 2014-11-03 11:37:49.866444 7f2912507700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.866444, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7297> 2014-11-03 11:37:49.866467 7f2914d0c700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.866467, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7296> 2014-11-03 11:37:49.866500 7f2912507700 5 -- op tracker -- seq: 2897, time: 2014-11-03 11:37:49.866500, event: done, op: osd_sub_op(client.7867900.0:14923925 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219453 snapset=0=[]:[] snapc=0=[]) -7295> 2014-11-03 11:37:49.866753 7f291450b700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.866753, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7294> 2014-11-03 11:37:49.866828 7f2911d06700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.866827, event: commit_sent, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7293> 2014-11-03 11:37:49.866883 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28228dc0 con 0x22fe0c60 -7292> 2014-11-03 11:37:49.867113 7f2912507700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.867113, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7291> 2014-11-03 11:37:49.867162 7f2912507700 5 -- op tracker -- seq: 2898, time: 2014-11-03 11:37:49.867162, event: done, op: osd_sub_op(client.7867900.0:14923926 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219454 snapset=0=[]:[] snapc=0=[]) -7290> 2014-11-03 11:37:49.872786 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x30dba00 con 0x22fbc420 -7289> 2014-11-03 11:37:49.873765 7f28c4099700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x248b6840 sd=178 :6802 s=0 pgs=0 cs=0 l=0 c=0xa26c580).accept sd=178 192.168.5.201:58648/0 -7288> 2014-11-03 11:37:49.874633 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450814 192.168.5.201:0/1028696 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.58e"}) v1 ==== 79+0+0 (3567090102 0 0) 0x26fb1e00 con 0xa26c580 -7287> 2014-11-03 11:37:49.875106 7f28fb812700 0 osd.21 105950 do_command r=0 -7286> 2014-11-03 11:37:49.875126 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028696 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f1c0 con 0xa26c580 -7285> 2014-11-03 11:37:49.876558 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.57 192.168.5.5:0/16411 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.875356) v2 ==== 47+0+0 (2124366900 0 0) 0x896dc00 con 0x2469e420 -7284> 2014-11-03 11:37:49.876614 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.875356) v2 -- ?+0 0x2a7c4400 con 0x2469e420 -7283> 2014-11-03 11:37:49.876553 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.57 192.168.5.5:0/16411 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.875356) v2 ==== 47+0+0 (2124366900 0 0) 0x896da00 con 0x242c0160 -7282> 2014-11-03 11:37:49.876731 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.875356) v2 -- ?+0 0x2342e800 con 0x242c0160 -7281> 2014-11-03 11:37:49.885272 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450814 192.168.5.201:0/1028696 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.58e"}) v1 ==== 73+0+0 (125580214 0 0) 0x23d2b800 con 0xa26c580 -7280> 2014-11-03 11:37:49.886366 7f28fb812700 0 osd.21 105950 do_command r=0 -7279> 2014-11-03 11:37:49.886383 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1028696 -- command_reply(tid 2: 0 ) v1 -- ?+16766 0x968f380 con 0xa26c580 -7278> 2014-11-03 11:37:49.891944 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028696 pipe(0x248b6840 sd=178 :6802 s=2 pgs=2 cs=1 l=1 c=0xa26c580).reader couldn't read tag, (0) Success -7277> 2014-11-03 11:37:49.891982 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1028696 pipe(0x248b6840 sd=178 :6802 s=2 pgs=2 cs=1 l=1 c=0xa26c580).fault (0) Success -7276> 2014-11-03 11:37:49.892066 7f290c834700 1 osd.21 105950 ms_handle_reset con 0xa26c580 session 0x78e6000 -7275> 2014-11-03 11:37:49.898518 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.110 192.168.5.8:0/29518 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.896408) v2 ==== 47+0+0 (1288760804 0 0) 0x23510400 con 0x238b8420 -7274> 2014-11-03 11:37:49.898558 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/29518 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.896408) v2 -- ?+0 0x896dc00 con 0x238b8420 -7273> 2014-11-03 11:37:49.898529 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.110 192.168.5.8:0/29518 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.896408) v2 ==== 47+0+0 (1288760804 0 0) 0x2353be00 con 0x239f82c0 -7272> 2014-11-03 11:37:49.898637 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/29518 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.896408) v2 -- ?+0 0x896da00 con 0x239f82c0 -7271> 2014-11-03 11:37:49.925846 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.7 192.168.5.1:0/3511 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.925426) v2 ==== 47+0+0 (1694645734 0 0) 0x234d5a00 con 0x24925000 -7270> 2014-11-03 11:37:49.925942 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.925426) v2 -- ?+0 0x2353be00 con 0x24925000 -7269> 2014-11-03 11:37:49.925938 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.7 192.168.5.1:0/3511 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.925426) v2 ==== 47+0+0 (1694645734 0 0) 0x244ab200 con 0x23ea4c60 -7268> 2014-11-03 11:37:49.926000 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.925426) v2 -- ?+0 0x23510400 con 0x23ea4c60 -7267> 2014-11-03 11:37:49.939904 7f28f4ad6700 1 -- 192.168.5.2:6803/7606 <== osd.89 192.168.5.7:6812/24271 43 ==== osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1202+0+0 (2770770704 0 0) 0xd55e800 con 0x22fe06e0 -7266> 2014-11-03 11:37:49.939970 7f28f4ad6700 5 -- op tracker -- seq: 2899, time: 2014-11-03 11:37:49.939699, event: header_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7265> 2014-11-03 11:37:49.940002 7f28f4ad6700 5 -- op tracker -- seq: 2899, time: 2014-11-03 11:37:49.939704, event: throttled, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7264> 2014-11-03 11:37:49.940049 7f28f4ad6700 5 -- op tracker -- seq: 2899, time: 2014-11-03 11:37:49.939880, event: all_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7263> 2014-11-03 11:37:49.940077 7f28f4ad6700 5 -- op tracker -- seq: 2899, time: 0.000000, event: dispatched, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7262> 2014-11-03 11:37:49.940194 7f290001b700 5 -- op tracker -- seq: 2899, time: 2014-11-03 11:37:49.940194, event: reached_pg, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7261> 2014-11-03 11:37:49.940294 7f290001b700 5 -- op tracker -- seq: 2899, time: 2014-11-03 11:37:49.940294, event: started, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7260> 2014-11-03 11:37:49.940351 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6812/24271 -- osd_sub_op_reply(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x2fcac000 con 0x22fe06e0 -7259> 2014-11-03 11:37:49.940382 7f290001b700 5 -- op tracker -- seq: 2899, time: 2014-11-03 11:37:49.940382, event: done, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7258> 2014-11-03 11:37:49.941545 7f28f4ad6700 1 -- 192.168.5.2:6803/7606 <== osd.89 192.168.5.7:6812/24271 44 ==== osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1202+0+0 (2566622843 0 0) 0xd55dc00 con 0x22fe06e0 -7257> 2014-11-03 11:37:49.941597 7f28f4ad6700 5 -- op tracker -- seq: 2900, time: 2014-11-03 11:37:49.941348, event: header_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7256> 2014-11-03 11:37:49.941625 7f28f4ad6700 5 -- op tracker -- seq: 2900, time: 2014-11-03 11:37:49.941350, event: throttled, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7255> 2014-11-03 11:37:49.941654 7f28f4ad6700 5 -- op tracker -- seq: 2900, time: 2014-11-03 11:37:49.941529, event: all_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7254> 2014-11-03 11:37:49.941682 7f28f4ad6700 5 -- op tracker -- seq: 2900, time: 0.000000, event: dispatched, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7253> 2014-11-03 11:37:49.941785 7f28fd816700 5 -- op tracker -- seq: 2900, time: 2014-11-03 11:37:49.941785, event: reached_pg, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7252> 2014-11-03 11:37:49.941820 7f28fd816700 5 -- op tracker -- seq: 2900, time: 2014-11-03 11:37:49.941820, event: started, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7251> 2014-11-03 11:37:49.941836 7f28fd816700 5 -- op tracker -- seq: 2900, time: 2014-11-03 11:37:49.941836, event: done, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -7250> 2014-11-03 11:37:49.944088 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.45 192.168.5.4:0/1781 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.942258) v2 ==== 47+0+0 (1321571057 0 0) 0x1106c400 con 0x2445c160 -7249> 2014-11-03 11:37:49.944161 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.942258) v2 -- ?+0 0x234d5a00 con 0x2445c160 -7248> 2014-11-03 11:37:49.944535 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.45 192.168.5.4:0/1781 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.942258) v2 ==== 47+0+0 (1321571057 0 0) 0xde23600 con 0x23ea42c0 -7247> 2014-11-03 11:37:49.944597 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.942258) v2 -- ?+0 0x244ab200 con 0x23ea42c0 -7246> 2014-11-03 11:37:49.984175 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 86 ==== MOSDPGPush(6.58e 105950 [PushOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, version: 105945'11622197, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1243321514 0 0) 0x30dba00 con 0x22fbc420 -7245> 2014-11-03 11:37:49.984225 7f28f72fe700 5 -- op tracker -- seq: 2901, time: 2014-11-03 11:37:49.972555, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, version: 105945'11622197, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7244> 2014-11-03 11:37:49.984251 7f28f72fe700 5 -- op tracker -- seq: 2901, time: 2014-11-03 11:37:49.972558, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, version: 105945'11622197, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7243> 2014-11-03 11:37:49.984268 7f28f72fe700 5 -- op tracker -- seq: 2901, time: 2014-11-03 11:37:49.984159, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, version: 105945'11622197, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7242> 2014-11-03 11:37:49.984292 7f28f72fe700 5 -- op tracker -- seq: 2901, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, version: 105945'11622197, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7241> 2014-11-03 11:37:49.984392 7f28fe818700 5 -- op tracker -- seq: 2901, time: 2014-11-03 11:37:49.984392, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, version: 105945'11622197, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7240> 2014-11-03 11:37:49.984659 7f28fe818700 5 -- op tracker -- seq: 2901, time: 2014-11-03 11:37:49.984659, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6, version: 105945'11622197, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(86af4d8e/rbd_data.89cd895a911542.000000000000a280/head//6@105945'11622197, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7239> 2014-11-03 11:37:49.997839 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 91 ==== osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1788801835 0 3511520646) 0x2d47bc00 con 0x2308b840 -7238> 2014-11-03 11:37:49.997902 7f28ef985700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.997680, event: header_read, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7237> 2014-11-03 11:37:49.997952 7f28ef985700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.997683, event: throttled, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7236> 2014-11-03 11:37:49.997978 7f28ef985700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.997818, event: all_read, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7235> 2014-11-03 11:37:49.998002 7f28ef985700 5 -- op tracker -- seq: 2902, time: 0.000000, event: dispatched, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7234> 2014-11-03 11:37:49.998095 7f28ff81a700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.998095, event: reached_pg, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7233> 2014-11-03 11:37:49.998163 7f28ff81a700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.998163, event: started, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7232> 2014-11-03 11:37:49.998331 7f28ff81a700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.998331, event: started, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7231> 2014-11-03 11:37:49.998378 7f28ff81a700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.998378, event: commit_queued_for_journal_write, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7230> 2014-11-03 11:37:49.998466 7f2914d0c700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.998466, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7229> 2014-11-03 11:37:49.998723 7f291450b700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.998723, event: journaled_completion_queued, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7228> 2014-11-03 11:37:49.998792 7f2911d06700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.998792, event: commit_sent, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7227> 2014-11-03 11:37:49.998838 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] ondisk, result = 0) v2 -- ?+0 0x246ba8c0 con 0x2308b840 -7226> 2014-11-03 11:37:49.998890 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x229e7c00 con 0x231abb00 -7225> 2014-11-03 11:37:49.999246 7f2912507700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.999246, event: sub_op_applied, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7224> 2014-11-03 11:37:49.999297 7f2912507700 5 -- op tracker -- seq: 2902, time: 2014-11-03 11:37:49.999297, event: done, op: osd_sub_op(client.8421911.0:15704377 6.690 4ebc3e90/rbd_data.805e322ae8944a.0000000000002028/head//6 [] v 105950'28900010 snapset=0=[]:[] snapc=0=[]) -7223> 2014-11-03 11:37:49.999788 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.90 192.168.5.7:0/27918 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.998304) v2 ==== 47+0+0 (1267660906 0 0) 0x22fad600 con 0x2445c6e0 -7222> 2014-11-03 11:37:49.999844 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/27918 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.998304) v2 -- ?+0 0x1106c400 con 0x2445c6e0 -7221> 2014-11-03 11:37:50.000041 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.90 192.168.5.7:0/27918 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:49.998304) v2 ==== 47+0+0 (1267660906 0 0) 0x23ba6400 con 0x23ea4840 -7220> 2014-11-03 11:37:50.000094 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/27918 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:49.998304) v2 -- ?+0 0xde23600 con 0x23ea4840 -7219> 2014-11-03 11:37:50.008161 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.60 192.168.5.5:0/5983 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.006278) v2 ==== 47+0+0 (3724806464 0 0) 0x2796c000 con 0x2445c9a0 -7218> 2014-11-03 11:37:50.008212 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.006278) v2 -- ?+0 0x22fad600 con 0x2445c9a0 -7217> 2014-11-03 11:37:50.008204 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.60 192.168.5.5:0/5983 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.006278) v2 ==== 47+0+0 (3724806464 0 0) 0x2a3f8400 con 0x23ea4b00 -7216> 2014-11-03 11:37:50.008395 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.006278) v2 -- ?+0 0x23ba6400 con 0x23ea4b00 -7215> 2014-11-03 11:37:50.014396 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 106 ==== osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4224896969 0 3511520646) 0x25ed5800 con 0x22fe0c60 -7214> 2014-11-03 11:37:50.014431 7f28f4eda700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014298, event: header_read, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7213> 2014-11-03 11:37:50.014446 7f28f4eda700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014300, event: throttled, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7212> 2014-11-03 11:37:50.014456 7f28f4eda700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014386, event: all_read, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7211> 2014-11-03 11:37:50.014464 7f28f4eda700 5 -- op tracker -- seq: 2903, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7210> 2014-11-03 11:37:50.014564 7f28fe017700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014564, event: reached_pg, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7209> 2014-11-03 11:37:50.014606 7f28fe017700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014605, event: started, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7208> 2014-11-03 11:37:50.014690 7f28fe017700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014690, event: started, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7207> 2014-11-03 11:37:50.014719 7f28fe017700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014719, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7206> 2014-11-03 11:37:50.014841 7f2914d0c700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.014840, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7205> 2014-11-03 11:37:50.015207 7f291450b700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.015207, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7204> 2014-11-03 11:37:50.015316 7f2911d06700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.015316, event: commit_sent, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7203> 2014-11-03 11:37:50.015375 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2494b000 con 0x22fe0c60 -7202> 2014-11-03 11:37:50.015672 7f2912507700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.015671, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7201> 2014-11-03 11:37:50.015727 7f2912507700 5 -- op tracker -- seq: 2903, time: 2014-11-03 11:37:50.015727, event: done, op: osd_sub_op(client.7873619.0:18867178 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219455 snapset=0=[]:[] snapc=0=[]) -7200> 2014-11-03 11:37:50.016348 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 107 ==== osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (499603064 0 3511520646) 0x24e85000 con 0x22fe0c60 -7199> 2014-11-03 11:37:50.016385 7f28f4eda700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016214, event: header_read, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7198> 2014-11-03 11:37:50.016399 7f28f4eda700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016216, event: throttled, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7197> 2014-11-03 11:37:50.016485 7f28f4eda700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016337, event: all_read, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7196> 2014-11-03 11:37:50.016501 7f28f4eda700 5 -- op tracker -- seq: 2904, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7195> 2014-11-03 11:37:50.016616 7f290081c700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016616, event: reached_pg, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7194> 2014-11-03 11:37:50.016668 7f290081c700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016668, event: started, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7193> 2014-11-03 11:37:50.016797 7f290081c700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016797, event: started, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7192> 2014-11-03 11:37:50.016840 7f290081c700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016840, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7191> 2014-11-03 11:37:50.016963 7f2914d0c700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.016962, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7190> 2014-11-03 11:37:50.017335 7f291450b700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.017335, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7189> 2014-11-03 11:37:50.017422 7f2911d06700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.017421, event: commit_sent, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7188> 2014-11-03 11:37:50.017485 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eb080 con 0x22fe0c60 -7187> 2014-11-03 11:37:50.017826 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 93 ==== osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (401389469 0 3511520646) 0xded2400 con 0x22fbc580 -7186> 2014-11-03 11:37:50.017862 7f28f70fc700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.017725, event: header_read, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7185> 2014-11-03 11:37:50.017877 7f28f70fc700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.017727, event: throttled, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7184> 2014-11-03 11:37:50.017886 7f28f70fc700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.017802, event: all_read, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7183> 2014-11-03 11:37:50.017894 7f28f70fc700 5 -- op tracker -- seq: 2905, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7182> 2014-11-03 11:37:50.017909 7f2912507700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.017909, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7181> 2014-11-03 11:37:50.017949 7f2912507700 5 -- op tracker -- seq: 2904, time: 2014-11-03 11:37:50.017949, event: done, op: osd_sub_op(client.7873619.0:18867179 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219456 snapset=0=[]:[] snapc=0=[]) -7180> 2014-11-03 11:37:50.017974 7f290201f700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.017974, event: reached_pg, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7179> 2014-11-03 11:37:50.017996 7f290201f700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.017996, event: started, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7178> 2014-11-03 11:37:50.018082 7f290201f700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.018082, event: started, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7177> 2014-11-03 11:37:50.018127 7f290201f700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.018127, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7176> 2014-11-03 11:37:50.018282 7f2914d0c700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.018281, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7175> 2014-11-03 11:37:50.018622 7f291450b700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.018622, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7174> 2014-11-03 11:37:50.018783 7f2911d06700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.018783, event: commit_sent, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7173> 2014-11-03 11:37:50.018843 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eadc0 con 0x22fbc580 -7172> 2014-11-03 11:37:50.019073 7f2912507700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.019072, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7171> 2014-11-03 11:37:50.019125 7f2912507700 5 -- op tracker -- seq: 2905, time: 2014-11-03 11:37:50.019125, event: done, op: osd_sub_op(client.7873619.0:18867212 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228183 snapset=0=[]:[] snapc=0=[]) -7170> 2014-11-03 11:37:50.019612 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 108 ==== osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2949192287 0 3511520646) 0x96eb800 con 0x22fe0c60 -7169> 2014-11-03 11:37:50.019684 7f28f4eda700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.019440, event: header_read, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7168> 2014-11-03 11:37:50.019722 7f28f4eda700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.019441, event: throttled, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7167> 2014-11-03 11:37:50.019755 7f28f4eda700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.019592, event: all_read, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7166> 2014-11-03 11:37:50.019805 7f28f4eda700 5 -- op tracker -- seq: 2906, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7165> 2014-11-03 11:37:50.019909 7f28fe017700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.019909, event: reached_pg, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7164> 2014-11-03 11:37:50.019974 7f28fe017700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.019973, event: started, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7163> 2014-11-03 11:37:50.020174 7f28fe017700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.020173, event: started, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7162> 2014-11-03 11:37:50.020237 7f28fe017700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.020237, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7161> 2014-11-03 11:37:50.020331 7f2914d0c700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.020330, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7160> 2014-11-03 11:37:50.020550 7f291450b700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.020550, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7159> 2014-11-03 11:37:50.020613 7f2911d06700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.020612, event: commit_sent, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7158> 2014-11-03 11:37:50.020680 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcadb80 con 0x22fe0c60 -7157> 2014-11-03 11:37:50.021079 7f2912507700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.021079, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7156> 2014-11-03 11:37:50.021122 7f2912507700 5 -- op tracker -- seq: 2906, time: 2014-11-03 11:37:50.021121, event: done, op: osd_sub_op(client.7873619.0:18867180 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219457 snapset=0=[]:[] snapc=0=[]) -7155> 2014-11-03 11:37:50.021499 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 40 ==== osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (337465169 0 3511520646) 0x2fe0c000 con 0x230c2c60 -7154> 2014-11-03 11:37:50.021540 7f28ee16d700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.021386, event: header_read, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7153> 2014-11-03 11:37:50.021564 7f28ee16d700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.021388, event: throttled, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7152> 2014-11-03 11:37:50.021583 7f28ee16d700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.021485, event: all_read, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7151> 2014-11-03 11:37:50.021603 7f28ee16d700 5 -- op tracker -- seq: 2907, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7150> 2014-11-03 11:37:50.021675 7f290001b700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.021674, event: reached_pg, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7149> 2014-11-03 11:37:50.021703 7f290001b700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.021703, event: started, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7148> 2014-11-03 11:37:50.021870 7f290001b700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.021870, event: started, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7147> 2014-11-03 11:37:50.021905 7f290001b700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.021905, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7146> 2014-11-03 11:37:50.022011 7f2914d0c700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.022011, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7145> 2014-11-03 11:37:50.022205 7f291450b700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.022204, event: journaled_completion_queued, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7144> 2014-11-03 11:37:50.022396 7f2911d06700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.022396, event: commit_sent, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7143> 2014-11-03 11:37:50.022460 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcad8c0 con 0x230c2c60 -7142> 2014-11-03 11:37:50.022905 7f2912507700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.022905, event: sub_op_applied, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7141> 2014-11-03 11:37:50.022985 7f2912d08700 5 -- op tracker -- seq: 2907, time: 2014-11-03 11:37:50.022985, event: done, op: osd_sub_op(client.7869107.0:21845085 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132373 snapset=0=[]:[] snapc=0=[]) -7140> 2014-11-03 11:37:50.023499 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 41 ==== osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2225491480 0 3511520646) 0x2cf7fc00 con 0x230c2c60 -7139> 2014-11-03 11:37:50.023574 7f28ee16d700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.023329, event: header_read, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7138> 2014-11-03 11:37:50.023598 7f28ee16d700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.023331, event: throttled, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7137> 2014-11-03 11:37:50.023622 7f28ee16d700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.023475, event: all_read, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7136> 2014-11-03 11:37:50.023641 7f28ee16d700 5 -- op tracker -- seq: 2908, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7135> 2014-11-03 11:37:50.023754 7f28fd816700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.023754, event: reached_pg, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7134> 2014-11-03 11:37:50.023813 7f28fd816700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.023813, event: started, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7133> 2014-11-03 11:37:50.023982 7f28fd816700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.023982, event: started, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7132> 2014-11-03 11:37:50.024046 7f28fd816700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.024046, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7131> 2014-11-03 11:37:50.024154 7f2914d0c700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.024154, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7130> 2014-11-03 11:37:50.024481 7f291450b700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.024480, event: journaled_completion_queued, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7129> 2014-11-03 11:37:50.024580 7f2911d06700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.024580, event: commit_sent, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7128> 2014-11-03 11:37:50.024629 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917e2c0 con 0x230c2c60 -7127> 2014-11-03 11:37:50.024993 7f2912507700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.024993, event: sub_op_applied, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7126> 2014-11-03 11:37:50.025005 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 42 ==== osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1073270054 0 3511520646) 0x2c690c00 con 0x230c2c60 -7125> 2014-11-03 11:37:50.025047 7f28ee16d700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.024833, event: header_read, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7124> 2014-11-03 11:37:50.025053 7f2912507700 5 -- op tracker -- seq: 2908, time: 2014-11-03 11:37:50.025053, event: done, op: osd_sub_op(client.7869107.0:21845086 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132374 snapset=0=[]:[] snapc=0=[]) -7123> 2014-11-03 11:37:50.025064 7f28ee16d700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.024835, event: throttled, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7122> 2014-11-03 11:37:50.025078 7f28ee16d700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.024983, event: all_read, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7121> 2014-11-03 11:37:50.025097 7f28ee16d700 5 -- op tracker -- seq: 2909, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7120> 2014-11-03 11:37:50.025145 7f290001b700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.025145, event: reached_pg, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7119> 2014-11-03 11:37:50.025166 7f290001b700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.025166, event: started, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7118> 2014-11-03 11:37:50.025276 7f290001b700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.025276, event: started, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7117> 2014-11-03 11:37:50.025318 7f290001b700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.025317, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7116> 2014-11-03 11:37:50.025416 7f2914d0c700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.025416, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7115> 2014-11-03 11:37:50.025662 7f291450b700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.025662, event: journaled_completion_queued, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7114> 2014-11-03 11:37:50.026207 7f2911d06700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.026207, event: commit_sent, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7113> 2014-11-03 11:37:50.026281 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294ae600 con 0x230c2c60 -7112> 2014-11-03 11:37:50.026624 7f2912507700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.026624, event: sub_op_applied, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7111> 2014-11-03 11:37:50.026692 7f2912507700 5 -- op tracker -- seq: 2909, time: 2014-11-03 11:37:50.026692, event: done, op: osd_sub_op(client.7869107.0:21845087 6.1c1 ed3e61c1/rbd_data.4f5ffb6c09e38c.00000000000000d0/head//6 [] v 105950'8132375 snapset=0=[]:[] snapc=0=[]) -7110> 2014-11-03 11:37:50.046340 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.93 192.168.5.7:0/7095 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.044741) v2 ==== 47+0+0 (909349989 0 0) 0x2579f200 con 0x24925160 -7109> 2014-11-03 11:37:50.046404 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/7095 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.044741) v2 -- ?+0 0x2796c000 con 0x24925160 -7108> 2014-11-03 11:37:50.046436 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.93 192.168.5.7:0/7095 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.044741) v2 ==== 47+0+0 (909349989 0 0) 0x236a8600 con 0x23ea4dc0 -7107> 2014-11-03 11:37:50.046482 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/7095 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.044741) v2 -- ?+0 0x2a3f8400 con 0x23ea4dc0 -7106> 2014-11-03 11:37:50.053834 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 94 ==== osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (201176669 0 3511520646) 0x2d3a7c00 con 0x22fbc580 -7105> 2014-11-03 11:37:50.053868 7f28f70fc700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.053709, event: header_read, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7104> 2014-11-03 11:37:50.053883 7f28f70fc700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.053710, event: throttled, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7103> 2014-11-03 11:37:50.053891 7f28f70fc700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.053821, event: all_read, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7102> 2014-11-03 11:37:50.053899 7f28f70fc700 5 -- op tracker -- seq: 2910, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7101> 2014-11-03 11:37:50.053952 7f290101d700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.053952, event: reached_pg, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7100> 2014-11-03 11:37:50.054002 7f290101d700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.054002, event: started, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7099> 2014-11-03 11:37:50.054234 7f290101d700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.054233, event: started, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7098> 2014-11-03 11:37:50.054281 7f290101d700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.054281, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7097> 2014-11-03 11:37:50.054368 7f2914d0c700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.054368, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7096> 2014-11-03 11:37:50.054695 7f291450b700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.054695, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7095> 2014-11-03 11:37:50.054799 7f2911d06700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.054799, event: commit_sent, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7094> 2014-11-03 11:37:50.054864 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240d3340 con 0x22fbc580 -7093> 2014-11-03 11:37:50.055126 7f2912507700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.055126, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7092> 2014-11-03 11:37:50.055195 7f2912507700 5 -- op tracker -- seq: 2910, time: 2014-11-03 11:37:50.055195, event: done, op: osd_sub_op(client.8236383.0:16630653 6.4ee 781eccee/rbd_data.7e655d2ae8944a.000000000000448f/head//6 [] v 105950'48068288 snapset=0=[]:[] snapc=0=[]) -7091> 2014-11-03 11:37:50.196646 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.63 192.168.5.5:0/21346 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.195101) v2 ==== 47+0+0 (78988173 0 0) 0x23fdfa00 con 0x233cb000 -7090> 2014-11-03 11:37:50.196646 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.63 192.168.5.5:0/21346 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.195101) v2 ==== 47+0+0 (78988173 0 0) 0x2a9d3000 con 0x233cb420 -7089> 2014-11-03 11:37:50.196736 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/21346 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.195101) v2 -- ?+0 0x2579f200 con 0x233cb000 -7088> 2014-11-03 11:37:50.196829 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/21346 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.195101) v2 -- ?+0 0x236a8600 con 0x233cb420 -7087> 2014-11-03 11:37:50.201431 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.3 192.168.5.1:0/28138 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.200860) v2 ==== 47+0+0 (3333837860 0 0) 0x8abfa00 con 0x23ea46e0 -7086> 2014-11-03 11:37:50.201510 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/28138 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.200860) v2 -- ?+0 0x2a9d3000 con 0x23ea46e0 -7085> 2014-11-03 11:37:50.201442 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.3 192.168.5.1:0/28138 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.200860) v2 ==== 47+0+0 (3333837860 0 0) 0x23510200 con 0x2445c580 -7084> 2014-11-03 11:37:50.201585 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/28138 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.200860) v2 -- ?+0 0x23fdfa00 con 0x2445c580 -7083> 2014-11-03 11:37:50.220642 7f2918d14700 5 osd.21 105950 tick -7082> 2014-11-03 11:37:50.228096 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.108 192.168.5.8:0/21567 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.226771) v2 ==== 47+0+0 (4151596602 0 0) 0x2372b400 con 0x243212c0 -7081> 2014-11-03 11:37:50.228103 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.108 192.168.5.8:0/21567 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.226771) v2 ==== 47+0+0 (4151596602 0 0) 0x273fd800 con 0x2484a840 -7080> 2014-11-03 11:37:50.228144 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.226771) v2 -- ?+0 0x8abfa00 con 0x243212c0 -7079> 2014-11-03 11:37:50.228325 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.226771) v2 -- ?+0 0x23510200 con 0x2484a840 -7078> 2014-11-03 11:37:50.230268 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 162 ==== osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2196868754 0 3511520646) 0x2605a800 con 0x22fefdc0 -7077> 2014-11-03 11:37:50.230332 7f28f36c2700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.229848, event: header_read, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7076> 2014-11-03 11:37:50.230374 7f28f36c2700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.229850, event: throttled, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7075> 2014-11-03 11:37:50.230453 7f28f36c2700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.230251, event: all_read, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7074> 2014-11-03 11:37:50.230499 7f28f36c2700 5 -- op tracker -- seq: 2911, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7073> 2014-11-03 11:37:50.230742 7f28fe818700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.230742, event: reached_pg, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7072> 2014-11-03 11:37:50.230791 7f28fe818700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.230791, event: started, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7071> 2014-11-03 11:37:50.230902 7f28fe818700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.230902, event: started, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7070> 2014-11-03 11:37:50.230935 7f28fe818700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.230935, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7069> 2014-11-03 11:37:50.231014 7f2914d0c700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.231014, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7068> 2014-11-03 11:37:50.231273 7f291450b700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.231272, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7067> 2014-11-03 11:37:50.231345 7f2911d06700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.231344, event: commit_sent, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7066> 2014-11-03 11:37:50.231413 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x240d22c0 con 0x22fefdc0 -7065> 2014-11-03 11:37:50.231821 7f2912507700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.231820, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7064> 2014-11-03 11:37:50.231855 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 91 ==== osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1047390845 0 3511520646) 0x2448d000 con 0x22fbcb00 -7063> 2014-11-03 11:37:50.231895 7f28f6cf8700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.231680, event: header_read, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7062> 2014-11-03 11:37:50.231908 7f2912d08700 5 -- op tracker -- seq: 2911, time: 2014-11-03 11:37:50.231908, event: done, op: osd_sub_op(client.8703713.0:32254383 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953144 snapset=0=[]:[] snapc=0=[]) -7061> 2014-11-03 11:37:50.231941 7f28f6cf8700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.231684, event: throttled, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7060> 2014-11-03 11:37:50.231952 7f28f6cf8700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.231821, event: all_read, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7059> 2014-11-03 11:37:50.231960 7f28f6cf8700 5 -- op tracker -- seq: 2912, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7058> 2014-11-03 11:37:50.232055 7f290081c700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232055, event: reached_pg, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7057> 2014-11-03 11:37:50.232099 7f290081c700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232099, event: started, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7056> 2014-11-03 11:37:50.232220 7f290081c700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232220, event: started, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7055> 2014-11-03 11:37:50.232285 7f290081c700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232284, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7054> 2014-11-03 11:37:50.232390 7f2914d0c700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232390, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7053> 2014-11-03 11:37:50.232655 7f291450b700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232655, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7052> 2014-11-03 11:37:50.232698 7f2911d06700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232698, event: commit_sent, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7051> 2014-11-03 11:37:50.232747 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240d2840 con 0x22fbcb00 -7050> 2014-11-03 11:37:50.232981 7f2912507700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.232980, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7049> 2014-11-03 11:37:50.233018 7f2912507700 5 -- op tracker -- seq: 2912, time: 2014-11-03 11:37:50.233018, event: done, op: osd_sub_op(client.7868288.0:25603437 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134367 snapset=0=[]:[] snapc=0=[]) -7048> 2014-11-03 11:37:50.243253 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 28 ==== MOSDPGPush(6.58e 105950 [PushOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, version: 105945'11622218, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (336136300 0 0) 0x229e7c00 con 0x231abb00 -7047> 2014-11-03 11:37:50.243315 7f28eb945700 5 -- op tracker -- seq: 2913, time: 2014-11-03 11:37:50.226168, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, version: 105945'11622218, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7046> 2014-11-03 11:37:50.243353 7f28eb945700 5 -- op tracker -- seq: 2913, time: 2014-11-03 11:37:50.226171, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, version: 105945'11622218, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7045> 2014-11-03 11:37:50.243380 7f28eb945700 5 -- op tracker -- seq: 2913, time: 2014-11-03 11:37:50.243236, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, version: 105945'11622218, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7044> 2014-11-03 11:37:50.243405 7f28eb945700 5 -- op tracker -- seq: 2913, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, version: 105945'11622218, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7043> 2014-11-03 11:37:50.243483 7f290101d700 5 -- op tracker -- seq: 2913, time: 2014-11-03 11:37:50.243483, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, version: 105945'11622218, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7042> 2014-11-03 11:37:50.243769 7f290101d700 5 -- op tracker -- seq: 2913, time: 2014-11-03 11:37:50.243769, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6, version: 105945'11622218, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(dee6958e/rbd_data.750d312780d680.00000000000002fb/head//6@105945'11622218, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -7041> 2014-11-03 11:37:50.246717 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 41 ==== osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) v2 ==== 178+0+0 (1282893342 0 0) 0x23e438c0 con 0x22fe0000 -7040> 2014-11-03 11:37:50.246749 7f28f4fdb700 5 -- op tracker -- seq: 2914, time: 2014-11-03 11:37:50.246603, event: header_read, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7039> 2014-11-03 11:37:50.246762 7f28f4fdb700 5 -- op tracker -- seq: 2914, time: 2014-11-03 11:37:50.246605, event: throttled, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7038> 2014-11-03 11:37:50.246771 7f28f4fdb700 5 -- op tracker -- seq: 2914, time: 2014-11-03 11:37:50.246704, event: all_read, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7037> 2014-11-03 11:37:50.246779 7f28f4fdb700 5 -- op tracker -- seq: 2914, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7036> 2014-11-03 11:37:50.246841 7f28fd816700 5 -- op tracker -- seq: 2914, time: 2014-11-03 11:37:50.246841, event: reached_pg, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7035> 2014-11-03 11:37:50.246867 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 42 ==== osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) v2 ==== 178+0+0 (2320947587 0 0) 0x272ea000 con 0x22fe0000 -7034> 2014-11-03 11:37:50.246878 7f28fd816700 5 -- op tracker -- seq: 2914, time: 2014-11-03 11:37:50.246878, event: started, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7033> 2014-11-03 11:37:50.246904 7f28f4fdb700 5 -- op tracker -- seq: 2915, time: 2014-11-03 11:37:50.246815, event: header_read, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7032> 2014-11-03 11:37:50.246933 7f28f4fdb700 5 -- op tracker -- seq: 2915, time: 2014-11-03 11:37:50.246816, event: throttled, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7031> 2014-11-03 11:37:50.246921 7f28fd816700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:50.246921, event: sub_op_applied_rec, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7030> 2014-11-03 11:37:50.246943 7f28f4fdb700 5 -- op tracker -- seq: 2915, time: 2014-11-03 11:37:50.246858, event: all_read, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7029> 2014-11-03 11:37:50.246954 7f28f4fdb700 5 -- op tracker -- seq: 2915, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7028> 2014-11-03 11:37:50.246951 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.3:0/1015168 -- osd_op_reply(9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] v105950'819291 uv819291 ack = 0) v6 -- ?+0 0x28a7d8c0 con 0x245379a0 -7027> 2014-11-03 11:37:50.246994 7f28fd816700 5 -- op tracker -- seq: 2914, time: 2014-11-03 11:37:50.246994, event: done, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7026> 2014-11-03 11:37:50.247001 7f290001b700 5 -- op tracker -- seq: 2915, time: 2014-11-03 11:37:50.247001, event: reached_pg, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7025> 2014-11-03 11:37:50.247024 7f290001b700 5 -- op tracker -- seq: 2915, time: 2014-11-03 11:37:50.247024, event: started, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7024> 2014-11-03 11:37:50.247035 7f290001b700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:50.247035, event: sub_op_applied_rec, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7023> 2014-11-03 11:37:50.247095 7f290001b700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.3:0/1015168 -- osd_op_reply(9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] v105950'819292 uv819292 ack = 0) v6 -- ?+0 0x3110d840 con 0x245379a0 -7022> 2014-11-03 11:37:50.247216 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 43 ==== osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (1750579135 0 0) 0x272ea580 con 0x22fe0000 -7021> 2014-11-03 11:37:50.247242 7f28f4fdb700 5 -- op tracker -- seq: 2916, time: 2014-11-03 11:37:50.246983, event: header_read, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7020> 2014-11-03 11:37:50.247248 7f290001b700 5 -- op tracker -- seq: 2915, time: 2014-11-03 11:37:50.247248, event: done, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ack, result = 0) -7019> 2014-11-03 11:37:50.247258 7f28f4fdb700 5 -- op tracker -- seq: 2916, time: 2014-11-03 11:37:50.246984, event: throttled, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7018> 2014-11-03 11:37:50.247299 7f28f4fdb700 5 -- op tracker -- seq: 2916, time: 2014-11-03 11:37:50.247206, event: all_read, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7017> 2014-11-03 11:37:50.247315 7f28f4fdb700 5 -- op tracker -- seq: 2916, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7016> 2014-11-03 11:37:50.247373 7f28fd816700 5 -- op tracker -- seq: 2916, time: 2014-11-03 11:37:50.247373, event: reached_pg, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7015> 2014-11-03 11:37:50.247406 7f28fd816700 5 -- op tracker -- seq: 2916, time: 2014-11-03 11:37:50.247406, event: started, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7014> 2014-11-03 11:37:50.247417 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 44 ==== osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (2745161479 0 0) 0x272eb600 con 0x22fe0000 -7013> 2014-11-03 11:37:50.247448 7f28fd816700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:50.247447, event: sub_op_commit_rec, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7012> 2014-11-03 11:37:50.247472 7f28f4fdb700 5 -- op tracker -- seq: 2917, time: 2014-11-03 11:37:50.247359, event: header_read, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7011> 2014-11-03 11:37:50.247492 7f28f4fdb700 5 -- op tracker -- seq: 2917, time: 2014-11-03 11:37:50.247360, event: throttled, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7010> 2014-11-03 11:37:50.247503 7f28f4fdb700 5 -- op tracker -- seq: 2917, time: 2014-11-03 11:37:50.247406, event: all_read, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7009> 2014-11-03 11:37:50.247514 7f28f4fdb700 5 -- op tracker -- seq: 2917, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7008> 2014-11-03 11:37:50.247506 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.3:0/1015168 -- osd_op_reply(9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] v105950'819291 uv819291 ondisk = 0) v6 -- ?+0 0x23e438c0 con 0x245379a0 -7007> 2014-11-03 11:37:50.247545 7f28fd816700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:50.247545, event: commit_sent, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7006> 2014-11-03 11:37:50.247613 7f28fd816700 5 -- op tracker -- seq: 2867, time: 2014-11-03 11:37:50.247613, event: done, op: osd_op(client.9545673.0:9861301 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3276800~12288] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7005> 2014-11-03 11:37:50.247668 7f28fd816700 5 -- op tracker -- seq: 2916, time: 2014-11-03 11:37:50.247668, event: done, op: osd_sub_op_reply(client.9545673.0:9861301 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7004> 2014-11-03 11:37:50.247676 7f290001b700 5 -- op tracker -- seq: 2917, time: 2014-11-03 11:37:50.247675, event: reached_pg, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7003> 2014-11-03 11:37:50.247702 7f290001b700 5 -- op tracker -- seq: 2917, time: 2014-11-03 11:37:50.247702, event: started, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -7002> 2014-11-03 11:37:50.247746 7f290001b700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:50.247746, event: sub_op_commit_rec, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -7001> 2014-11-03 11:37:50.247783 7f290001b700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.3:0/1015168 -- osd_op_reply(9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] v105950'819292 uv819292 ondisk = 0) v6 -- ?+0 0x272ea000 con 0x245379a0 -7000> 2014-11-03 11:37:50.247808 7f290001b700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:50.247808, event: commit_sent, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -6999> 2014-11-03 11:37:50.247928 7f290001b700 5 -- op tracker -- seq: 2868, time: 2014-11-03 11:37:50.247928, event: done, op: osd_op(client.9545673.0:9861302 rbd_data.5348922ae8944a.0000000000001e2e [set-alloc-hint object_size 4194304 write_size 4194304,write 3366912~4096] 24.258b1db8 snapc 301=[301,2fd,2a6] ack+ondisk+write e105950) -6998> 2014-11-03 11:37:50.247998 7f290001b700 5 -- op tracker -- seq: 2917, time: 2014-11-03 11:37:50.247997, event: done, op: osd_sub_op_reply(client.9545673.0:9861302 24.5b8 258b1db8/rbd_data.5348922ae8944a.0000000000001e2e/head//24 [] ondisk, result = 0) -6997> 2014-11-03 11:37:50.248148 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 163 ==== osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (297976384 0 3511520646) 0x2bd4ec00 con 0x22fefdc0 -6996> 2014-11-03 11:37:50.248178 7f28f36c2700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.247972, event: header_read, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6995> 2014-11-03 11:37:50.248190 7f28f36c2700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.247974, event: throttled, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6994> 2014-11-03 11:37:50.248201 7f28f36c2700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.248134, event: all_read, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6993> 2014-11-03 11:37:50.248214 7f28f36c2700 5 -- op tracker -- seq: 2918, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6992> 2014-11-03 11:37:50.248305 7f28fe818700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.248305, event: reached_pg, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6991> 2014-11-03 11:37:50.248360 7f28fe818700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.248359, event: started, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6990> 2014-11-03 11:37:50.248545 7f28fe818700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.248544, event: started, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6989> 2014-11-03 11:37:50.248609 7f28fe818700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.248609, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6988> 2014-11-03 11:37:50.251035 7f2914d0c700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.251035, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6987> 2014-11-03 11:37:50.251351 7f291450b700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.251351, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6986> 2014-11-03 11:37:50.251425 7f2911d06700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.251425, event: commit_sent, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6985> 2014-11-03 11:37:50.251462 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x240d2000 con 0x22fefdc0 -6984> 2014-11-03 11:37:50.251707 7f2912507700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.251707, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6983> 2014-11-03 11:37:50.251772 7f2912507700 5 -- op tracker -- seq: 2918, time: 2014-11-03 11:37:50.251772, event: done, op: osd_sub_op(client.8703713.0:32254386 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953145 snapset=0=[]:[] snapc=0=[]) -6982> 2014-11-03 11:37:50.255991 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x2340da00 con 0x22fbc420 -6981> 2014-11-03 11:37:50.268234 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.38 192.168.5.3:0/18942 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.267186) v2 ==== 47+0+0 (2814524930 0 0) 0x2a8d7800 con 0x2469e6e0 -6980> 2014-11-03 11:37:50.268278 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.38 192.168.5.3:0/18942 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.267186) v2 ==== 47+0+0 (2814524930 0 0) 0x22fad000 con 0x240ad160 -6979> 2014-11-03 11:37:50.268296 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/18942 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.267186) v2 -- ?+0 0x2372b400 con 0x2469e6e0 -6978> 2014-11-03 11:37:50.268372 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/18942 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.267186) v2 -- ?+0 0x273fd800 con 0x240ad160 -6977> 2014-11-03 11:37:50.295664 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 95 ==== osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4086111018 0 3511520646) 0x2c336000 con 0x22fbc580 -6976> 2014-11-03 11:37:50.295726 7f28f70fc700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.295483, event: header_read, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6975> 2014-11-03 11:37:50.295752 7f28f70fc700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.295488, event: throttled, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6974> 2014-11-03 11:37:50.295775 7f28f70fc700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.295642, event: all_read, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6973> 2014-11-03 11:37:50.295787 7f28f70fc700 5 -- op tracker -- seq: 2919, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6972> 2014-11-03 11:37:50.295943 7f28ff81a700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.295943, event: reached_pg, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6971> 2014-11-03 11:37:50.296005 7f28ff81a700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.296005, event: started, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6970> 2014-11-03 11:37:50.296169 7f28ff81a700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.296169, event: started, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6969> 2014-11-03 11:37:50.296239 7f28ff81a700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.296239, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6968> 2014-11-03 11:37:50.296372 7f2914d0c700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.296372, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6967> 2014-11-03 11:37:50.296775 7f291450b700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.296774, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6966> 2014-11-03 11:37:50.296945 7f2911d06700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.296945, event: commit_sent, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6965> 2014-11-03 11:37:50.296984 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x922cdc0 con 0x22fbc580 -6964> 2014-11-03 11:37:50.297352 7f2912507700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.297352, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6963> 2014-11-03 11:37:50.297414 7f2912507700 5 -- op tracker -- seq: 2919, time: 2014-11-03 11:37:50.297413, event: done, op: osd_sub_op(client.7873619.0:18867213 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228184 snapset=0=[]:[] snapc=0=[]) -6962> 2014-11-03 11:37:50.340899 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 87 ==== MOSDPGPush(6.58e 105950 [PushOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, version: 105945'11622234, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (2650603678 0 0) 0x2340da00 con 0x22fbc420 -6961> 2014-11-03 11:37:50.340954 7f28f72fe700 5 -- op tracker -- seq: 2920, time: 2014-11-03 11:37:50.330536, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, version: 105945'11622234, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6960> 2014-11-03 11:37:50.340991 7f28f72fe700 5 -- op tracker -- seq: 2920, time: 2014-11-03 11:37:50.330553, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, version: 105945'11622234, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6959> 2014-11-03 11:37:50.341026 7f28f72fe700 5 -- op tracker -- seq: 2920, time: 2014-11-03 11:37:50.340887, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, version: 105945'11622234, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6958> 2014-11-03 11:37:50.341051 7f28f72fe700 5 -- op tracker -- seq: 2920, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, version: 105945'11622234, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6957> 2014-11-03 11:37:50.341130 7f290101d700 5 -- op tracker -- seq: 2920, time: 2014-11-03 11:37:50.341130, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, version: 105945'11622234, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6956> 2014-11-03 11:37:50.341362 7f290101d700 5 -- op tracker -- seq: 2920, time: 2014-11-03 11:37:50.341362, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6, version: 105945'11622234, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e449058e/rbd_data.45e62779c99cf1.00000000000004a3/head//6@105945'11622234, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6955> 2014-11-03 11:37:50.355321 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x187f4400 con 0x231abb00 -6954> 2014-11-03 11:37:50.362855 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.36 192.168.5.3:0/9756 5 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.362181) v2 ==== 47+0+0 (758476956 0 0) 0x23408000 con 0x240ad2c0 -6953> 2014-11-03 11:37:50.362915 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/9756 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.362181) v2 -- ?+0 0x22fad000 con 0x240ad2c0 -6952> 2014-11-03 11:37:50.362974 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.36 192.168.5.3:0/9756 5 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.362181) v2 ==== 47+0+0 (758476956 0 0) 0x2d6eca00 con 0x2469e840 -6951> 2014-11-03 11:37:50.363019 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/9756 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.362181) v2 -- ?+0 0x2a8d7800 con 0x2469e840 -6950> 2014-11-03 11:37:50.365721 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 92 ==== osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1849848342 0 3511520646) 0x26942c00 con 0x22fbcb00 -6949> 2014-11-03 11:37:50.365776 7f28f6cf8700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.365590, event: header_read, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6948> 2014-11-03 11:37:50.365796 7f28f6cf8700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.365592, event: throttled, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6947> 2014-11-03 11:37:50.365809 7f28f6cf8700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.365700, event: all_read, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6946> 2014-11-03 11:37:50.365835 7f28f6cf8700 5 -- op tracker -- seq: 2921, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6945> 2014-11-03 11:37:50.365973 7f28fe017700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.365973, event: reached_pg, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6944> 2014-11-03 11:37:50.366031 7f28fe017700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.366031, event: started, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6943> 2014-11-03 11:37:50.366269 7f28fe017700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.366269, event: started, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6942> 2014-11-03 11:37:50.366336 7f28fe017700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.366336, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6941> 2014-11-03 11:37:50.366506 7f2914d0c700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.366506, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6940> 2014-11-03 11:37:50.366948 7f291450b700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.366947, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6939> 2014-11-03 11:37:50.367078 7f2911d06700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.367078, event: commit_sent, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6938> 2014-11-03 11:37:50.367133 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] ondisk, result = 0) v2 -- ?+0 0x922d8c0 con 0x22fbcb00 -6937> 2014-11-03 11:37:50.367498 7f2912507700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.367497, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6936> 2014-11-03 11:37:50.367574 7f2912507700 5 -- op tracker -- seq: 2921, time: 2014-11-03 11:37:50.367574, event: done, op: osd_sub_op(client.7868288.0:25603438 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134368 snapset=0=[]:[] snapc=0=[]) -6935> 2014-11-03 11:37:50.371371 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 93 ==== osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1252884821 0 3511520646) 0x26942000 con 0x22fbcb00 -6934> 2014-11-03 11:37:50.371420 7f28f6cf8700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371243, event: header_read, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6933> 2014-11-03 11:37:50.371444 7f28f6cf8700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371245, event: throttled, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6932> 2014-11-03 11:37:50.371458 7f28f6cf8700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371355, event: all_read, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6931> 2014-11-03 11:37:50.371471 7f28f6cf8700 5 -- op tracker -- seq: 2922, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6930> 2014-11-03 11:37:50.371605 7f290081c700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371605, event: reached_pg, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6929> 2014-11-03 11:37:50.371663 7f290081c700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371662, event: started, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6928> 2014-11-03 11:37:50.371813 7f290081c700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371813, event: started, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6927> 2014-11-03 11:37:50.371859 7f290081c700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371859, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6926> 2014-11-03 11:37:50.371986 7f2914d0c700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.371985, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6925> 2014-11-03 11:37:50.372416 7f291450b700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.372416, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6924> 2014-11-03 11:37:50.372594 7f2911d06700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.372594, event: commit_sent, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6923> 2014-11-03 11:37:50.372652 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bd000 con 0x22fbcb00 -6922> 2014-11-03 11:37:50.372965 7f2912507700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.372964, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6921> 2014-11-03 11:37:50.373030 7f2912507700 5 -- op tracker -- seq: 2922, time: 2014-11-03 11:37:50.373030, event: done, op: osd_sub_op(client.7868288.0:25603439 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134369 snapset=0=[]:[] snapc=0=[]) -6920> 2014-11-03 11:37:50.375539 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 94 ==== osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3577653892 0 3511520646) 0x27072400 con 0x22fbcb00 -6919> 2014-11-03 11:37:50.375589 7f28f6cf8700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.375414, event: header_read, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6918> 2014-11-03 11:37:50.375634 7f28f6cf8700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.375416, event: throttled, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6917> 2014-11-03 11:37:50.375669 7f28f6cf8700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.375523, event: all_read, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6916> 2014-11-03 11:37:50.375696 7f28f6cf8700 5 -- op tracker -- seq: 2923, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6915> 2014-11-03 11:37:50.375822 7f28fe017700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.375822, event: reached_pg, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6914> 2014-11-03 11:37:50.375876 7f28fe017700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.375876, event: started, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6913> 2014-11-03 11:37:50.376019 7f28fe017700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.376019, event: started, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6912> 2014-11-03 11:37:50.376064 7f28fe017700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.376063, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6911> 2014-11-03 11:37:50.376231 7f2914d0c700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.376231, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6910> 2014-11-03 11:37:50.376670 7f291450b700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.376669, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6909> 2014-11-03 11:37:50.376812 7f2911d06700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.376812, event: commit_sent, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6908> 2014-11-03 11:37:50.376874 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bd840 con 0x22fbcb00 -6907> 2014-11-03 11:37:50.377104 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 95 ==== osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (530529624 0 3511520646) 0x2cff3000 con 0x22fbcb00 -6906> 2014-11-03 11:37:50.377137 7f28f6cf8700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.376895, event: header_read, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6905> 2014-11-03 11:37:50.377151 7f28f6cf8700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.376899, event: throttled, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6904> 2014-11-03 11:37:50.377159 7f28f6cf8700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377042, event: all_read, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6903> 2014-11-03 11:37:50.377169 7f28f6cf8700 5 -- op tracker -- seq: 2924, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6902> 2014-11-03 11:37:50.377167 7f2912507700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.377167, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6901> 2014-11-03 11:37:50.377236 7f2912507700 5 -- op tracker -- seq: 2923, time: 2014-11-03 11:37:50.377236, event: done, op: osd_sub_op(client.7868288.0:25603440 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134370 snapset=0=[]:[] snapc=0=[]) -6900> 2014-11-03 11:37:50.377293 7f290081c700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377293, event: reached_pg, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6899> 2014-11-03 11:37:50.377322 7f290081c700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377322, event: started, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6898> 2014-11-03 11:37:50.377404 7f290081c700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377404, event: started, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6897> 2014-11-03 11:37:50.377443 7f290081c700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377442, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6896> 2014-11-03 11:37:50.377497 7f2914d0c700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377497, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6895> 2014-11-03 11:37:50.377719 7f291450b700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377719, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6894> 2014-11-03 11:37:50.377827 7f2911d06700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.377827, event: commit_sent, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6893> 2014-11-03 11:37:50.377884 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bd2c0 con 0x22fbcb00 -6892> 2014-11-03 11:37:50.378066 7f2912507700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.378066, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6891> 2014-11-03 11:37:50.378104 7f2912507700 5 -- op tracker -- seq: 2924, time: 2014-11-03 11:37:50.378103, event: done, op: osd_sub_op(client.7868288.0:25603441 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134371 snapset=0=[]:[] snapc=0=[]) -6890> 2014-11-03 11:37:50.378678 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 96 ==== osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2248809650 0 3511520646) 0x26c42000 con 0x22fbcb00 -6889> 2014-11-03 11:37:50.378724 7f28f6cf8700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.378534, event: header_read, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6888> 2014-11-03 11:37:50.378754 7f28f6cf8700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.378536, event: throttled, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6887> 2014-11-03 11:37:50.378769 7f28f6cf8700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.378657, event: all_read, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6886> 2014-11-03 11:37:50.378788 7f28f6cf8700 5 -- op tracker -- seq: 2925, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6885> 2014-11-03 11:37:50.378850 7f28fe017700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.378850, event: reached_pg, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6884> 2014-11-03 11:37:50.378876 7f28fe017700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.378876, event: started, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6883> 2014-11-03 11:37:50.378954 7f28fe017700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.378953, event: started, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6882> 2014-11-03 11:37:50.378982 7f28fe017700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.378982, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6881> 2014-11-03 11:37:50.379069 7f2914d0c700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.379069, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6880> 2014-11-03 11:37:50.379359 7f291450b700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.379359, event: journaled_completion_queued, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6879> 2014-11-03 11:37:50.379972 7f2911d06700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.379972, event: commit_sent, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6878> 2014-11-03 11:37:50.380000 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258be600 con 0x22fbcb00 -6877> 2014-11-03 11:37:50.380259 7f2912507700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.380258, event: sub_op_applied, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6876> 2014-11-03 11:37:50.380325 7f2912507700 5 -- op tracker -- seq: 2925, time: 2014-11-03 11:37:50.380325, event: done, op: osd_sub_op(client.7868288.0:25603442 6.323 3e36a323/rbd_data.52354a3c982097.0000000000000342/head//6 [] v 105950'14134372 snapset=0=[]:[] snapc=0=[]) -6875> 2014-11-03 11:37:50.397759 7f290c834700 1 -- 192.168.5.2:6802/7606 <== mon.0 192.168.5.201:6789/0 44 ==== pg_stats_ack(4 pgs tid 5) v1 ==== 120+0+0 (1144305286 0 0) 0x2350a800 con 0x22c29160 -6874> 2014-11-03 11:37:50.407122 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.53 192.168.5.4:0/5692 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.405421) v2 ==== 47+0+0 (3790849447 0 0) 0x2782fe00 con 0x242c0b00 -6873> 2014-11-03 11:37:50.407214 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/5692 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.405421) v2 -- ?+0 0x2d6eca00 con 0x242c0b00 -6872> 2014-11-03 11:37:50.407261 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.53 192.168.5.4:0/5692 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.405421) v2 ==== 47+0+0 (3790849447 0 0) 0x23307e00 con 0x23b1c9a0 -6871> 2014-11-03 11:37:50.407363 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/5692 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.405421) v2 -- ?+0 0x23408000 con 0x23b1c9a0 -6870> 2014-11-03 11:37:50.412549 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 107 ==== osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2598102209 0 3511520646) 0x28ea4800 con 0x22fb76e0 -6869> 2014-11-03 11:37:50.412596 7f28edd69700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.412375, event: header_read, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6868> 2014-11-03 11:37:50.412616 7f28edd69700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.412377, event: throttled, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6867> 2014-11-03 11:37:50.412629 7f28edd69700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.412534, event: all_read, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6866> 2014-11-03 11:37:50.412645 7f28edd69700 5 -- op tracker -- seq: 2926, time: 0.000000, event: dispatched, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6865> 2014-11-03 11:37:50.412708 7f28fd816700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.412708, event: reached_pg, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6864> 2014-11-03 11:37:50.412745 7f28fd816700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.412745, event: started, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6863> 2014-11-03 11:37:50.412899 7f28fd816700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.412899, event: started, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6862> 2014-11-03 11:37:50.412936 7f28fd816700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.412936, event: commit_queued_for_journal_write, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6861> 2014-11-03 11:37:50.413049 7f2914d0c700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.413049, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6860> 2014-11-03 11:37:50.413370 7f291450b700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.413370, event: journaled_completion_queued, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6859> 2014-11-03 11:37:50.413430 7f2911d06700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.413430, event: commit_sent, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6858> 2014-11-03 11:37:50.413467 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258beb80 con 0x22fb76e0 -6857> 2014-11-03 11:37:50.413816 7f2912507700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.413816, event: sub_op_applied, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6856> 2014-11-03 11:37:50.413862 7f2912507700 5 -- op tracker -- seq: 2926, time: 2014-11-03 11:37:50.413862, event: done, op: osd_sub_op(client.7921724.0:2173582 6.46e 363c46e/rbd_data.5007ab5e072ab6.000000000000032a/head//6 [] v 105950'10288359 snapset=0=[]:[] snapc=0=[]) -6855> 2014-11-03 11:37:50.418295 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.81 192.168.5.6:0/27170 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.416959) v2 ==== 47+0+0 (1187600389 0 0) 0x2579f400 con 0x24925b00 -6854> 2014-11-03 11:37:50.418352 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.416959) v2 -- ?+0 0x23307e00 con 0x24925b00 -6853> 2014-11-03 11:37:50.418424 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.81 192.168.5.6:0/27170 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.416959) v2 ==== 47+0+0 (1187600389 0 0) 0x26bdb200 con 0x24229840 -6852> 2014-11-03 11:37:50.418466 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.416959) v2 -- ?+0 0x2782fe00 con 0x24229840 -6851> 2014-11-03 11:37:50.419763 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 29 ==== MOSDPGPush(6.58e 105950 [PushOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, version: 105946'11622236, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195207+0+0 (1228726925 0 0) 0x187f4400 con 0x231abb00 -6850> 2014-11-03 11:37:50.419802 7f28eb945700 5 -- op tracker -- seq: 2927, time: 2014-11-03 11:37:50.404307, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, version: 105946'11622236, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6849> 2014-11-03 11:37:50.419825 7f28eb945700 5 -- op tracker -- seq: 2927, time: 2014-11-03 11:37:50.404310, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, version: 105946'11622236, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6848> 2014-11-03 11:37:50.419845 7f28eb945700 5 -- op tracker -- seq: 2927, time: 2014-11-03 11:37:50.419751, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, version: 105946'11622236, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6847> 2014-11-03 11:37:50.419863 7f28eb945700 5 -- op tracker -- seq: 2927, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, version: 105946'11622236, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6846> 2014-11-03 11:37:50.419942 7f28fe818700 5 -- op tracker -- seq: 2927, time: 2014-11-03 11:37:50.419941, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, version: 105946'11622236, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6845> 2014-11-03 11:37:50.420267 7f28fe818700 5 -- op tracker -- seq: 2927, time: 2014-11-03 11:37:50.420267, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6, version: 105946'11622236, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(95346d8e/rbd_data.885435b2bbeeb.000000000000d434/head//6@105946'11622236, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6844> 2014-11-03 11:37:50.449862 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 96 ==== osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (709642485 0 3511520646) 0xd55e800 con 0x22fbc580 -6843> 2014-11-03 11:37:50.449900 7f28f70fc700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.449776, event: header_read, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6842> 2014-11-03 11:37:50.449915 7f28f70fc700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.449778, event: throttled, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6841> 2014-11-03 11:37:50.449923 7f28f70fc700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.449849, event: all_read, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6840> 2014-11-03 11:37:50.449931 7f28f70fc700 5 -- op tracker -- seq: 2928, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6839> 2014-11-03 11:37:50.449990 7f290201f700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.449990, event: reached_pg, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6838> 2014-11-03 11:37:50.450028 7f290201f700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.450028, event: started, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6837> 2014-11-03 11:37:50.450177 7f290201f700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.450176, event: started, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6836> 2014-11-03 11:37:50.450222 7f290201f700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.450221, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6835> 2014-11-03 11:37:50.450386 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 45 ==== MOSDPGPush(6.1d3 105950 [PushOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6, version: 105945'9525350, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6@105945'9525350, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6, version: 105945'9525404, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6@105945'9525404, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390497+0+0 (1047761368 0 0) 0x286bb200 con 0x22fe0000 -6834> 2014-11-03 11:37:50.450562 7f28f4fdb700 5 -- op tracker -- seq: 2929, time: 2014-11-03 11:37:50.429427, event: header_read, op: MOSDPGPush(6.1d3 105950 [PushOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6, version: 105945'9525350, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6@105945'9525350, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6, version: 105945'9525404, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6@105945'9525404, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6833> 2014-11-03 11:37:50.450610 7f28f4fdb700 5 -- op tracker -- seq: 2929, time: 2014-11-03 11:37:50.429429, event: throttled, op: MOSDPGPush(6.1d3 105950 [PushOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6, version: 105945'9525350, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6@105945'9525350, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6, version: 105945'9525404, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6@105945'9525404, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6832> 2014-11-03 11:37:50.450636 7f28f4fdb700 5 -- op tracker -- seq: 2929, time: 2014-11-03 11:37:50.450364, event: all_read, op: MOSDPGPush(6.1d3 105950 [PushOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6, version: 105945'9525350, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6@105945'9525350, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6, version: 105945'9525404, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6@105945'9525404, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6831> 2014-11-03 11:37:50.450658 7f28f4fdb700 5 -- op tracker -- seq: 2929, time: 0.000000, event: dispatched, op: MOSDPGPush(6.1d3 105950 [PushOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6, version: 105945'9525350, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6@105945'9525350, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6, version: 105945'9525404, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6@105945'9525404, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6830> 2014-11-03 11:37:50.450795 7f290101d700 5 -- op tracker -- seq: 2929, time: 2014-11-03 11:37:50.450795, event: reached_pg, op: MOSDPGPush(6.1d3 105950 [PushOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6, version: 105945'9525350, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6@105945'9525350, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6, version: 105945'9525404, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6@105945'9525404, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6829> 2014-11-03 11:37:50.451019 7f290101d700 5 -- op tracker -- seq: 2929, time: 2014-11-03 11:37:50.451019, event: done, op: MOSDPGPush(6.1d3 105950 [PushOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6, version: 105945'9525350, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6@105945'9525350, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6, version: 105945'9525404, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6@105945'9525404, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6828> 2014-11-03 11:37:50.468013 7f2914d0c700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.468013, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6827> 2014-11-03 11:37:50.468285 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 46 ==== MOSDPGPush(6.1d3 105950 [PushOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6, version: 105946'9525435, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6@105946'9525435, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6, version: 105946'9525462, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6@105946'9525462, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (1276143238 0 0) 0x27256800 con 0x22fe0000 -6826> 2014-11-03 11:37:50.468371 7f28f4fdb700 5 -- op tracker -- seq: 2930, time: 2014-11-03 11:37:50.450758, event: header_read, op: MOSDPGPush(6.1d3 105950 [PushOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6, version: 105946'9525435, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6@105946'9525435, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6, version: 105946'9525462, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6@105946'9525462, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6825> 2014-11-03 11:37:50.468431 7f28f4fdb700 5 -- op tracker -- seq: 2930, time: 2014-11-03 11:37:50.450759, event: throttled, op: MOSDPGPush(6.1d3 105950 [PushOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6, version: 105946'9525435, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6@105946'9525435, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6, version: 105946'9525462, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6@105946'9525462, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6824> 2014-11-03 11:37:50.468459 7f28f4fdb700 5 -- op tracker -- seq: 2930, time: 2014-11-03 11:37:50.468261, event: all_read, op: MOSDPGPush(6.1d3 105950 [PushOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6, version: 105946'9525435, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6@105946'9525435, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6, version: 105946'9525462, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6@105946'9525462, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6823> 2014-11-03 11:37:50.468482 7f28f4fdb700 5 -- op tracker -- seq: 2930, time: 0.000000, event: dispatched, op: MOSDPGPush(6.1d3 105950 [PushOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6, version: 105946'9525435, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6@105946'9525435, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6, version: 105946'9525462, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6@105946'9525462, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6822> 2014-11-03 11:37:50.468560 7f28fe818700 5 -- op tracker -- seq: 2930, time: 2014-11-03 11:37:50.468559, event: reached_pg, op: MOSDPGPush(6.1d3 105950 [PushOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6, version: 105946'9525435, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6@105946'9525435, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6, version: 105946'9525462, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6@105946'9525462, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6821> 2014-11-03 11:37:50.468775 7f28fe818700 5 -- op tracker -- seq: 2930, time: 2014-11-03 11:37:50.468775, event: done, op: MOSDPGPush(6.1d3 105950 [PushOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6, version: 105946'9525435, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6@105946'9525435, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6, version: 105946'9525462, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6@105946'9525462, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6820> 2014-11-03 11:37:50.471549 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.111 192.168.5.8:0/32649 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.470478) v2 ==== 47+0+0 (1917961844 0 0) 0x243ffc00 con 0x23b1c000 -6819> 2014-11-03 11:37:50.471612 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.470478) v2 -- ?+0 0x2579f400 con 0x23b1c000 -6818> 2014-11-03 11:37:50.471664 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.111 192.168.5.8:0/32649 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.470478) v2 ==== 47+0+0 (1917961844 0 0) 0x2a347600 con 0x24229c60 -6817> 2014-11-03 11:37:50.471737 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.470478) v2 -- ?+0 0x26bdb200 con 0x24229c60 -6816> 2014-11-03 11:37:50.473471 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x187f4000 con 0x22fbc420 -6815> 2014-11-03 11:37:50.478395 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 47 ==== MOSDPGPush(6.1d3 105950 [PushOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6, version: 105946'9525533, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6@105946'9525533, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (2011434680 0 0) 0x2318aa00 con 0x22fe0000 -6814> 2014-11-03 11:37:50.478459 7f28f4fdb700 5 -- op tracker -- seq: 2931, time: 2014-11-03 11:37:50.468552, event: header_read, op: MOSDPGPush(6.1d3 105950 [PushOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6, version: 105946'9525533, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6@105946'9525533, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6813> 2014-11-03 11:37:50.478508 7f28f4fdb700 5 -- op tracker -- seq: 2931, time: 2014-11-03 11:37:50.468553, event: throttled, op: MOSDPGPush(6.1d3 105950 [PushOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6, version: 105946'9525533, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6@105946'9525533, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6812> 2014-11-03 11:37:50.478536 7f28f4fdb700 5 -- op tracker -- seq: 2931, time: 2014-11-03 11:37:50.478372, event: all_read, op: MOSDPGPush(6.1d3 105950 [PushOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6, version: 105946'9525533, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6@105946'9525533, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6811> 2014-11-03 11:37:50.478557 7f28f4fdb700 5 -- op tracker -- seq: 2931, time: 0.000000, event: dispatched, op: MOSDPGPush(6.1d3 105950 [PushOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6, version: 105946'9525533, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6@105946'9525533, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6810> 2014-11-03 11:37:50.478681 7f290101d700 5 -- op tracker -- seq: 2931, time: 2014-11-03 11:37:50.478680, event: reached_pg, op: MOSDPGPush(6.1d3 105950 [PushOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6, version: 105946'9525533, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6@105946'9525533, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6809> 2014-11-03 11:37:50.478864 7f290101d700 5 -- op tracker -- seq: 2931, time: 2014-11-03 11:37:50.478864, event: done, op: MOSDPGPush(6.1d3 105950 [PushOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6, version: 105946'9525533, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6@105946'9525533, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6808> 2014-11-03 11:37:50.490985 7f291450b700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.490984, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6807> 2014-11-03 11:37:50.491026 7f2911d06700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.491026, event: commit_sent, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6806> 2014-11-03 11:37:50.491092 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258be8c0 con 0x22fbc580 -6805> 2014-11-03 11:37:50.491378 7f2912507700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.491378, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6804> 2014-11-03 11:37:50.491429 7f2912507700 5 -- op tracker -- seq: 2928, time: 2014-11-03 11:37:50.491429, event: done, op: osd_sub_op(client.7873619.0:18867214 6.668 cbc23668/rbd_data.4f62fb60cec91a.00000000000000d0/head//6 [] v 105950'12228185 snapset=0=[]:[] snapc=0=[]) -6803> 2014-11-03 11:37:50.503373 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 66 ==== osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2970418051 0 3511520646) 0xd55f400 con 0x2308b2c0 -6802> 2014-11-03 11:37:50.503432 7f28eed79700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.503259, event: header_read, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6801> 2014-11-03 11:37:50.503457 7f28eed79700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.503262, event: throttled, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6800> 2014-11-03 11:37:50.503469 7f28eed79700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.503357, event: all_read, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6799> 2014-11-03 11:37:50.503521 7f28eed79700 5 -- op tracker -- seq: 2932, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6798> 2014-11-03 11:37:50.503649 7f28ff81a700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.503648, event: reached_pg, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6797> 2014-11-03 11:37:50.503719 7f28ff81a700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.503719, event: started, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6796> 2014-11-03 11:37:50.503880 7f28ff81a700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.503879, event: started, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6795> 2014-11-03 11:37:50.503931 7f28ff81a700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.503931, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6794> 2014-11-03 11:37:50.504345 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 67 ==== osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2411125830 0 3511520646) 0x191b7400 con 0x2308b2c0 -6793> 2014-11-03 11:37:50.504409 7f28eed79700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.504013, event: header_read, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6792> 2014-11-03 11:37:50.504448 7f28eed79700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.504014, event: throttled, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6791> 2014-11-03 11:37:50.504477 7f28eed79700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.504243, event: all_read, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6790> 2014-11-03 11:37:50.504509 7f28eed79700 5 -- op tracker -- seq: 2933, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6789> 2014-11-03 11:37:50.504608 7f290201f700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.504608, event: reached_pg, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6788> 2014-11-03 11:37:50.504671 7f290201f700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.504671, event: started, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6787> 2014-11-03 11:37:50.504820 7f290201f700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.504819, event: started, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6786> 2014-11-03 11:37:50.504867 7f290201f700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.504867, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6785> 2014-11-03 11:37:50.504881 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 68 ==== osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (1548626767 0 3511520646) 0x27806c00 con 0x2308b2c0 -6784> 2014-11-03 11:37:50.504880 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 92 ==== osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2377355480 0 3511520646) 0x2d6e7800 con 0x2308b840 -6783> 2014-11-03 11:37:50.504963 7f28eed79700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.504601, event: header_read, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6782> 2014-11-03 11:37:50.504972 7f28ef985700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.504728, event: header_read, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6781> 2014-11-03 11:37:50.505026 7f28eed79700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.504603, event: throttled, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6780> 2014-11-03 11:37:50.505060 7f28ef985700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.504732, event: throttled, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6779> 2014-11-03 11:37:50.505105 7f28eed79700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.504864, event: all_read, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6778> 2014-11-03 11:37:50.505110 7f28ef985700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.504853, event: all_read, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6777> 2014-11-03 11:37:50.505156 7f28eed79700 5 -- op tracker -- seq: 2934, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6776> 2014-11-03 11:37:50.505180 7f28ef985700 5 -- op tracker -- seq: 2935, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6775> 2014-11-03 11:37:50.505300 7f28ff81a700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.505299, event: reached_pg, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6774> 2014-11-03 11:37:50.505309 7f290201f700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.505309, event: reached_pg, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6773> 2014-11-03 11:37:50.505341 7f290201f700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.505341, event: started, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6772> 2014-11-03 11:37:50.505334 7f28ff81a700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.505334, event: started, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6771> 2014-11-03 11:37:50.505466 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 69 ==== osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (1126602422 0 3511520646) 0x26c44400 con 0x2308b2c0 -6770> 2014-11-03 11:37:50.505480 7f290201f700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.505479, event: started, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6769> 2014-11-03 11:37:50.505523 7f28eed79700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.505312, event: header_read, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6768> 2014-11-03 11:37:50.505503 7f28ff81a700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.505503, event: started, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6767> 2014-11-03 11:37:50.505542 7f28eed79700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.505315, event: throttled, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6766> 2014-11-03 11:37:50.505566 7f28eed79700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.505451, event: all_read, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6765> 2014-11-03 11:37:50.505565 7f290201f700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.505565, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6764> 2014-11-03 11:37:50.505591 7f28eed79700 5 -- op tracker -- seq: 2936, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6763> 2014-11-03 11:37:50.505634 7f28ff81a700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.505633, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6762> 2014-11-03 11:37:50.505668 7f290201f700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.505667, event: reached_pg, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6761> 2014-11-03 11:37:50.505706 7f290201f700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.505706, event: started, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6760> 2014-11-03 11:37:50.505836 7f290201f700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.505836, event: started, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6759> 2014-11-03 11:37:50.505884 7f290201f700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.505883, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6758> 2014-11-03 11:37:50.522737 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 33 ==== osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (935961000 0 3511520646) 0x2fdf4000 con 0x22ff3580 -6757> 2014-11-03 11:37:50.522778 7f28f17a3700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.522637, event: header_read, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6756> 2014-11-03 11:37:50.522793 7f28f17a3700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.522640, event: throttled, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6755> 2014-11-03 11:37:50.522806 7f28f17a3700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.522721, event: all_read, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6754> 2014-11-03 11:37:50.522814 7f28f17a3700 5 -- op tracker -- seq: 2937, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6753> 2014-11-03 11:37:50.522885 7f290001b700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.522885, event: reached_pg, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6752> 2014-11-03 11:37:50.522919 7f290001b700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.522919, event: started, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6751> 2014-11-03 11:37:50.523015 7f290001b700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.523015, event: started, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6750> 2014-11-03 11:37:50.523051 7f290001b700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.523050, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6749> 2014-11-03 11:37:50.523354 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 34 ==== osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3247481729 0 3511520646) 0x2c337800 con 0x22ff3580 -6748> 2014-11-03 11:37:50.523381 7f28f17a3700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.523261, event: header_read, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6747> 2014-11-03 11:37:50.523396 7f28f17a3700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.523263, event: throttled, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6746> 2014-11-03 11:37:50.523408 7f28f17a3700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.523344, event: all_read, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6745> 2014-11-03 11:37:50.523420 7f28f17a3700 5 -- op tracker -- seq: 2938, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6744> 2014-11-03 11:37:50.523469 7f28fd816700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.523469, event: reached_pg, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6743> 2014-11-03 11:37:50.523494 7f28fd816700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.523494, event: started, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6742> 2014-11-03 11:37:50.523590 7f28fd816700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.523590, event: started, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6741> 2014-11-03 11:37:50.523614 7f28fd816700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.523614, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6740> 2014-11-03 11:37:50.525004 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 35 ==== osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (91673353 0 3511520646) 0x2c336c00 con 0x22ff3580 -6739> 2014-11-03 11:37:50.525029 7f28f17a3700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.524930, event: header_read, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6738> 2014-11-03 11:37:50.525045 7f28f17a3700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.524931, event: throttled, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6737> 2014-11-03 11:37:50.525056 7f28f17a3700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.524996, event: all_read, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6736> 2014-11-03 11:37:50.525068 7f28f17a3700 5 -- op tracker -- seq: 2939, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6735> 2014-11-03 11:37:50.525116 7f290001b700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.525116, event: reached_pg, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6734> 2014-11-03 11:37:50.525139 7f290001b700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.525139, event: started, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6733> 2014-11-03 11:37:50.525224 7f290001b700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.525224, event: started, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6732> 2014-11-03 11:37:50.525206 7f2914d0c700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.525206, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6731> 2014-11-03 11:37:50.525243 7f290001b700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.525243, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6730> 2014-11-03 11:37:50.525254 7f2914d0c700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.525254, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6729> 2014-11-03 11:37:50.525285 7f2914d0c700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.525285, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6728> 2014-11-03 11:37:50.525307 7f2914d0c700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.525307, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6727> 2014-11-03 11:37:50.525334 7f2914d0c700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.525334, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6726> 2014-11-03 11:37:50.525351 7f2914d0c700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.525351, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6725> 2014-11-03 11:37:50.525367 7f2914d0c700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.525367, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6724> 2014-11-03 11:37:50.525388 7f2914d0c700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.525388, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6723> 2014-11-03 11:37:50.525968 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 36 ==== osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (441287316 0 3511520646) 0x29e1dc00 con 0x22ff3580 -6722> 2014-11-03 11:37:50.525993 7f28f17a3700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.525900, event: header_read, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6721> 2014-11-03 11:37:50.526008 7f28f17a3700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.525901, event: throttled, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6720> 2014-11-03 11:37:50.526019 7f28f17a3700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.525959, event: all_read, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6719> 2014-11-03 11:37:50.526031 7f28f17a3700 5 -- op tracker -- seq: 2940, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6718> 2014-11-03 11:37:50.526087 7f28fd816700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.526086, event: reached_pg, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6717> 2014-11-03 11:37:50.526127 7f28fd816700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.526127, event: started, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6716> 2014-11-03 11:37:50.526222 7f28fd816700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.526222, event: started, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6715> 2014-11-03 11:37:50.526250 7f28fd816700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.526250, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6714> 2014-11-03 11:37:50.526723 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 37 ==== osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (34528339 0 3511520646) 0x25ed5800 con 0x22ff3580 -6713> 2014-11-03 11:37:50.526766 7f28f17a3700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.526603, event: header_read, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6712> 2014-11-03 11:37:50.526791 7f28f17a3700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.526604, event: throttled, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6711> 2014-11-03 11:37:50.526821 7f28f17a3700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.526709, event: all_read, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6710> 2014-11-03 11:37:50.526834 7f28f17a3700 5 -- op tracker -- seq: 2941, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6709> 2014-11-03 11:37:50.526889 7f290001b700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.526889, event: reached_pg, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6708> 2014-11-03 11:37:50.526913 7f290001b700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.526913, event: started, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6707> 2014-11-03 11:37:50.526987 7f290001b700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.526987, event: started, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6706> 2014-11-03 11:37:50.527013 7f290001b700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.527013, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6705> 2014-11-03 11:37:50.528266 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 38 ==== osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4110584944 0 3511520646) 0x24e87400 con 0x22ff3580 -6704> 2014-11-03 11:37:50.528294 7f28f17a3700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.528179, event: header_read, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6703> 2014-11-03 11:37:50.528312 7f28f17a3700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.528181, event: throttled, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6702> 2014-11-03 11:37:50.528325 7f28f17a3700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.528256, event: all_read, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6701> 2014-11-03 11:37:50.528338 7f28f17a3700 5 -- op tracker -- seq: 2942, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6700> 2014-11-03 11:37:50.528392 7f28fd816700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.528392, event: reached_pg, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6699> 2014-11-03 11:37:50.528416 7f28fd816700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.528416, event: started, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6698> 2014-11-03 11:37:50.528489 7f28fd816700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.528489, event: started, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6697> 2014-11-03 11:37:50.528515 7f28fd816700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.528515, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6696> 2014-11-03 11:37:50.529008 7f291450b700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.529008, event: journaled_completion_queued, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6695> 2014-11-03 11:37:50.529060 7f291450b700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.529060, event: journaled_completion_queued, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6694> 2014-11-03 11:37:50.529083 7f291450b700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.529083, event: journaled_completion_queued, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6693> 2014-11-03 11:37:50.529082 7f2911d06700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:50.529082, event: commit_sent, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6692> 2014-11-03 11:37:50.529104 7f291450b700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.529104, event: journaled_completion_queued, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6691> 2014-11-03 11:37:50.529126 7f291450b700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.529126, event: journaled_completion_queued, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6690> 2014-11-03 11:37:50.529120 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 39 ==== osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3137441494 0 3511520646) 0x24e85000 con 0x22ff3580 -6689> 2014-11-03 11:37:50.529120 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28229080 con 0x2308b2c0 -6688> 2014-11-03 11:37:50.529165 7f291450b700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.529165, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6687> 2014-11-03 11:37:50.529167 7f28f17a3700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.529027, event: header_read, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6686> 2014-11-03 11:37:50.529197 7f28f17a3700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.529029, event: throttled, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6685> 2014-11-03 11:37:50.529198 7f2911d06700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:50.529198, event: commit_sent, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6684> 2014-11-03 11:37:50.529203 7f291450b700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.529202, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6683> 2014-11-03 11:37:50.529206 7f28f17a3700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.529109, event: all_read, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6682> 2014-11-03 11:37:50.529268 7f28f17a3700 5 -- op tracker -- seq: 2943, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6681> 2014-11-03 11:37:50.529276 7f291450b700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.529276, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6680> 2014-11-03 11:37:50.529328 7f290001b700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.529328, event: reached_pg, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6679> 2014-11-03 11:37:50.529345 7f2914d0c700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.529345, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6678> 2014-11-03 11:37:50.529355 7f290001b700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.529355, event: started, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6677> 2014-11-03 11:37:50.529366 7f2914d0c700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.529366, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6676> 2014-11-03 11:37:50.529399 7f2914d0c700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.529399, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6675> 2014-11-03 11:37:50.529377 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28228000 con 0x2308b2c0 -6674> 2014-11-03 11:37:50.529467 7f290001b700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.529467, event: started, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6673> 2014-11-03 11:37:50.529456 7f2911d06700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:50.529456, event: commit_sent, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6672> 2014-11-03 11:37:50.529532 7f290001b700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.529532, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6671> 2014-11-03 11:37:50.529551 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28228dc0 con 0x2308b2c0 -6670> 2014-11-03 11:37:50.529610 7f2911d06700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:50.529610, event: commit_sent, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6669> 2014-11-03 11:37:50.529643 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28228580 con 0x2308b840 -6668> 2014-11-03 11:37:50.529715 7f2911d06700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:50.529714, event: commit_sent, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6667> 2014-11-03 11:37:50.529747 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28229340 con 0x2308b2c0 -6666> 2014-11-03 11:37:50.529788 7f2911d06700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:50.529788, event: commit_sent, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6665> 2014-11-03 11:37:50.529790 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 40 ==== osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2283217167 0 3511520646) 0x27807800 con 0x22ff3580 -6664> 2014-11-03 11:37:50.529832 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28229b80 con 0x22ff3580 -6663> 2014-11-03 11:37:50.529878 7f28f17a3700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.529704, event: header_read, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6662> 2014-11-03 11:37:50.529948 7f28f17a3700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.529706, event: throttled, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6661> 2014-11-03 11:37:50.529974 7f28f17a3700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.529780, event: all_read, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6660> 2014-11-03 11:37:50.530010 7f28f17a3700 5 -- op tracker -- seq: 2944, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6659> 2014-11-03 11:37:50.529971 7f2911d06700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:50.529971, event: commit_sent, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6658> 2014-11-03 11:37:50.530063 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f600 con 0x22ff3580 -6657> 2014-11-03 11:37:50.530096 7f2911d06700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:50.530096, event: commit_sent, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6656> 2014-11-03 11:37:50.530178 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f080 con 0x22ff3580 -6655> 2014-11-03 11:37:50.530254 7f28fd816700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.530254, event: reached_pg, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6654> 2014-11-03 11:37:50.530308 7f28fd816700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.530308, event: started, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6653> 2014-11-03 11:37:50.530468 7f28fd816700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.530468, event: started, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6652> 2014-11-03 11:37:50.530523 7f28fd816700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.530523, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6651> 2014-11-03 11:37:50.532686 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 41 ==== osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2013124208 0 3511520646) 0x28ea3000 con 0x22ff3580 -6650> 2014-11-03 11:37:50.532718 7f28f17a3700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.532600, event: header_read, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6649> 2014-11-03 11:37:50.532731 7f28f17a3700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.532601, event: throttled, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6648> 2014-11-03 11:37:50.532739 7f28f17a3700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.532676, event: all_read, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6647> 2014-11-03 11:37:50.532747 7f28f17a3700 5 -- op tracker -- seq: 2945, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6646> 2014-11-03 11:37:50.532808 7f290001b700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.532808, event: reached_pg, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6645> 2014-11-03 11:37:50.532847 7f290001b700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.532847, event: started, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6644> 2014-11-03 11:37:50.533046 7f290001b700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.533046, event: started, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6643> 2014-11-03 11:37:50.533134 7f290001b700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.533134, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6642> 2014-11-03 11:37:50.534132 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 42 ==== osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (230298168 0 3511520646) 0x28ea3c00 con 0x22ff3580 -6641> 2014-11-03 11:37:50.534170 7f28f17a3700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.534008, event: header_read, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6640> 2014-11-03 11:37:50.534195 7f28f17a3700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.534009, event: throttled, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6639> 2014-11-03 11:37:50.534204 7f28f17a3700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.534099, event: all_read, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6638> 2014-11-03 11:37:50.534213 7f28f17a3700 5 -- op tracker -- seq: 2946, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6637> 2014-11-03 11:37:50.534319 7f28fd816700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.534318, event: reached_pg, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6636> 2014-11-03 11:37:50.534377 7f28fd816700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.534376, event: started, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6635> 2014-11-03 11:37:50.534551 7f28fd816700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.534551, event: started, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6634> 2014-11-03 11:37:50.534608 7f28fd816700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.534607, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6633> 2014-11-03 11:37:50.536064 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 43 ==== osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3432638753 0 3511520646) 0x2c692400 con 0x22ff3580 -6632> 2014-11-03 11:37:50.536093 7f28f17a3700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.535973, event: header_read, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6631> 2014-11-03 11:37:50.536106 7f28f17a3700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.535974, event: throttled, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6630> 2014-11-03 11:37:50.536114 7f28f17a3700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.536051, event: all_read, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6629> 2014-11-03 11:37:50.536124 7f28f17a3700 5 -- op tracker -- seq: 2947, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6628> 2014-11-03 11:37:50.536171 7f290001b700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.536170, event: reached_pg, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6627> 2014-11-03 11:37:50.536202 7f290001b700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.536202, event: started, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6626> 2014-11-03 11:37:50.536287 7f290001b700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.536287, event: started, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6625> 2014-11-03 11:37:50.536321 7f290001b700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.536321, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6624> 2014-11-03 11:37:50.536829 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 44 ==== osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4286817079 0 3511520646) 0x2c691800 con 0x22ff3580 -6623> 2014-11-03 11:37:50.536863 7f28f17a3700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.536753, event: header_read, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6622> 2014-11-03 11:37:50.536875 7f28f17a3700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.536755, event: throttled, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6621> 2014-11-03 11:37:50.536884 7f28f17a3700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.536820, event: all_read, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6620> 2014-11-03 11:37:50.536893 7f28f17a3700 5 -- op tracker -- seq: 2948, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6619> 2014-11-03 11:37:50.536975 7f28fd816700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.536974, event: reached_pg, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6618> 2014-11-03 11:37:50.537025 7f28fd816700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.537024, event: started, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6617> 2014-11-03 11:37:50.537166 7f28fd816700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.537166, event: started, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6616> 2014-11-03 11:37:50.537207 7f28fd816700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.537207, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6615> 2014-11-03 11:37:50.537391 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 45 ==== osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2479566888 0 3511520646) 0x2c690c00 con 0x22ff3580 -6614> 2014-11-03 11:37:50.537416 7f28f17a3700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.537322, event: header_read, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6613> 2014-11-03 11:37:50.537428 7f28f17a3700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.537323, event: throttled, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6612> 2014-11-03 11:37:50.537436 7f28f17a3700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.537383, event: all_read, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6611> 2014-11-03 11:37:50.537445 7f28f17a3700 5 -- op tracker -- seq: 2949, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6610> 2014-11-03 11:37:50.537486 7f290001b700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.537486, event: reached_pg, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6609> 2014-11-03 11:37:50.537507 7f290001b700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.537507, event: started, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6608> 2014-11-03 11:37:50.537576 7f290001b700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.537576, event: started, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6607> 2014-11-03 11:37:50.537607 7f290001b700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.537607, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6606> 2014-11-03 11:37:50.538185 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 46 ==== osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1013033080 0 3511520646) 0x2398e400 con 0x22ff3580 -6605> 2014-11-03 11:37:50.538210 7f28f17a3700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.538099, event: header_read, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6604> 2014-11-03 11:37:50.538222 7f28f17a3700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.538101, event: throttled, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6603> 2014-11-03 11:37:50.538230 7f28f17a3700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.538176, event: all_read, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6602> 2014-11-03 11:37:50.538238 7f28f17a3700 5 -- op tracker -- seq: 2950, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6601> 2014-11-03 11:37:50.538335 7f28fd816700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.538335, event: reached_pg, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6600> 2014-11-03 11:37:50.538386 7f28fd816700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.538386, event: started, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6599> 2014-11-03 11:37:50.538509 7f28fd816700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.538508, event: started, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6598> 2014-11-03 11:37:50.538556 7f28fd816700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.538556, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6597> 2014-11-03 11:37:50.538706 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 47 ==== osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2912547154 0 3511520646) 0x29e1f400 con 0x22ff3580 -6596> 2014-11-03 11:37:50.538731 7f28f17a3700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.538622, event: header_read, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6595> 2014-11-03 11:37:50.538748 7f28f17a3700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.538635, event: throttled, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6594> 2014-11-03 11:37:50.538764 7f28f17a3700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.538697, event: all_read, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6593> 2014-11-03 11:37:50.538772 7f28f17a3700 5 -- op tracker -- seq: 2951, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6592> 2014-11-03 11:37:50.538808 7f290001b700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.538807, event: reached_pg, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6591> 2014-11-03 11:37:50.538829 7f290001b700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.538829, event: started, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6590> 2014-11-03 11:37:50.538910 7f290001b700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.538910, event: started, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6589> 2014-11-03 11:37:50.538935 7f290001b700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.538935, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6588> 2014-11-03 11:37:50.539421 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 48 ==== osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3956305859 0 3511520646) 0x29e1e800 con 0x22ff3580 -6587> 2014-11-03 11:37:50.539446 7f28f17a3700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.539350, event: header_read, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6586> 2014-11-03 11:37:50.539458 7f28f17a3700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.539351, event: throttled, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6585> 2014-11-03 11:37:50.539466 7f28f17a3700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.539412, event: all_read, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6584> 2014-11-03 11:37:50.539480 7f28f17a3700 5 -- op tracker -- seq: 2952, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6583> 2014-11-03 11:37:50.539577 7f28fd816700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.539577, event: reached_pg, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6582> 2014-11-03 11:37:50.539630 7f28fd816700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.539629, event: started, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6581> 2014-11-03 11:37:50.539770 7f28fd816700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.539770, event: started, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6580> 2014-11-03 11:37:50.539820 7f28fd816700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.539820, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6579> 2014-11-03 11:37:50.540239 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 49 ==== osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4162593554 0 3511520646) 0x2bd4e000 con 0x22ff3580 -6578> 2014-11-03 11:37:50.540265 7f28f17a3700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.540166, event: header_read, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6577> 2014-11-03 11:37:50.540277 7f28f17a3700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.540168, event: throttled, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6576> 2014-11-03 11:37:50.540286 7f28f17a3700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.540231, event: all_read, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6575> 2014-11-03 11:37:50.540295 7f28f17a3700 5 -- op tracker -- seq: 2953, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6574> 2014-11-03 11:37:50.540335 7f290001b700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.540335, event: reached_pg, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6573> 2014-11-03 11:37:50.540355 7f290001b700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.540355, event: started, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6572> 2014-11-03 11:37:50.540431 7f290001b700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.540431, event: started, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6571> 2014-11-03 11:37:50.540460 7f290001b700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.540460, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6570> 2014-11-03 11:37:50.544703 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 93 ==== osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (17331789 0 3511520646) 0x2d6e6c00 con 0x2308b840 -6569> 2014-11-03 11:37:50.544773 7f28ef985700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.544519, event: header_read, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6568> 2014-11-03 11:37:50.544828 7f28ef985700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.544522, event: throttled, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6567> 2014-11-03 11:37:50.544861 7f28ef985700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.544673, event: all_read, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6566> 2014-11-03 11:37:50.544892 7f28ef985700 5 -- op tracker -- seq: 2954, time: 0.000000, event: dispatched, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6565> 2014-11-03 11:37:50.545005 7f290081c700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.545005, event: reached_pg, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6564> 2014-11-03 11:37:50.545057 7f290081c700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.545057, event: started, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6563> 2014-11-03 11:37:50.545198 7f290081c700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.545198, event: started, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6562> 2014-11-03 11:37:50.545248 7f290081c700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.545248, event: commit_queued_for_journal_write, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6561> 2014-11-03 11:37:50.546367 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.39 192.168.5.3:0/23401 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.545656) v2 ==== 47+0+0 (525200983 0 0) 0x2a8d7800 con 0x31a9000 -6560> 2014-11-03 11:37:50.546374 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.39 192.168.5.3:0/23401 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.545656) v2 ==== 47+0+0 (525200983 0 0) 0x2a7c4000 con 0x2389f000 -6559> 2014-11-03 11:37:50.546470 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/23401 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.545656) v2 -- ?+0 0x243ffc00 con 0x31a9000 -6558> 2014-11-03 11:37:50.546579 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/23401 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.545656) v2 -- ?+0 0x2a347600 con 0x2389f000 -6557> 2014-11-03 11:37:50.552798 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 88 ==== MOSDPGPush(6.58e 105950 [PushOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, version: 105946'11622260, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (2150198137 0 0) 0x187f4000 con 0x22fbc420 -6556> 2014-11-03 11:37:50.552869 7f28f72fe700 5 -- op tracker -- seq: 2955, time: 2014-11-03 11:37:50.541167, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, version: 105946'11622260, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6555> 2014-11-03 11:37:50.552900 7f28f72fe700 5 -- op tracker -- seq: 2955, time: 2014-11-03 11:37:50.541172, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, version: 105946'11622260, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6554> 2014-11-03 11:37:50.552923 7f28f72fe700 5 -- op tracker -- seq: 2955, time: 2014-11-03 11:37:50.552775, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, version: 105946'11622260, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6553> 2014-11-03 11:37:50.552972 7f28f72fe700 5 -- op tracker -- seq: 2955, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, version: 105946'11622260, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6552> 2014-11-03 11:37:50.553129 7f28fe818700 5 -- op tracker -- seq: 2955, time: 2014-11-03 11:37:50.553129, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, version: 105946'11622260, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6551> 2014-11-03 11:37:50.553389 7f28fe818700 5 -- op tracker -- seq: 2955, time: 2014-11-03 11:37:50.553388, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6, version: 105946'11622260, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c1ac7d8e/rbd_data.89cd895a911542.0000000000006681/head//6@105946'11622260, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6550> 2014-11-03 11:37:50.558926 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 48 ==== MOSDPGPush(6.1d3 105950 [PushOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6, version: 105946'9525541, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6@105946'9525541, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1303272700 0 0) 0x23296a00 con 0x22fe0000 -6549> 2014-11-03 11:37:50.558981 7f28f4fdb700 5 -- op tracker -- seq: 2956, time: 2014-11-03 11:37:50.546217, event: header_read, op: MOSDPGPush(6.1d3 105950 [PushOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6, version: 105946'9525541, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6@105946'9525541, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6548> 2014-11-03 11:37:50.559005 7f28f4fdb700 5 -- op tracker -- seq: 2956, time: 2014-11-03 11:37:50.546219, event: throttled, op: MOSDPGPush(6.1d3 105950 [PushOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6, version: 105946'9525541, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6@105946'9525541, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6547> 2014-11-03 11:37:50.559026 7f28f4fdb700 5 -- op tracker -- seq: 2956, time: 2014-11-03 11:37:50.558910, event: all_read, op: MOSDPGPush(6.1d3 105950 [PushOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6, version: 105946'9525541, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6@105946'9525541, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6546> 2014-11-03 11:37:50.559043 7f28f4fdb700 5 -- op tracker -- seq: 2956, time: 0.000000, event: dispatched, op: MOSDPGPush(6.1d3 105950 [PushOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6, version: 105946'9525541, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6@105946'9525541, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6545> 2014-11-03 11:37:50.559151 7f290101d700 5 -- op tracker -- seq: 2956, time: 2014-11-03 11:37:50.559151, event: reached_pg, op: MOSDPGPush(6.1d3 105950 [PushOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6, version: 105946'9525541, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6@105946'9525541, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6544> 2014-11-03 11:37:50.559210 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 49 ==== osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+17199 (192632920 0 3505555881) 0x25eb7000 con 0x22fe0000 -6543> 2014-11-03 11:37:50.559237 7f28f4fdb700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.559099, event: header_read, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6542> 2014-11-03 11:37:50.559253 7f28f4fdb700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.559101, event: throttled, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6541> 2014-11-03 11:37:50.559274 7f28f4fdb700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.559202, event: all_read, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6540> 2014-11-03 11:37:50.559284 7f28f4fdb700 5 -- op tracker -- seq: 2957, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6539> 2014-11-03 11:37:50.559406 7f290101d700 5 -- op tracker -- seq: 2956, time: 2014-11-03 11:37:50.559406, event: done, op: MOSDPGPush(6.1d3 105950 [PushOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6, version: 105946'9525541, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6@105946'9525541, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6538> 2014-11-03 11:37:50.559438 7f28fe818700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.559438, event: reached_pg, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6537> 2014-11-03 11:37:50.559488 7f28fe818700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.559488, event: started, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6536> 2014-11-03 11:37:50.559638 7f28fe818700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.559637, event: started, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6535> 2014-11-03 11:37:50.559683 7f28fe818700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.559683, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6534> 2014-11-03 11:37:50.572069 7f291450b700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.572069, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6533> 2014-11-03 11:37:50.572124 7f291450b700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.572124, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6532> 2014-11-03 11:37:50.572138 7f291450b700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.572137, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6531> 2014-11-03 11:37:50.572196 7f2911d06700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:50.572196, event: commit_sent, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6530> 2014-11-03 11:37:50.572212 7f2914d0c700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.572212, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6529> 2014-11-03 11:37:50.572266 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f8c0 con 0x22ff3580 -6528> 2014-11-03 11:37:50.572293 7f2914d0c700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.572293, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6527> 2014-11-03 11:37:50.572318 7f2911d06700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:50.572318, event: commit_sent, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6526> 2014-11-03 11:37:50.572341 7f2914d0c700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.572340, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6525> 2014-11-03 11:37:50.572372 7f2914d0c700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.572372, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6524> 2014-11-03 11:37:50.572399 7f2914d0c700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.572399, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6523> 2014-11-03 11:37:50.572427 7f2914d0c700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.572426, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6522> 2014-11-03 11:37:50.572454 7f2914d0c700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.572454, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6521> 2014-11-03 11:37:50.572479 7f2914d0c700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.572479, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6520> 2014-11-03 11:37:50.572479 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917fb80 con 0x22ff3580 -6519> 2014-11-03 11:37:50.572509 7f2914d0c700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.572509, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6518> 2014-11-03 11:37:50.572530 7f2914d0c700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.572530, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6517> 2014-11-03 11:37:50.572521 7f2911d06700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:50.572521, event: commit_sent, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6516> 2014-11-03 11:37:50.572576 7f2914d0c700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.572575, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6515> 2014-11-03 11:37:50.572602 7f2914d0c700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.572602, event: write_thread_in_journal_buffer, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6514> 2014-11-03 11:37:50.572609 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917eb00 con 0x22ff3580 -6513> 2014-11-03 11:37:50.574835 7f2914d0c700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.574835, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6512> 2014-11-03 11:37:50.624028 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.6 192.168.5.1:0/1181 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.623366) v2 ==== 47+0+0 (2010849447 0 0) 0x2355b000 con 0x242299a0 -6511> 2014-11-03 11:37:50.624081 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.6 192.168.5.1:0/1181 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.623366) v2 ==== 47+0+0 (2010849447 0 0) 0x2de29a00 con 0x24925c60 -6510> 2014-11-03 11:37:50.624096 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/1181 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.623366) v2 -- ?+0 0x2a7c4000 con 0x242299a0 -6509> 2014-11-03 11:37:50.624166 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/1181 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.623366) v2 -- ?+0 0x2a8d7800 con 0x24925c60 -6508> 2014-11-03 11:37:50.635948 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 108 ==== osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3629826260 0 3511520646) 0x2fe0c000 con 0x22fb76e0 -6507> 2014-11-03 11:37:50.635990 7f28edd69700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.635833, event: header_read, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6506> 2014-11-03 11:37:50.636005 7f28edd69700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.635839, event: throttled, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6505> 2014-11-03 11:37:50.636013 7f28edd69700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.635933, event: all_read, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6504> 2014-11-03 11:37:50.636021 7f28edd69700 5 -- op tracker -- seq: 2958, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6503> 2014-11-03 11:37:50.636112 7f28fd816700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.636112, event: reached_pg, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6502> 2014-11-03 11:37:50.636190 7f28fd816700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.636190, event: started, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6501> 2014-11-03 11:37:50.636405 7f28fd816700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.636404, event: started, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6500> 2014-11-03 11:37:50.636477 7f28fd816700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.636477, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6499> 2014-11-03 11:37:50.637635 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 109 ==== osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3460178595 0 3511520646) 0x2448d000 con 0x22fb76e0 -6498> 2014-11-03 11:37:50.637671 7f28edd69700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.637487, event: header_read, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6497> 2014-11-03 11:37:50.637684 7f28edd69700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.637489, event: throttled, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6496> 2014-11-03 11:37:50.637698 7f28edd69700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.637622, event: all_read, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6495> 2014-11-03 11:37:50.637706 7f28edd69700 5 -- op tracker -- seq: 2959, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6494> 2014-11-03 11:37:50.637762 7f290001b700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.637762, event: reached_pg, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6493> 2014-11-03 11:37:50.637790 7f290001b700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.637790, event: started, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6492> 2014-11-03 11:37:50.637879 7f290001b700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.637879, event: started, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6491> 2014-11-03 11:37:50.637908 7f290001b700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.637908, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6490> 2014-11-03 11:37:50.653112 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.102 192.168.5.8:0/21153 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.651940) v2 ==== 47+0+0 (3756941194 0 0) 0x2893de00 con 0x233cb160 -6489> 2014-11-03 11:37:50.653180 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.651940) v2 -- ?+0 0x2de29a00 con 0x233cb160 -6488> 2014-11-03 11:37:50.653259 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.102 192.168.5.8:0/21153 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.651940) v2 ==== 47+0+0 (3756941194 0 0) 0x23307800 con 0x233cb580 -6487> 2014-11-03 11:37:50.653322 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.651940) v2 -- ?+0 0x2355b000 con 0x233cb580 -6486> 2014-11-03 11:37:50.656203 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 89 ==== osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1499521031 0 3511520646) 0x26151c00 con 0x22fbc420 -6485> 2014-11-03 11:37:50.656248 7f28f72fe700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.656024, event: header_read, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6484> 2014-11-03 11:37:50.656279 7f28f72fe700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.656034, event: throttled, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6483> 2014-11-03 11:37:50.656291 7f28f72fe700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.656175, event: all_read, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6482> 2014-11-03 11:37:50.656305 7f28f72fe700 5 -- op tracker -- seq: 2960, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6481> 2014-11-03 11:37:50.656371 7f28ff81a700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.656370, event: reached_pg, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6480> 2014-11-03 11:37:50.656406 7f28ff81a700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.656406, event: started, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6479> 2014-11-03 11:37:50.656542 7f28ff81a700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.656542, event: started, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6478> 2014-11-03 11:37:50.656590 7f28ff81a700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.656590, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6477> 2014-11-03 11:37:50.661935 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 90 ==== osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (667710099 0 3511520646) 0x27812800 con 0x22fbc420 -6476> 2014-11-03 11:37:50.661981 7f28f72fe700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.661791, event: header_read, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6475> 2014-11-03 11:37:50.661999 7f28f72fe700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.661793, event: throttled, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6474> 2014-11-03 11:37:50.662011 7f28f72fe700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.661918, event: all_read, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6473> 2014-11-03 11:37:50.662043 7f28f72fe700 5 -- op tracker -- seq: 2961, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6472> 2014-11-03 11:37:50.662198 7f290201f700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.662197, event: reached_pg, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6471> 2014-11-03 11:37:50.662260 7f290201f700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.662260, event: started, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6470> 2014-11-03 11:37:50.662412 7f290201f700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.662412, event: started, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6469> 2014-11-03 11:37:50.662468 7f290201f700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.662467, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6468> 2014-11-03 11:37:50.674443 7f28f58e4700 1 -- 192.168.5.2:6803/7606 <== osd.65 192.168.5.5:6843/30042 37 ==== osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3892414088 0 3511520646) 0x3125b000 con 0x22fe02c0 -6467> 2014-11-03 11:37:50.674472 7f28f58e4700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.674359, event: header_read, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6466> 2014-11-03 11:37:50.674493 7f28f58e4700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.674360, event: throttled, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6465> 2014-11-03 11:37:50.674505 7f28f58e4700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.674432, event: all_read, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6464> 2014-11-03 11:37:50.674521 7f28f58e4700 5 -- op tracker -- seq: 2962, time: 0.000000, event: dispatched, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6463> 2014-11-03 11:37:50.674621 7f28ff019700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.674621, event: reached_pg, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6462> 2014-11-03 11:37:50.674655 7f28ff019700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.674655, event: started, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6461> 2014-11-03 11:37:50.674748 7f28ff019700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.674748, event: started, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6460> 2014-11-03 11:37:50.674776 7f28ff019700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.674776, event: commit_queued_for_journal_write, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6459> 2014-11-03 11:37:50.675232 7f28f58e4700 1 -- 192.168.5.2:6803/7606 <== osd.65 192.168.5.5:6843/30042 38 ==== osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2137815535 0 3511520646) 0x2cf7f000 con 0x22fe02c0 -6458> 2014-11-03 11:37:50.675259 7f28f58e4700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.675093, event: header_read, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6457> 2014-11-03 11:37:50.675275 7f28f58e4700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.675094, event: throttled, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6456> 2014-11-03 11:37:50.675287 7f28f58e4700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.675224, event: all_read, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6455> 2014-11-03 11:37:50.675301 7f28f58e4700 5 -- op tracker -- seq: 2963, time: 0.000000, event: dispatched, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6454> 2014-11-03 11:37:50.675344 7f290181e700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.675344, event: reached_pg, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6453> 2014-11-03 11:37:50.675372 7f290181e700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.675372, event: started, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6452> 2014-11-03 11:37:50.675471 7f290181e700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.675471, event: started, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6451> 2014-11-03 11:37:50.675506 7f290181e700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.675506, event: commit_queued_for_journal_write, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6450> 2014-11-03 11:37:50.679382 7f291450b700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.679382, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6449> 2014-11-03 11:37:50.679411 7f291450b700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.679411, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6448> 2014-11-03 11:37:50.679423 7f291450b700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.679423, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6447> 2014-11-03 11:37:50.679438 7f291450b700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.679438, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6446> 2014-11-03 11:37:50.679454 7f291450b700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.679454, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6445> 2014-11-03 11:37:50.679471 7f2911d06700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:50.679470, event: commit_sent, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6444> 2014-11-03 11:37:50.679482 7f291450b700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.679482, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6443> 2014-11-03 11:37:50.679554 7f291450b700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.679554, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6442> 2014-11-03 11:37:50.679552 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f340 con 0x22ff3580 -6441> 2014-11-03 11:37:50.679572 7f291450b700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.679572, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6440> 2014-11-03 11:37:50.679602 7f291450b700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.679601, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6439> 2014-11-03 11:37:50.679596 7f2911d06700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:50.679596, event: commit_sent, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6438> 2014-11-03 11:37:50.679637 7f291450b700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.679637, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6437> 2014-11-03 11:37:50.679648 7f291450b700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.679648, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6436> 2014-11-03 11:37:50.679677 7f291450b700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.679677, event: journaled_completion_queued, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6435> 2014-11-03 11:37:50.679698 7f291450b700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.679698, event: journaled_completion_queued, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6434> 2014-11-03 11:37:50.679757 7f2914d0c700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.679757, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6433> 2014-11-03 11:37:50.679780 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650eb80 con 0x22ff3580 -6432> 2014-11-03 11:37:50.679824 7f2914d0c700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.679824, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6431> 2014-11-03 11:37:50.679829 7f2911d06700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:50.679829, event: commit_sent, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6430> 2014-11-03 11:37:50.679854 7f2914d0c700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.679854, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6429> 2014-11-03 11:37:50.679899 7f2914d0c700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.679899, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6428> 2014-11-03 11:37:50.679921 7f2914d0c700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.679921, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6427> 2014-11-03 11:37:50.679947 7f2914d0c700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.679947, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6426> 2014-11-03 11:37:50.679969 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23de7000 con 0x22ff3580 -6425> 2014-11-03 11:37:50.680074 7f2911d06700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:50.680074, event: commit_sent, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6424> 2014-11-03 11:37:50.680260 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d423840 con 0x22ff3580 -6423> 2014-11-03 11:37:50.680326 7f2911d06700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:50.680326, event: commit_sent, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6422> 2014-11-03 11:37:50.680464 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d424b80 con 0x22ff3580 -6421> 2014-11-03 11:37:50.680531 7f2911d06700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:50.680531, event: commit_sent, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6420> 2014-11-03 11:37:50.680715 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x290592c0 con 0x22ff3580 -6419> 2014-11-03 11:37:50.680780 7f2911d06700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:50.680780, event: commit_sent, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6418> 2014-11-03 11:37:50.680988 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x29059b00 con 0x22ff3580 -6417> 2014-11-03 11:37:50.681054 7f2911d06700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:50.681054, event: commit_sent, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6416> 2014-11-03 11:37:50.681236 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa2688c0 con 0x22ff3580 -6415> 2014-11-03 11:37:50.681302 7f2911d06700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:50.681302, event: commit_sent, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6414> 2014-11-03 11:37:50.681516 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eadc0 con 0x22ff3580 -6413> 2014-11-03 11:37:50.681583 7f2911d06700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:50.681582, event: commit_sent, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6412> 2014-11-03 11:37:50.681745 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a98c0 con 0x22ff3580 -6411> 2014-11-03 11:37:50.681794 7f2911d06700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:50.681794, event: commit_sent, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6410> 2014-11-03 11:37:50.681896 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a9080 con 0x22ff3580 -6409> 2014-11-03 11:37:50.681945 7f2911d06700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:50.681945, event: commit_sent, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6408> 2014-11-03 11:37:50.681981 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a8dc0 con 0x2308b840 -6407> 2014-11-03 11:37:50.682036 7f2911d06700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:50.682035, event: commit_sent, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6406> 2014-11-03 11:37:50.682079 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- osd_sub_op_reply(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a8000 con 0x22fe0000 -6405> 2014-11-03 11:37:50.682743 7f291450b700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.682742, event: journaled_completion_queued, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6404> 2014-11-03 11:37:50.682777 7f291450b700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.682777, event: journaled_completion_queued, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6403> 2014-11-03 11:37:50.682792 7f291450b700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.682792, event: journaled_completion_queued, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6402> 2014-11-03 11:37:50.682818 7f291450b700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.682818, event: journaled_completion_queued, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6401> 2014-11-03 11:37:50.682818 7f2911d06700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:50.682818, event: commit_sent, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6400> 2014-11-03 11:37:50.682843 7f291450b700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.682843, event: journaled_completion_queued, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6399> 2014-11-03 11:37:50.682872 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a82c0 con 0x22fb76e0 -6398> 2014-11-03 11:37:50.682880 7f291450b700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.682880, event: journaled_completion_queued, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6397> 2014-11-03 11:37:50.682922 7f2911d06700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:50.682922, event: commit_sent, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6396> 2014-11-03 11:37:50.682950 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a9340 con 0x22fb76e0 -6395> 2014-11-03 11:37:50.683049 7f2911d06700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:50.683049, event: commit_sent, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6394> 2014-11-03 11:37:50.683082 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfbadc0 con 0x22fbc420 -6393> 2014-11-03 11:37:50.683113 7f2911d06700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:50.683113, event: commit_sent, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6392> 2014-11-03 11:37:50.683190 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfbbb80 con 0x22fbc420 -6391> 2014-11-03 11:37:50.683212 7f2911d06700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:50.683212, event: commit_sent, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6390> 2014-11-03 11:37:50.683228 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6843/30042 -- osd_sub_op_reply(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfba2c0 con 0x22fe02c0 -6389> 2014-11-03 11:37:50.683247 7f2911d06700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:50.683246, event: commit_sent, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6388> 2014-11-03 11:37:50.683328 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6843/30042 -- osd_sub_op_reply(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfba840 con 0x22fe02c0 -6387> 2014-11-03 11:37:50.705520 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.76 192.168.5.6:0/22493 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.705096) v2 ==== 47+0+0 (1263914819 0 0) 0x2470aa00 con 0x23b1c160 -6386> 2014-11-03 11:37:50.705576 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.705096) v2 -- ?+0 0x2893de00 con 0x23b1c160 -6385> 2014-11-03 11:37:50.705573 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.76 192.168.5.6:0/22493 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.705096) v2 ==== 47+0+0 (1263914819 0 0) 0x245ffe00 con 0x24229dc0 -6384> 2014-11-03 11:37:50.705731 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.705096) v2 -- ?+0 0x23307800 con 0x24229dc0 -6383> 2014-11-03 11:37:50.732503 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.94 192.168.5.7:0/10578 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.730655) v2 ==== 47+0+0 (1277872339 0 0) 0x9459a00 con 0x2327a6e0 -6382> 2014-11-03 11:37:50.732605 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/10578 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.730655) v2 -- ?+0 0x245ffe00 con 0x2327a6e0 -6381> 2014-11-03 11:37:50.732476 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.94 192.168.5.7:0/10578 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.730655) v2 ==== 47+0+0 (1277872339 0 0) 0x9459800 con 0x2327a420 -6380> 2014-11-03 11:37:50.732659 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/10578 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.730655) v2 -- ?+0 0x2470aa00 con 0x2327a420 -6379> 2014-11-03 11:37:50.735977 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.108 192.168.5.8:0/21567 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.734236) v2 ==== 47+0+0 (2400731722 0 0) 0x91ae600 con 0x2484a840 -6378> 2014-11-03 11:37:50.736020 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.734236) v2 -- ?+0 0x9459800 con 0x2484a840 -6377> 2014-11-03 11:37:50.736101 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.108 192.168.5.8:0/21567 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.734236) v2 ==== 47+0+0 (2400731722 0 0) 0x23ba6600 con 0x243212c0 -6376> 2014-11-03 11:37:50.736179 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.734236) v2 -- ?+0 0x9459a00 con 0x243212c0 -6375> 2014-11-03 11:37:50.736924 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 45 ==== osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1447823229 0 3511520646) 0x2cf80800 con 0x230a9b00 -6374> 2014-11-03 11:37:50.736970 7f28f038f700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.736815, event: header_read, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6373> 2014-11-03 11:37:50.736990 7f28f038f700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.736816, event: throttled, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6372> 2014-11-03 11:37:50.737009 7f28f038f700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.736909, event: all_read, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6371> 2014-11-03 11:37:50.737030 7f28f038f700 5 -- op tracker -- seq: 2964, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6370> 2014-11-03 11:37:50.737093 7f290101d700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.737093, event: reached_pg, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6369> 2014-11-03 11:37:50.737124 7f290101d700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.737124, event: started, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6368> 2014-11-03 11:37:50.737279 7f290101d700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.737279, event: started, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6367> 2014-11-03 11:37:50.737317 7f290101d700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.737316, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6366> 2014-11-03 11:37:50.737387 7f2914d0c700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.737387, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6365> 2014-11-03 11:37:50.738531 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 46 ==== osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3608666044 0 3511520646) 0x2cf7fc00 con 0x230a9b00 -6364> 2014-11-03 11:37:50.738563 7f28f038f700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.738446, event: header_read, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6363> 2014-11-03 11:37:50.738579 7f28f038f700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.738448, event: throttled, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6362> 2014-11-03 11:37:50.738592 7f28f038f700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.738517, event: all_read, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6361> 2014-11-03 11:37:50.738604 7f28f038f700 5 -- op tracker -- seq: 2965, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6360> 2014-11-03 11:37:50.738662 7f28fe818700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.738662, event: reached_pg, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6359> 2014-11-03 11:37:50.738690 7f28fe818700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.738690, event: started, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6358> 2014-11-03 11:37:50.738789 7f28fe818700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.738789, event: started, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6357> 2014-11-03 11:37:50.738817 7f28fe818700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.738817, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6356> 2014-11-03 11:37:50.739902 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 47 ==== osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (763397784 0 3511520646) 0x295a2000 con 0x230a9b00 -6355> 2014-11-03 11:37:50.739941 7f28f038f700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.739792, event: header_read, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6354> 2014-11-03 11:37:50.739958 7f28f038f700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.739793, event: throttled, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6353> 2014-11-03 11:37:50.739967 7f28f038f700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.739888, event: all_read, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6352> 2014-11-03 11:37:50.739974 7f28f038f700 5 -- op tracker -- seq: 2966, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6351> 2014-11-03 11:37:50.740022 7f290101d700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.740022, event: reached_pg, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6350> 2014-11-03 11:37:50.740063 7f290101d700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.740063, event: started, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6349> 2014-11-03 11:37:50.740192 7f290101d700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.740191, event: started, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6348> 2014-11-03 11:37:50.740253 7f290101d700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.740253, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6347> 2014-11-03 11:37:50.742029 7f291450b700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.742029, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6346> 2014-11-03 11:37:50.742085 7f2911d06700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:50.742085, event: commit_sent, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6345> 2014-11-03 11:37:50.742094 7f2914d0c700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.742094, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6344> 2014-11-03 11:37:50.742133 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfbb600 con 0x230a9b00 -6343> 2014-11-03 11:37:50.742140 7f2914d0c700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.742140, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6342> 2014-11-03 11:37:50.742503 7f291450b700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.742503, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6341> 2014-11-03 11:37:50.742534 7f291450b700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.742534, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6340> 2014-11-03 11:37:50.742638 7f2911d06700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:50.742638, event: commit_sent, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6339> 2014-11-03 11:37:50.742701 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfbb8c0 con 0x230a9b00 -6338> 2014-11-03 11:37:50.742778 7f2911d06700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:50.742777, event: commit_sent, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6337> 2014-11-03 11:37:50.742909 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfbb340 con 0x230a9b00 -6336> 2014-11-03 11:37:50.799069 7f28ea632700 1 -- 192.168.5.2:6803/7606 <== osd.58 192.168.5.5:6813/24563 26 ==== osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1549917516 0 3511520646) 0x2de2a000 con 0x231e5dc0 -6335> 2014-11-03 11:37:50.799106 7f28ea632700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.798958, event: header_read, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6334> 2014-11-03 11:37:50.799122 7f28ea632700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.798961, event: throttled, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6333> 2014-11-03 11:37:50.799131 7f28ea632700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799055, event: all_read, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6332> 2014-11-03 11:37:50.799139 7f28ea632700 5 -- op tracker -- seq: 2967, time: 0.000000, event: dispatched, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6331> 2014-11-03 11:37:50.799196 7f28fe017700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799195, event: reached_pg, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6330> 2014-11-03 11:37:50.799234 7f28fe017700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799234, event: started, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6329> 2014-11-03 11:37:50.799353 7f28fe017700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799353, event: started, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6328> 2014-11-03 11:37:50.799390 7f28fe017700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799390, event: commit_queued_for_journal_write, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6327> 2014-11-03 11:37:50.799461 7f2914d0c700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799461, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6326> 2014-11-03 11:37:50.799638 7f291450b700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799638, event: journaled_completion_queued, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6325> 2014-11-03 11:37:50.799808 7f2911d06700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:50.799808, event: commit_sent, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6324> 2014-11-03 11:37:50.799891 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6813/24563 -- osd_sub_op_reply(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfbb080 con 0x231e5dc0 -6323> 2014-11-03 11:37:50.807268 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.121 192.168.5.9:0/12205 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.805723) v2 ==== 47+0+0 (2906801556 0 0) 0x23cbe800 con 0x24925dc0 -6322> 2014-11-03 11:37:50.807314 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/12205 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.805723) v2 -- ?+0 0x91ae600 con 0x24925dc0 -6321> 2014-11-03 11:37:50.807351 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.121 192.168.5.9:0/12205 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.805723) v2 ==== 47+0+0 (2906801556 0 0) 0xaf2ee00 con 0x24229b00 -6320> 2014-11-03 11:37:50.807390 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/12205 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.805723) v2 -- ?+0 0x23ba6600 con 0x24229b00 -6319> 2014-11-03 11:37:50.815569 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 74 ==== osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1324012889 0 3511520646) 0x24661c00 con 0x230c2000 -6318> 2014-11-03 11:37:50.815627 7f28ebc48700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.815413, event: header_read, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6317> 2014-11-03 11:37:50.815658 7f28ebc48700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.815417, event: throttled, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6316> 2014-11-03 11:37:50.815673 7f28ebc48700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.815548, event: all_read, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6315> 2014-11-03 11:37:50.815686 7f28ebc48700 5 -- op tracker -- seq: 2968, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6314> 2014-11-03 11:37:50.815788 7f28fe818700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.815788, event: reached_pg, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6313> 2014-11-03 11:37:50.815831 7f28fe818700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.815831, event: started, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6312> 2014-11-03 11:37:50.815997 7f28fe818700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.815997, event: started, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6311> 2014-11-03 11:37:50.816042 7f28fe818700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.816042, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6310> 2014-11-03 11:37:50.816163 7f2914d0c700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.816163, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6309> 2014-11-03 11:37:50.818102 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 75 ==== osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3160985492 0 3511520646) 0x2d3a8800 con 0x230c2000 -6308> 2014-11-03 11:37:50.818192 7f28ebc48700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.817952, event: header_read, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6307> 2014-11-03 11:37:50.818234 7f28ebc48700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.817955, event: throttled, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6306> 2014-11-03 11:37:50.818256 7f28ebc48700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.818075, event: all_read, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6305> 2014-11-03 11:37:50.818276 7f28ebc48700 5 -- op tracker -- seq: 2969, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6304> 2014-11-03 11:37:50.818358 7f290101d700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.818358, event: reached_pg, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6303> 2014-11-03 11:37:50.818406 7f290101d700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.818406, event: started, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6302> 2014-11-03 11:37:50.818544 7f290101d700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.818543, event: started, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6301> 2014-11-03 11:37:50.818582 7f290101d700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.818582, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6300> 2014-11-03 11:37:50.820110 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 76 ==== osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3044138883 0 3511520646) 0x26151000 con 0x230c2000 -6299> 2014-11-03 11:37:50.820183 7f28ebc48700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.819938, event: header_read, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6298> 2014-11-03 11:37:50.820225 7f28ebc48700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.819941, event: throttled, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6297> 2014-11-03 11:37:50.820243 7f28ebc48700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.820089, event: all_read, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6296> 2014-11-03 11:37:50.820262 7f28ebc48700 5 -- op tracker -- seq: 2970, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6295> 2014-11-03 11:37:50.820354 7f28fe818700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.820353, event: reached_pg, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6294> 2014-11-03 11:37:50.820408 7f28fe818700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.820408, event: started, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6293> 2014-11-03 11:37:50.820550 7f28fe818700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.820550, event: started, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6292> 2014-11-03 11:37:50.820597 7f28fe818700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.820596, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6291> 2014-11-03 11:37:50.829163 7f291450b700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.829163, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6290> 2014-11-03 11:37:50.829289 7f2911d06700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:50.829288, event: commit_sent, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6289> 2014-11-03 11:37:50.829339 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2aa6c8c0 con 0x230c2000 -6288> 2014-11-03 11:37:50.829335 7f2914d0c700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.829335, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6287> 2014-11-03 11:37:50.829383 7f2914d0c700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.829382, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6286> 2014-11-03 11:37:50.836095 7f291450b700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.836094, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6285> 2014-11-03 11:37:50.836144 7f291450b700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.836144, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6284> 2014-11-03 11:37:50.836199 7f2911d06700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:50.836198, event: commit_sent, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6283> 2014-11-03 11:37:50.836266 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2aa6c340 con 0x230c2000 -6282> 2014-11-03 11:37:50.836324 7f2911d06700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:50.836324, event: commit_sent, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6281> 2014-11-03 11:37:50.836474 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2aa6b840 con 0x230c2000 -6280> 2014-11-03 11:37:50.839088 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- MOSDPGPushReply(6.1d3 105950 [PushReplyOp(842db1d3/rbd_data.4eeda241ca92c8.0000000000000579/head//6),PushReplyOp(b36a71d3/rbd_data.4ee99a858b72c.0000000000002029/head//6)]) v2 -- ?+0 0x2340da00 con 0x22fe0000 -6279> 2014-11-03 11:37:50.840706 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.31 192.168.5.3:0/20618 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.839756) v2 ==== 47+0+0 (2676028625 0 0) 0x2cf94000 con 0x24925840 -6278> 2014-11-03 11:37:50.840793 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.839756) v2 -- ?+0 0x23cbe800 con 0x24925840 -6277> 2014-11-03 11:37:50.841313 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.31 192.168.5.3:0/20618 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.839756) v2 ==== 47+0+0 (2676028625 0 0) 0x2355bc00 con 0x24229580 -6276> 2014-11-03 11:37:50.841381 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.839756) v2 -- ?+0 0xaf2ee00 con 0x24229580 -6275> 2014-11-03 11:37:50.864901 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.34 192.168.5.3:0/2034 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.864078) v2 ==== 47+0+0 (855056943 0 0) 0xaf2ee00 con 0x238b8b00 -6274> 2014-11-03 11:37:50.864968 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/2034 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.864078) v2 -- ?+0 0x2355bc00 con 0x238b8b00 -6273> 2014-11-03 11:37:50.865035 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.34 192.168.5.3:0/2034 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.864078) v2 ==== 47+0+0 (855056943 0 0) 0x1adc4a00 con 0x239f89a0 -6272> 2014-11-03 11:37:50.865065 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/2034 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.864078) v2 -- ?+0 0x2cf94000 con 0x239f89a0 -6271> 2014-11-03 11:37:50.896126 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.7870073 192.168.1.72:0/2026476 7 ==== ping magic: 0 v1 ==== 0+0+0 (0 0 0) 0x24500700 con 0x2311c2c0 -6270> 2014-11-03 11:37:50.896307 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.35 192.168.5.3:0/6455 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.895364) v2 ==== 47+0+0 (1976625137 0 0) 0x1106c000 con 0x2484a6e0 -6269> 2014-11-03 11:37:50.896362 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.895364) v2 -- ?+0 0x1adc4a00 con 0x2484a6e0 -6268> 2014-11-03 11:37:50.896362 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.35 192.168.5.3:0/6455 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.895364) v2 ==== 47+0+0 (1976625137 0 0) 0x6352e00 con 0x24321160 -6267> 2014-11-03 11:37:50.896404 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.895364) v2 -- ?+0 0xaf2ee00 con 0x24321160 -6266> 2014-11-03 11:37:50.903052 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.83 192.168.5.6:0/4005 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.902185) v2 ==== 47+0+0 (2088239251 0 0) 0x4d90c00 con 0x2445c2c0 -6265> 2014-11-03 11:37:50.903111 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.902185) v2 -- ?+0 0x1106c000 con 0x2445c2c0 -6264> 2014-11-03 11:37:50.903634 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.83 192.168.5.6:0/4005 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.902185) v2 ==== 47+0+0 (2088239251 0 0) 0x13d27800 con 0x23ea4420 -6263> 2014-11-03 11:37:50.903688 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.902185) v2 -- ?+0 0x6352e00 con 0x23ea4420 -6262> 2014-11-03 11:37:50.911633 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.29 192.168.5.3:0/1192 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.910374) v2 ==== 47+0+0 (3038306132 0 0) 0x27b01200 con 0x23335000 -6261> 2014-11-03 11:37:50.911691 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/1192 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.910374) v2 -- ?+0 0x4d90c00 con 0x23335000 -6260> 2014-11-03 11:37:50.911722 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.29 192.168.5.3:0/1192 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.910374) v2 ==== 47+0+0 (3038306132 0 0) 0x8714400 con 0x234ed000 -6259> 2014-11-03 11:37:50.911772 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/1192 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.910374) v2 -- ?+0 0x13d27800 con 0x234ed000 -6258> 2014-11-03 11:37:50.913699 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.53 192.168.5.4:0/5692 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.911675) v2 ==== 47+0+0 (3570124473 0 0) 0x2796ea00 con 0x23b1c9a0 -6257> 2014-11-03 11:37:50.913741 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/5692 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.911675) v2 -- ?+0 0x27b01200 con 0x23b1c9a0 -6256> 2014-11-03 11:37:50.913771 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.53 192.168.5.4:0/5692 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.911675) v2 ==== 47+0+0 (3570124473 0 0) 0x6352e00 con 0x242c0b00 -6255> 2014-11-03 11:37:50.913817 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/5692 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.911675) v2 -- ?+0 0x8714400 con 0x242c0b00 -6254> 2014-11-03 11:37:50.960951 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 164 ==== osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (756025185 0 3511520646) 0x2d3a9400 con 0x22fefdc0 -6253> 2014-11-03 11:37:50.960996 7f28f36c2700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.960823, event: header_read, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6252> 2014-11-03 11:37:50.961015 7f28f36c2700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.960827, event: throttled, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6251> 2014-11-03 11:37:50.961028 7f28f36c2700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.960936, event: all_read, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6250> 2014-11-03 11:37:50.961039 7f28f36c2700 5 -- op tracker -- seq: 2971, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6249> 2014-11-03 11:37:50.961118 7f290101d700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.961118, event: reached_pg, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6248> 2014-11-03 11:37:50.961148 7f290101d700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.961148, event: started, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6247> 2014-11-03 11:37:50.961233 7f290101d700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.961233, event: started, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6246> 2014-11-03 11:37:50.961266 7f290101d700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.961266, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6245> 2014-11-03 11:37:50.961413 7f2914d0c700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.961412, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6244> 2014-11-03 11:37:50.973473 7f291450b700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.973473, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6243> 2014-11-03 11:37:50.973588 7f2911d06700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:50.973588, event: commit_sent, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6242> 2014-11-03 11:37:50.973672 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2aa6c080 con 0x22fefdc0 -6241> 2014-11-03 11:37:50.975706 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.111 192.168.5.8:0/32649 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.974766) v2 ==== 47+0+0 (579259258 0 0) 0x243ff600 con 0x23b1c000 -6240> 2014-11-03 11:37:50.975752 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.974766) v2 -- ?+0 0x2796ea00 con 0x23b1c000 -6239> 2014-11-03 11:37:50.975763 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.111 192.168.5.8:0/32649 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:50.974766) v2 ==== 47+0+0 (579259258 0 0) 0x187f4400 con 0x24229c60 -6238> 2014-11-03 11:37:50.975826 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:50.974766) v2 -- ?+0 0x6352e00 con 0x24229c60 -6237> 2014-11-03 11:37:51.022363 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.40 192.168.5.3:0/25675 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.021293) v2 ==== 47+0+0 (772473696 0 0) 0x2796ce00 con 0x23b1c580 -6236> 2014-11-03 11:37:51.022386 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.40 192.168.5.3:0/25675 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.021293) v2 ==== 47+0+0 (772473696 0 0) 0x23296a00 con 0x242c0420 -6235> 2014-11-03 11:37:51.022478 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/25675 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.021293) v2 -- ?+0 0x243ff600 con 0x23b1c580 -6234> 2014-11-03 11:37:51.022529 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/25675 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.021293) v2 -- ?+0 0x187f4400 con 0x242c0420 -6233> 2014-11-03 11:37:51.050387 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.45 192.168.5.4:0/1781 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.048691) v2 ==== 47+0+0 (699635557 0 0) 0x1106cc00 con 0x2445c160 -6232> 2014-11-03 11:37:51.050441 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.048691) v2 -- ?+0 0x2796ce00 con 0x2445c160 -6231> 2014-11-03 11:37:51.050427 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.45 192.168.5.4:0/1781 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.048691) v2 ==== 47+0+0 (699635557 0 0) 0xde23e00 con 0x23ea42c0 -6230> 2014-11-03 11:37:51.050506 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.048691) v2 -- ?+0 0x23296a00 con 0x23ea42c0 -6229> 2014-11-03 11:37:51.062347 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.79 192.168.5.6:0/12953 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.060711) v2 ==== 47+0+0 (174270101 0 0) 0x2324be00 con 0x23b1c420 -6228> 2014-11-03 11:37:51.062403 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/12953 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.060711) v2 -- ?+0 0x1106cc00 con 0x23b1c420 -6227> 2014-11-03 11:37:51.062409 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.79 192.168.5.6:0/12953 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.060711) v2 ==== 47+0+0 (174270101 0 0) 0x23576200 con 0x242052c0 -6226> 2014-11-03 11:37:51.062464 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/12953 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.060711) v2 -- ?+0 0xde23e00 con 0x242052c0 -6225> 2014-11-03 11:37:51.077342 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.61 192.168.5.5:0/12481 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.076271) v2 ==== 47+0+0 (753883258 0 0) 0x27b01000 con 0x2445c840 -6224> 2014-11-03 11:37:51.077398 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/12481 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.076271) v2 -- ?+0 0x2324be00 con 0x2445c840 -6223> 2014-11-03 11:37:51.077499 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.61 192.168.5.5:0/12481 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.076271) v2 ==== 47+0+0 (753883258 0 0) 0x23a9ea00 con 0x23ea49a0 -6222> 2014-11-03 11:37:51.077546 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/12481 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.076271) v2 -- ?+0 0x23576200 con 0x23ea49a0 -6221> 2014-11-03 11:37:51.077945 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.75 192.168.5.6:0/12612 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.076225) v2 ==== 47+0+0 (456559649 0 0) 0x23354a00 con 0x239f8420 -6220> 2014-11-03 11:37:51.078023 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/12612 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.076225) v2 -- ?+0 0x27b01000 con 0x239f8420 -6219> 2014-11-03 11:37:51.078033 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.75 192.168.5.6:0/12612 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.076225) v2 ==== 47+0+0 (456559649 0 0) 0x2355ba00 con 0x238b8580 -6218> 2014-11-03 11:37:51.078144 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/12612 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.076225) v2 -- ?+0 0x23a9ea00 con 0x238b8580 -6217> 2014-11-03 11:37:51.157205 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.102 192.168.5.8:0/21153 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.156059) v2 ==== 47+0+0 (1669968563 0 0) 0x253e5000 con 0x233cb160 -6216> 2014-11-03 11:37:51.157336 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.156059) v2 -- ?+0 0x23354a00 con 0x233cb160 -6215> 2014-11-03 11:37:51.157231 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.102 192.168.5.8:0/21153 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.156059) v2 ==== 47+0+0 (1669968563 0 0) 0x2d361200 con 0x233cb580 -6214> 2014-11-03 11:37:51.157424 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.156059) v2 -- ?+0 0x2355ba00 con 0x233cb580 -6213> 2014-11-03 11:37:51.182957 7f28c4099700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x248b7b80 sd=118 :6802 s=0 pgs=0 cs=0 l=0 c=0xa26c420).accept sd=118 192.168.5.201:58661/0 -6212> 2014-11-03 11:37:51.183888 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450841 192.168.5.201:0/1029036 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "24.7d7"}) v1 ==== 80+0+0 (3756934197 0 0) 0x2d361c00 con 0xa26c420 -6211> 2014-11-03 11:37:51.184610 7f28fb812700 0 osd.21 105950 do_command r=0 -6210> 2014-11-03 11:37:51.184627 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029036 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x17c608c0 con 0xa26c420 -6209> 2014-11-03 11:37:51.196883 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450841 192.168.5.201:0/1029036 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "24.7d7"}) v1 ==== 74+0+0 (774687807 0 0) 0x2893de00 con 0xa26c420 -6208> 2014-11-03 11:37:51.198017 7f28fb812700 0 osd.21 105950 do_command r=0 -6207> 2014-11-03 11:37:51.198047 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029036 -- command_reply(tid 2: 0 ) v1 -- ?+13413 0x17c60000 con 0xa26c420 -6206> 2014-11-03 11:37:51.199599 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- MOSDPGPushReply(6.1d3 105950 [PushReplyOp(21a651d3/rbd_data.3ea57474ca1fcf.00000000000015f7/head//6),PushReplyOp(4f0ed1d3/rbd_data.7a4ad22ae8944a.0000000000000818/head//6)]) v2 -- ?+0 0x29fcd000 con 0x22fe0000 -6205> 2014-11-03 11:37:51.200294 7f2912507700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:51.200294, event: sub_op_applied, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6204> 2014-11-03 11:37:51.200441 7f2912507700 5 -- op tracker -- seq: 2932, time: 2014-11-03 11:37:51.200441, event: done, op: osd_sub_op(client.7868162.0:44218336 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870430 snapset=0=[]:[] snapc=0=[]) -6203> 2014-11-03 11:37:51.200646 7f2912507700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:51.200646, event: sub_op_applied, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6202> 2014-11-03 11:37:51.200717 7f2912507700 5 -- op tracker -- seq: 2933, time: 2014-11-03 11:37:51.200717, event: done, op: osd_sub_op(client.7868162.0:44218337 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870431 snapset=0=[]:[] snapc=0=[]) -6201> 2014-11-03 11:37:51.201096 7f2912507700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:51.201095, event: sub_op_applied, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6200> 2014-11-03 11:37:51.201152 7f2912507700 5 -- op tracker -- seq: 2934, time: 2014-11-03 11:37:51.201152, event: done, op: osd_sub_op(client.7868162.0:44218338 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870432 snapset=0=[]:[] snapc=0=[]) -6199> 2014-11-03 11:37:51.201652 7f2912507700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:51.201652, event: sub_op_applied, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6198> 2014-11-03 11:37:51.201725 7f2912507700 5 -- op tracker -- seq: 2935, time: 2014-11-03 11:37:51.201725, event: done, op: osd_sub_op(client.7868162.0:44218352 6.690 ae3bbe90/rbd_data.205eab46ed32b.00000000000031de/head//6 [] v 105950'28900011 snapset=0=[]:[] snapc=0=[]) -6197> 2014-11-03 11:37:51.201924 7f2912507700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:51.201924, event: sub_op_applied, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6196> 2014-11-03 11:37:51.202016 7f2912507700 5 -- op tracker -- seq: 2936, time: 2014-11-03 11:37:51.202015, event: done, op: osd_sub_op(client.7868162.0:44218339 6.456 8e598c56/rbd_data.205eab46ed32b.0000000000001917/head//6 [] v 105950'31870433 snapset=0=[]:[] snapc=0=[]) -6195> 2014-11-03 11:37:51.202281 7f2912507700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:51.202281, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6194> 2014-11-03 11:37:51.202374 7f2912507700 5 -- op tracker -- seq: 2937, time: 2014-11-03 11:37:51.202374, event: done, op: osd_sub_op(client.7914641.0:33713478 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649537 snapset=0=[]:[] snapc=0=[]) -6193> 2014-11-03 11:37:51.202687 7f2912507700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:51.202687, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6192> 2014-11-03 11:37:51.202755 7f2912507700 5 -- op tracker -- seq: 2938, time: 2014-11-03 11:37:51.202755, event: done, op: osd_sub_op(client.7914641.0:33713479 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649538 snapset=0=[]:[] snapc=0=[]) -6191> 2014-11-03 11:37:51.203014 7f2912507700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:51.203014, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6190> 2014-11-03 11:37:51.203078 7f2912507700 5 -- op tracker -- seq: 2939, time: 2014-11-03 11:37:51.203078, event: done, op: osd_sub_op(client.7914641.0:33713480 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649539 snapset=0=[]:[] snapc=0=[]) -6189> 2014-11-03 11:37:51.203357 7f2912507700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:51.203356, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6188> 2014-11-03 11:37:51.203455 7f2912507700 5 -- op tracker -- seq: 2940, time: 2014-11-03 11:37:51.203455, event: done, op: osd_sub_op(client.7914641.0:33713481 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649540 snapset=0=[]:[] snapc=0=[]) -6187> 2014-11-03 11:37:51.203533 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029036 pipe(0x248b7b80 sd=118 :6802 s=2 pgs=2 cs=1 l=1 c=0xa26c420).reader couldn't read tag, (0) Success -6186> 2014-11-03 11:37:51.203640 7f28c4099700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029036 pipe(0x248b7b80 sd=118 :6802 s=2 pgs=2 cs=1 l=1 c=0xa26c420).fault (0) Success -6185> 2014-11-03 11:37:51.203688 7f2912507700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:51.203688, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6184> 2014-11-03 11:37:51.203751 7f2912507700 5 -- op tracker -- seq: 2941, time: 2014-11-03 11:37:51.203751, event: done, op: osd_sub_op(client.7914641.0:33713482 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649541 snapset=0=[]:[] snapc=0=[]) -6183> 2014-11-03 11:37:51.203897 7f290c834700 1 osd.21 105950 ms_handle_reset con 0xa26c420 session 0x5fee280 -6182> 2014-11-03 11:37:51.204004 7f2912507700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:51.204004, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6181> 2014-11-03 11:37:51.204079 7f2912507700 5 -- op tracker -- seq: 2942, time: 2014-11-03 11:37:51.204079, event: done, op: osd_sub_op(client.7914641.0:33713483 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649542 snapset=0=[]:[] snapc=0=[]) -6180> 2014-11-03 11:37:51.204335 7f2912507700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:51.204334, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6179> 2014-11-03 11:37:51.204392 7f2912507700 5 -- op tracker -- seq: 2943, time: 2014-11-03 11:37:51.204392, event: done, op: osd_sub_op(client.7914641.0:33713484 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649543 snapset=0=[]:[] snapc=0=[]) -6178> 2014-11-03 11:37:51.204646 7f2912507700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:51.204646, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6177> 2014-11-03 11:37:51.204697 7f2912507700 5 -- op tracker -- seq: 2944, time: 2014-11-03 11:37:51.204697, event: done, op: osd_sub_op(client.7914641.0:33713485 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649544 snapset=0=[]:[] snapc=0=[]) -6176> 2014-11-03 11:37:51.204969 7f2912507700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:51.204969, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6175> 2014-11-03 11:37:51.205011 7f2912507700 5 -- op tracker -- seq: 2945, time: 2014-11-03 11:37:51.205011, event: done, op: osd_sub_op(client.7914641.0:33713486 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649545 snapset=0=[]:[] snapc=0=[]) -6174> 2014-11-03 11:37:51.205166 7f2912507700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:51.205166, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6173> 2014-11-03 11:37:51.205205 7f2912507700 5 -- op tracker -- seq: 2946, time: 2014-11-03 11:37:51.205205, event: done, op: osd_sub_op(client.7914641.0:33713487 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649546 snapset=0=[]:[] snapc=0=[]) -6172> 2014-11-03 11:37:51.205376 7f2912507700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:51.205376, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6171> 2014-11-03 11:37:51.205428 7f2912507700 5 -- op tracker -- seq: 2947, time: 2014-11-03 11:37:51.205428, event: done, op: osd_sub_op(client.7914641.0:33713488 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649547 snapset=0=[]:[] snapc=0=[]) -6170> 2014-11-03 11:37:51.205589 7f2912507700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:51.205588, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6169> 2014-11-03 11:37:51.205627 7f2912507700 5 -- op tracker -- seq: 2948, time: 2014-11-03 11:37:51.205627, event: done, op: osd_sub_op(client.7914641.0:33713489 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649548 snapset=0=[]:[] snapc=0=[]) -6168> 2014-11-03 11:37:51.205792 7f2912507700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:51.205792, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6167> 2014-11-03 11:37:51.205829 7f2912507700 5 -- op tracker -- seq: 2949, time: 2014-11-03 11:37:51.205829, event: done, op: osd_sub_op(client.7914641.0:33713490 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649549 snapset=0=[]:[] snapc=0=[]) -6166> 2014-11-03 11:37:51.206002 7f2912507700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:51.206002, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6165> 2014-11-03 11:37:51.206046 7f2912507700 5 -- op tracker -- seq: 2950, time: 2014-11-03 11:37:51.206046, event: done, op: osd_sub_op(client.7914641.0:33713491 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649550 snapset=0=[]:[] snapc=0=[]) -6164> 2014-11-03 11:37:51.206291 7f2912507700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:51.206291, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6163> 2014-11-03 11:37:51.206341 7f2912507700 5 -- op tracker -- seq: 2951, time: 2014-11-03 11:37:51.206340, event: done, op: osd_sub_op(client.7914641.0:33713492 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649551 snapset=0=[]:[] snapc=0=[]) -6162> 2014-11-03 11:37:51.206514 7f2912507700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:51.206514, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6161> 2014-11-03 11:37:51.206554 7f2912507700 5 -- op tracker -- seq: 2952, time: 2014-11-03 11:37:51.206553, event: done, op: osd_sub_op(client.7914641.0:33713493 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649552 snapset=0=[]:[] snapc=0=[]) -6160> 2014-11-03 11:37:51.206722 7f2912507700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:51.206722, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6159> 2014-11-03 11:37:51.206767 7f2912507700 5 -- op tracker -- seq: 2953, time: 2014-11-03 11:37:51.206767, event: done, op: osd_sub_op(client.7914641.0:33713494 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649553 snapset=0=[]:[] snapc=0=[]) -6158> 2014-11-03 11:37:51.206941 7f2912507700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:51.206941, event: sub_op_applied, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6157> 2014-11-03 11:37:51.206985 7f2912507700 5 -- op tracker -- seq: 2954, time: 2014-11-03 11:37:51.206984, event: done, op: osd_sub_op(client.11363859.0:247724 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000602 snapset=0=[]:[] snapc=0=[]) -6156> 2014-11-03 11:37:51.212071 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.118 192.168.5.9:0/23392 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.210957) v2 ==== 47+0+0 (2294323219 0 0) 0x234d5400 con 0x23c15000 -6155> 2014-11-03 11:37:51.212128 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/23392 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.210957) v2 -- ?+0 0x253e5000 con 0x23c15000 -6154> 2014-11-03 11:37:51.214338 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.118 192.168.5.9:0/23392 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.210957) v2 ==== 47+0+0 (2294323219 0 0) 0x247cde00 con 0x2469ec60 -6153> 2014-11-03 11:37:51.214447 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/23392 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.210957) v2 -- ?+0 0x2d361200 con 0x2469ec60 -6152> 2014-11-03 11:37:51.220833 7f2918d14700 5 osd.21 105950 tick -6151> 2014-11-03 11:37:51.226456 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- MOSDPGPushReply(6.1d3 105950 [PushReplyOp(96f011d3/rbd_data.89cd895a911542.0000000000008a5d/head//6)]) v2 -- ?+0 0x286bb200 con 0x22fe0000 -6150> 2014-11-03 11:37:51.230586 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x261a4800 con 0x22fbc420 -6149> 2014-11-03 11:37:51.231049 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- MOSDPGPushReply(6.1d3 105950 [PushReplyOp(c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6)]) v2 -- ?+0 0x2318aa00 con 0x22fe0000 -6148> 2014-11-03 11:37:51.231392 7f2912507700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:51.231391, event: sub_op_applied, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6147> 2014-11-03 11:37:51.231442 7f2912507700 5 -- op tracker -- seq: 2958, time: 2014-11-03 11:37:51.231442, event: done, op: osd_sub_op(client.7867870.0:20391786 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187821 snapset=0=[]:[] snapc=0=[]) -6146> 2014-11-03 11:37:51.231605 7f2912507700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:51.231605, event: sub_op_applied, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6145> 2014-11-03 11:37:51.231653 7f2912507700 5 -- op tracker -- seq: 2959, time: 2014-11-03 11:37:51.231653, event: done, op: osd_sub_op(client.7867870.0:20391787 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187822 snapset=0=[]:[] snapc=0=[]) -6144> 2014-11-03 11:37:51.231686 7f2912507700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:51.231686, event: sub_op_applied, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6143> 2014-11-03 11:37:51.231714 7f2912507700 5 -- op tracker -- seq: 2957, time: 2014-11-03 11:37:51.231714, event: done, op: osd_sub_op(client.7914290.0:2740627 6.1d3 b8be71d3/rbd_data.78c317aa34e0.0000000000000510/head//6 [] v 105950'9525546 snapset=0=[]:[] snapc=0=[]) -6142> 2014-11-03 11:37:51.231754 7f2912507700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:51.231754, event: sub_op_applied, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6141> 2014-11-03 11:37:51.231783 7f2912507700 5 -- op tracker -- seq: 2960, time: 2014-11-03 11:37:51.231783, event: done, op: osd_sub_op(client.7912580.0:17282986 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944774 snapset=0=[]:[] snapc=0=[]) -6140> 2014-11-03 11:37:51.231991 7f2912507700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:51.231991, event: sub_op_applied, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6139> 2014-11-03 11:37:51.232038 7f2912507700 5 -- op tracker -- seq: 2962, time: 2014-11-03 11:37:51.232038, event: done, op: osd_sub_op(client.8709331.0:2167934 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412483 snapset=0=[]:[] snapc=0=[]) -6138> 2014-11-03 11:37:51.232071 7f2912507700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:51.232071, event: sub_op_applied, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6137> 2014-11-03 11:37:51.232105 7f2912507700 5 -- op tracker -- seq: 2961, time: 2014-11-03 11:37:51.232105, event: done, op: osd_sub_op(client.7912580.0:17282987 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944775 snapset=0=[]:[] snapc=0=[]) -6136> 2014-11-03 11:37:51.232211 7f2912507700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:51.232210, event: sub_op_applied, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6135> 2014-11-03 11:37:51.232257 7f2912507700 5 -- op tracker -- seq: 2963, time: 2014-11-03 11:37:51.232256, event: done, op: osd_sub_op(client.8709331.0:2167935 6.425 d8943425/rbd_data.7986317d92ecf2.0000000000005200/head//6 [] v 105950'12412484 snapset=0=[]:[] snapc=0=[]) -6134> 2014-11-03 11:37:51.232297 7f2912507700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:51.232297, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6133> 2014-11-03 11:37:51.232329 7f2912507700 5 -- op tracker -- seq: 2964, time: 2014-11-03 11:37:51.232329, event: done, op: osd_sub_op(client.7869116.0:21661153 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850148 snapset=0=[]:[] snapc=0=[]) -6132> 2014-11-03 11:37:51.232372 7f2912507700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:51.232371, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6131> 2014-11-03 11:37:51.232394 7f2912507700 5 -- op tracker -- seq: 2965, time: 2014-11-03 11:37:51.232394, event: done, op: osd_sub_op(client.7869116.0:21661154 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850149 snapset=0=[]:[] snapc=0=[]) -6130> 2014-11-03 11:37:51.232573 7f2912507700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:51.232572, event: sub_op_applied, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6129> 2014-11-03 11:37:51.232615 7f2912507700 5 -- op tracker -- seq: 2967, time: 2014-11-03 11:37:51.232615, event: done, op: osd_sub_op(client.8901060.0:765732 6.1f2 2a1d1f2/rbd_data.751b802ae8944a.0000000000000409/head//6 [] v 105950'9270000 snapset=0=[]:[] snapc=0=[]) -6128> 2014-11-03 11:37:51.232648 7f2912507700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:51.232648, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6127> 2014-11-03 11:37:51.232679 7f2912507700 5 -- op tracker -- seq: 2966, time: 2014-11-03 11:37:51.232679, event: done, op: osd_sub_op(client.7869116.0:21661155 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850150 snapset=0=[]:[] snapc=0=[]) -6126> 2014-11-03 11:37:51.232749 7f2912507700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:51.232749, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6125> 2014-11-03 11:37:51.232781 7f2912507700 5 -- op tracker -- seq: 2968, time: 2014-11-03 11:37:51.232781, event: done, op: osd_sub_op(client.7868828.0:25297283 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166716 snapset=0=[]:[] snapc=0=[]) -6124> 2014-11-03 11:37:51.232796 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 50 ==== osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4921 (216537333 0 672935632) 0x25eb9400 con 0x22fe0000 -6123> 2014-11-03 11:37:51.232829 7f28f4fdb700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.232679, event: header_read, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6122> 2014-11-03 11:37:51.232850 7f28f4fdb700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.232681, event: throttled, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6121> 2014-11-03 11:37:51.232863 7f28f4fdb700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.232783, event: all_read, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6120> 2014-11-03 11:37:51.232876 7f28f4fdb700 5 -- op tracker -- seq: 2972, time: 0.000000, event: dispatched, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6119> 2014-11-03 11:37:51.232925 7f2912507700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:51.232925, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6118> 2014-11-03 11:37:51.232955 7f2912507700 5 -- op tracker -- seq: 2969, time: 2014-11-03 11:37:51.232955, event: done, op: osd_sub_op(client.7868828.0:25297284 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166717 snapset=0=[]:[] snapc=0=[]) -6117> 2014-11-03 11:37:51.232963 7f28fe818700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.232962, event: reached_pg, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6116> 2014-11-03 11:37:51.232998 7f28fe818700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.232998, event: started, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6115> 2014-11-03 11:37:51.233134 7f2912507700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:51.233134, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6114> 2014-11-03 11:37:51.233165 7f2912507700 5 -- op tracker -- seq: 2970, time: 2014-11-03 11:37:51.233165, event: done, op: osd_sub_op(client.7868828.0:25297285 6.773 cfb53773/rbd_data.4d48f961268364.000000000000014c/head//6 [] v 105950'12166718 snapset=0=[]:[] snapc=0=[]) -6113> 2014-11-03 11:37:51.233246 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 51 ==== MRecoveryReserve RELEASE pgid: 6.1d3, query_epoch: 105950 v2 ==== 26+0+0 (357554565 0 0) 0x2318aa00 con 0x22fe0000 -6112> 2014-11-03 11:37:51.233273 7f290b031700 5 -- op tracker -- seq: 2973, time: 2014-11-03 11:37:51.233084, event: header_read, op: MRecoveryReserve RELEASE pgid: 6.1d3, query_epoch: 105950 -6111> 2014-11-03 11:37:51.233260 7f2912507700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:51.233260, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6110> 2014-11-03 11:37:51.233297 7f290b031700 5 -- op tracker -- seq: 2973, time: 2014-11-03 11:37:51.233086, event: throttled, op: MRecoveryReserve RELEASE pgid: 6.1d3, query_epoch: 105950 -6109> 2014-11-03 11:37:51.233335 7f2912507700 5 -- op tracker -- seq: 2971, time: 2014-11-03 11:37:51.233335, event: done, op: osd_sub_op(client.8703713.0:32254391 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953146 snapset=0=[]:[] snapc=0=[]) -6108> 2014-11-03 11:37:51.233331 7f290b031700 5 -- op tracker -- seq: 2973, time: 2014-11-03 11:37:51.233134, event: all_read, op: MRecoveryReserve RELEASE pgid: 6.1d3, query_epoch: 105950 -6107> 2014-11-03 11:37:51.233371 7f290b031700 5 -- op tracker -- seq: 2973, time: 2014-11-03 11:37:51.233267, event: dispatched, op: MRecoveryReserve RELEASE pgid: 6.1d3, query_epoch: 105950 -6106> 2014-11-03 11:37:51.233384 7f290b031700 5 -- op tracker -- seq: 2973, time: 2014-11-03 11:37:51.233384, event: waiting_for_osdmap, op: MRecoveryReserve RELEASE pgid: 6.1d3, query_epoch: 105950 -6105> 2014-11-03 11:37:51.233802 7f28fe818700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.233802, event: started, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6104> 2014-11-03 11:37:51.233861 7f28fe818700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.233861, event: commit_queued_for_journal_write, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6103> 2014-11-03 11:37:51.233939 7f2914d0c700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.233939, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6102> 2014-11-03 11:37:51.233972 7f290b031700 5 -- op tracker -- seq: 2973, time: 2014-11-03 11:37:51.233972, event: done, op: MRecoveryReserve RELEASE pgid: 6.1d3, query_epoch: 105950 -6101> 2014-11-03 11:37:51.234006 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 52 ==== pg_info(1 pgs e105950:6.1d3) v4 ==== 1743+0+0 (3696792593 0 0) 0x286bb200 con 0x22fe0000 -6100> 2014-11-03 11:37:51.234021 7f290b031700 5 -- op tracker -- seq: 2974, time: 2014-11-03 11:37:51.233688, event: header_read, op: pg_info(1 pgs e105950:6.1d3) -6099> 2014-11-03 11:37:51.234028 7f290b031700 5 -- op tracker -- seq: 2974, time: 2014-11-03 11:37:51.233690, event: throttled, op: pg_info(1 pgs e105950:6.1d3) -6098> 2014-11-03 11:37:51.234037 7f290b031700 5 -- op tracker -- seq: 2974, time: 2014-11-03 11:37:51.233794, event: all_read, op: pg_info(1 pgs e105950:6.1d3) -6097> 2014-11-03 11:37:51.234047 7f290b031700 5 -- op tracker -- seq: 2974, time: 2014-11-03 11:37:51.234018, event: dispatched, op: pg_info(1 pgs e105950:6.1d3) -6096> 2014-11-03 11:37:51.234023 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.1d3( v 105950'9525547 (105913'9521552,105950'9525547] local-les=105948 n=4612 ec=164 les/c 105948/105943 105947/105947/105111) [70,125,21] r=2 lpr=105947 pi=58133-105946/394 luod=0'0 crt=105950'9525545 lcod 105950'9525545 active] exit Started/ReplicaActive/RepRecovering 4.881016 1 0.000148 -6095> 2014-11-03 11:37:51.234052 7f290b031700 5 -- op tracker -- seq: 2974, time: 2014-11-03 11:37:51.234052, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:6.1d3) -6094> 2014-11-03 11:37:51.234061 7f290b031700 5 -- op tracker -- seq: 2974, time: 2014-11-03 11:37:51.234060, event: started, op: pg_info(1 pgs e105950:6.1d3) -6093> 2014-11-03 11:37:51.234080 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.1d3( v 105950'9525547 (105913'9521552,105950'9525547] local-les=105948 n=4612 ec=164 les/c 105948/105943 105947/105947/105111) [70,125,21] r=2 lpr=105947 pi=58133-105946/394 luod=0'0 crt=105950'9525545 lcod 105950'9525545 active] enter Started/ReplicaActive/RepNotRecovering -6092> 2014-11-03 11:37:51.234215 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MRecoveryReserve GRANT pgid: 6.91c, query_epoch: 105950 v2 -- ?+0 0x2b950c00 con 0x23000b00 -6091> 2014-11-03 11:37:51.234221 7f290b031700 5 -- op tracker -- seq: 2974, time: 2014-11-03 11:37:51.234221, event: done, op: pg_info(1 pgs e105950:6.1d3) -6090> 2014-11-03 11:37:51.234252 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.91c( v 105950'10711003 lc 105945'10710475 (105913'10707148,105950'10711003] local-les=105948 n=4406 ec=164 les/c 105948/105943 105947/105947/105254) [97,21,72] r=1 lpr=105947 pi=93257-105946/345 luod=0'0 crt=105946'10710996 active m=8] exit Started/ReplicaActive/RepWaitRecoveryReserved 24.590779 5 0.000113 -6089> 2014-11-03 11:37:51.234296 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.91c( v 105950'10711003 lc 105945'10710475 (105913'10707148,105950'10711003] local-les=105948 n=4406 ec=164 les/c 105948/105943 105947/105947/105254) [97,21,72] r=1 lpr=105947 pi=93257-105946/345 luod=0'0 crt=105946'10710996 active m=8] enter Started/ReplicaActive/RepRecovering -6088> 2014-11-03 11:37:51.236035 7f291450b700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.236035, event: journaled_completion_queued, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6087> 2014-11-03 11:37:51.236136 7f2911d06700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.236136, event: commit_sent, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6086> 2014-11-03 11:37:51.236170 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- osd_sub_op_reply(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2aa6bb00 con 0x22fe0000 -6085> 2014-11-03 11:37:51.236625 7f2912507700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.236624, event: sub_op_applied, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6084> 2014-11-03 11:37:51.236672 7f2912507700 5 -- op tracker -- seq: 2972, time: 2014-11-03 11:37:51.236672, event: done, op: osd_sub_op(client.8421911.0:15704357 6.1d3 c2dc51d3/rbd_data.805e322ae8944a.0000000000001d94/head//6 [] v 105950'9525547 snapset=0=[]:[] snapc=0=[]) -6083> 2014-11-03 11:37:51.243382 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.108 192.168.5.8:0/21567 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.242110) v2 ==== 47+0+0 (1941069684 0 0) 0x2320be00 con 0x2484a840 -6082> 2014-11-03 11:37:51.243434 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.242110) v2 -- ?+0 0x234d5400 con 0x2484a840 -6081> 2014-11-03 11:37:51.243439 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.108 192.168.5.8:0/21567 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.242110) v2 ==== 47+0+0 (1941069684 0 0) 0x3353600 con 0x243212c0 -6080> 2014-11-03 11:37:51.243498 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.242110) v2 -- ?+0 0x247cde00 con 0x243212c0 -6079> 2014-11-03 11:37:51.271769 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 49 ==== osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2826438841 0 3511520646) 0x2fe0c000 con 0x2301fb00 -6078> 2014-11-03 11:37:51.271824 7f28f31bd700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.271529, event: header_read, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6077> 2014-11-03 11:37:51.271901 7f28f31bd700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.271533, event: throttled, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6076> 2014-11-03 11:37:51.271922 7f28f31bd700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.271751, event: all_read, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6075> 2014-11-03 11:37:51.271941 7f28f31bd700 5 -- op tracker -- seq: 2975, time: 0.000000, event: dispatched, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6074> 2014-11-03 11:37:51.272046 7f290101d700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.272045, event: reached_pg, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6073> 2014-11-03 11:37:51.272113 7f290101d700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.272113, event: started, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6072> 2014-11-03 11:37:51.272298 7f290101d700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.272298, event: started, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6071> 2014-11-03 11:37:51.272348 7f290101d700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.272348, event: commit_queued_for_journal_write, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6070> 2014-11-03 11:37:51.272453 7f2914d0c700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.272453, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6069> 2014-11-03 11:37:51.307370 7f291450b700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.307369, event: journaled_completion_queued, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6068> 2014-11-03 11:37:51.307518 7f2911d06700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.307518, event: commit_sent, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6067> 2014-11-03 11:37:51.307604 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2aa6c600 con 0x2301fb00 -6066> 2014-11-03 11:37:51.308006 7f2912507700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.308006, event: sub_op_applied, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6065> 2014-11-03 11:37:51.308044 7f2912507700 5 -- op tracker -- seq: 2975, time: 2014-11-03 11:37:51.308044, event: done, op: osd_sub_op(client.8393371.0:2616615 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027706 snapset=0=[]:[] snapc=0=[]) -6064> 2014-11-03 11:37:51.311216 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 50 ==== osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3698598036 0 3511520646) 0x2cf7fc00 con 0x2301fb00 -6063> 2014-11-03 11:37:51.311270 7f28f31bd700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311048, event: header_read, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6062> 2014-11-03 11:37:51.311322 7f28f31bd700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311051, event: throttled, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6061> 2014-11-03 11:37:51.311352 7f28f31bd700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311197, event: all_read, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6060> 2014-11-03 11:37:51.311383 7f28f31bd700 5 -- op tracker -- seq: 2976, time: 0.000000, event: dispatched, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6059> 2014-11-03 11:37:51.311490 7f28fe818700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311489, event: reached_pg, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6058> 2014-11-03 11:37:51.311544 7f28fe818700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311544, event: started, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6057> 2014-11-03 11:37:51.311838 7f28fe818700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311838, event: started, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6056> 2014-11-03 11:37:51.311897 7f28fe818700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311897, event: commit_queued_for_journal_write, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6055> 2014-11-03 11:37:51.311997 7f2914d0c700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.311997, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6054> 2014-11-03 11:37:51.315422 7f291450b700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.315421, event: journaled_completion_queued, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6053> 2014-11-03 11:37:51.315557 7f2911d06700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.315557, event: commit_sent, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6052> 2014-11-03 11:37:51.315615 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2aa6cb80 con 0x2301fb00 -6051> 2014-11-03 11:37:51.316113 7f2912507700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.316113, event: sub_op_applied, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6050> 2014-11-03 11:37:51.316151 7f2912507700 5 -- op tracker -- seq: 2976, time: 2014-11-03 11:37:51.316151, event: done, op: osd_sub_op(client.8393371.0:2616619 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027707 snapset=0=[]:[] snapc=0=[]) -6049> 2014-11-03 11:37:51.322305 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 91 ==== MOSDPGPush(6.58e 105950 [PushOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, version: 105946'11622267, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (652434644 0 0) 0x261a4800 con 0x22fbc420 -6048> 2014-11-03 11:37:51.322378 7f28f72fe700 5 -- op tracker -- seq: 2977, time: 2014-11-03 11:37:51.304101, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, version: 105946'11622267, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6047> 2014-11-03 11:37:51.322413 7f28f72fe700 5 -- op tracker -- seq: 2977, time: 2014-11-03 11:37:51.304103, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, version: 105946'11622267, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6046> 2014-11-03 11:37:51.322450 7f28f72fe700 5 -- op tracker -- seq: 2977, time: 2014-11-03 11:37:51.322275, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, version: 105946'11622267, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6045> 2014-11-03 11:37:51.322507 7f28f72fe700 5 -- op tracker -- seq: 2977, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, version: 105946'11622267, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6044> 2014-11-03 11:37:51.322617 7f290101d700 5 -- op tracker -- seq: 2977, time: 2014-11-03 11:37:51.322617, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, version: 105946'11622267, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6043> 2014-11-03 11:37:51.322960 7f290101d700 5 -- op tracker -- seq: 2977, time: 2014-11-03 11:37:51.322960, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6, version: 105946'11622267, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c0834d8e/rbd_data.89cd895a911542.0000000000007049/head//6@105946'11622267, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6042> 2014-11-03 11:37:51.338480 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x261a4e00 con 0x22fbc420 -6041> 2014-11-03 11:37:51.344179 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.109 192.168.5.8:0/25883 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.343160) v2 ==== 47+0+0 (1801513749 0 0) 0x2d329400 con 0x2327a000 -6040> 2014-11-03 11:37:51.344208 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.109 192.168.5.8:0/25883 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.343160) v2 ==== 47+0+0 (1801513749 0 0) 0x2355b600 con 0x2327a160 -6039> 2014-11-03 11:37:51.344251 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/25883 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.343160) v2 -- ?+0 0x2320be00 con 0x2327a000 -6038> 2014-11-03 11:37:51.344327 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/25883 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.343160) v2 -- ?+0 0x3353600 con 0x2327a160 -6037> 2014-11-03 11:37:51.356820 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.74 192.168.5.6:0/9268 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.355603) v2 ==== 47+0+0 (237347499 0 0) 0x2a25be00 con 0x24229160 -6036> 2014-11-03 11:37:51.356808 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.74 192.168.5.6:0/9268 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.355603) v2 ==== 47+0+0 (237347499 0 0) 0x27256c00 con 0x24925420 -6035> 2014-11-03 11:37:51.356894 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/9268 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.355603) v2 -- ?+0 0x2355b600 con 0x24229160 -6034> 2014-11-03 11:37:51.356990 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/9268 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.355603) v2 -- ?+0 0x2d329400 con 0x24925420 -6033> 2014-11-03 11:37:51.360520 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 92 ==== MOSDPGPush(6.58e 105950 [PushOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, version: 105946'11622286, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1182280823 0 0) 0x261a4e00 con 0x22fbc420 -6032> 2014-11-03 11:37:51.360593 7f28f72fe700 5 -- op tracker -- seq: 2978, time: 2014-11-03 11:37:51.344088, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, version: 105946'11622286, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6031> 2014-11-03 11:37:51.360635 7f28f72fe700 5 -- op tracker -- seq: 2978, time: 2014-11-03 11:37:51.344091, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, version: 105946'11622286, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6030> 2014-11-03 11:37:51.360656 7f28f72fe700 5 -- op tracker -- seq: 2978, time: 2014-11-03 11:37:51.360504, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, version: 105946'11622286, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6029> 2014-11-03 11:37:51.360675 7f28f72fe700 5 -- op tracker -- seq: 2978, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, version: 105946'11622286, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6028> 2014-11-03 11:37:51.360769 7f28fe818700 5 -- op tracker -- seq: 2978, time: 2014-11-03 11:37:51.360769, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, version: 105946'11622286, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6027> 2014-11-03 11:37:51.361064 7f28fe818700 5 -- op tracker -- seq: 2978, time: 2014-11-03 11:37:51.361064, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6, version: 105946'11622286, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(14cc2d8e/rbd_data.78c5f91f5323e4.00000000000010d4/head//6@105946'11622286, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -6026> 2014-11-03 11:37:51.442251 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x26a9f000 con 0x22fbc420 -6025> 2014-11-03 11:37:51.449832 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 65 ==== osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (316568244 0 3511520646) 0x26059000 con 0x22fbc840 -6024> 2014-11-03 11:37:51.449882 7f28f68f4700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.449629, event: header_read, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6023> 2014-11-03 11:37:51.449905 7f28f68f4700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.449632, event: throttled, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6022> 2014-11-03 11:37:51.449940 7f28f68f4700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.449815, event: all_read, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6021> 2014-11-03 11:37:51.449950 7f28f68f4700 5 -- op tracker -- seq: 2979, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6020> 2014-11-03 11:37:51.450163 7f290081c700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.450162, event: reached_pg, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6019> 2014-11-03 11:37:51.450272 7f290081c700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.450272, event: started, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6018> 2014-11-03 11:37:51.450485 7f290081c700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.450485, event: started, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6017> 2014-11-03 11:37:51.450525 7f290081c700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.450525, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6016> 2014-11-03 11:37:51.450641 7f2914d0c700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.450640, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6015> 2014-11-03 11:37:51.468134 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.36 192.168.5.3:0/9756 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.467456) v2 ==== 47+0+0 (3874566974 0 0) 0x252d7000 con 0x240ad2c0 -6014> 2014-11-03 11:37:51.468206 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/9756 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.467456) v2 -- ?+0 0x27256c00 con 0x240ad2c0 -6013> 2014-11-03 11:37:51.468190 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.36 192.168.5.3:0/9756 6 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.467456) v2 ==== 47+0+0 (3874566974 0 0) 0xcd57000 con 0x2469e840 -6012> 2014-11-03 11:37:51.468294 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/9756 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.467456) v2 -- ?+0 0x2a25be00 con 0x2469e840 -6011> 2014-11-03 11:37:51.468444 7f291450b700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.468444, event: journaled_completion_queued, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6010> 2014-11-03 11:37:51.468534 7f2911d06700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.468534, event: commit_sent, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6009> 2014-11-03 11:37:51.468604 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2aa6b580 con 0x22fbc840 -6008> 2014-11-03 11:37:51.468940 7f2912507700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.468940, event: sub_op_applied, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6007> 2014-11-03 11:37:51.469006 7f2912507700 5 -- op tracker -- seq: 2979, time: 2014-11-03 11:37:51.469006, event: done, op: osd_sub_op(client.7867856.0:1671196 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195960 snapset=0=[]:[] snapc=0=[]) -6006> 2014-11-03 11:37:51.472849 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 66 ==== osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (271804405 0 3511520646) 0xd4a6800 con 0x22fbc840 -6005> 2014-11-03 11:37:51.472882 7f28f68f4700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.472759, event: header_read, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -6004> 2014-11-03 11:37:51.472897 7f28f68f4700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.472761, event: throttled, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -6003> 2014-11-03 11:37:51.472906 7f28f68f4700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.472839, event: all_read, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -6002> 2014-11-03 11:37:51.472914 7f28f68f4700 5 -- op tracker -- seq: 2980, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -6001> 2014-11-03 11:37:51.472968 7f28fe017700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.472968, event: reached_pg, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -6000> 2014-11-03 11:37:51.473022 7f28fe017700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.473022, event: started, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5999> 2014-11-03 11:37:51.473227 7f28fe017700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.473227, event: started, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5998> 2014-11-03 11:37:51.473277 7f28fe017700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.473277, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5997> 2014-11-03 11:37:51.473363 7f2914d0c700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.473362, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5996> 2014-11-03 11:37:51.473694 7f291450b700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.473693, event: journaled_completion_queued, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5995> 2014-11-03 11:37:51.473781 7f2911d06700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.473780, event: commit_sent, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5994> 2014-11-03 11:37:51.473823 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7dc0 con 0x22fbc840 -5993> 2014-11-03 11:37:51.474397 7f2912507700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.474397, event: sub_op_applied, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5992> 2014-11-03 11:37:51.474638 7f2912507700 5 -- op tracker -- seq: 2980, time: 2014-11-03 11:37:51.474638, event: done, op: osd_sub_op(client.7867856.0:1671198 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195961 snapset=0=[]:[] snapc=0=[]) -5991> 2014-11-03 11:37:51.476438 7f28f0a96700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 91 ==== osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1090792613 0 3511520646) 0x24661000 con 0x230a92c0 -5990> 2014-11-03 11:37:51.476473 7f28f0a96700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.476310, event: header_read, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5989> 2014-11-03 11:37:51.476492 7f28f0a96700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.476312, event: throttled, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5988> 2014-11-03 11:37:51.476516 7f28f0a96700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.476427, event: all_read, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5987> 2014-11-03 11:37:51.476535 7f28f0a96700 5 -- op tracker -- seq: 2981, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5986> 2014-11-03 11:37:51.476608 7f290101d700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.476608, event: reached_pg, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5985> 2014-11-03 11:37:51.476639 7f290101d700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.476639, event: started, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5984> 2014-11-03 11:37:51.476792 7f290101d700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.476792, event: started, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5983> 2014-11-03 11:37:51.476842 7f290101d700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.476842, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5982> 2014-11-03 11:37:51.477084 7f2914d0c700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.477084, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5981> 2014-11-03 11:37:51.477405 7f291450b700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.477405, event: journaled_completion_queued, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5980> 2014-11-03 11:37:51.477488 7f28f0a96700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 92 ==== osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2467640392 0 3511520646) 0x24663400 con 0x230a92c0 -5979> 2014-11-03 11:37:51.477539 7f28f0a96700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.477276, event: header_read, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5978> 2014-11-03 11:37:51.477564 7f2911d06700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.477564, event: commit_sent, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5977> 2014-11-03 11:37:51.477584 7f28f0a96700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.477278, event: throttled, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5976> 2014-11-03 11:37:51.477595 7f28f0a96700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.477470, event: all_read, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5975> 2014-11-03 11:37:51.477600 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- osd_sub_op_reply(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7580 con 0x230a92c0 -5974> 2014-11-03 11:37:51.477624 7f28f0a96700 5 -- op tracker -- seq: 2982, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5973> 2014-11-03 11:37:51.477713 7f28fe818700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.477713, event: reached_pg, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5972> 2014-11-03 11:37:51.477752 7f28fe818700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.477752, event: started, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5971> 2014-11-03 11:37:51.477864 7f28fe818700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.477864, event: started, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5970> 2014-11-03 11:37:51.477901 7f28fe818700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.477901, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5969> 2014-11-03 11:37:51.477978 7f28f0a96700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 93 ==== osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1760622256 0 3511520646) 0x2cf81400 con 0x230a92c0 -5968> 2014-11-03 11:37:51.478027 7f28f0a96700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.477788, event: header_read, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5967> 2014-11-03 11:37:51.478043 7f28f0a96700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.477791, event: throttled, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5966> 2014-11-03 11:37:51.478057 7f28f0a96700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.477963, event: all_read, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5965> 2014-11-03 11:37:51.478070 7f28f0a96700 5 -- op tracker -- seq: 2983, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5964> 2014-11-03 11:37:51.478135 7f2914d0c700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.478134, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5963> 2014-11-03 11:37:51.478161 7f2912507700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.478161, event: sub_op_applied, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5962> 2014-11-03 11:37:51.478261 7f2912507700 5 -- op tracker -- seq: 2981, time: 2014-11-03 11:37:51.478261, event: done, op: osd_sub_op(client.7868798.0:25415449 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108018 snapset=0=[]:[] snapc=0=[]) -5961> 2014-11-03 11:37:51.478368 7f290101d700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.478367, event: reached_pg, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5960> 2014-11-03 11:37:51.478436 7f290101d700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.478436, event: started, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5959> 2014-11-03 11:37:51.478569 7f290101d700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.478569, event: started, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5958> 2014-11-03 11:37:51.478611 7f290101d700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.478611, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5957> 2014-11-03 11:37:51.478729 7f291450b700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.478729, event: journaled_completion_queued, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5956> 2014-11-03 11:37:51.478808 7f2911d06700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.478808, event: commit_sent, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5955> 2014-11-03 11:37:51.478827 7f2914d0c700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.478827, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5954> 2014-11-03 11:37:51.478851 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- osd_sub_op_reply(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7b00 con 0x230a92c0 -5953> 2014-11-03 11:37:51.479162 7f291450b700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.479162, event: journaled_completion_queued, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5952> 2014-11-03 11:37:51.479144 7f2912507700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.479144, event: sub_op_applied, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5951> 2014-11-03 11:37:51.479215 7f2912507700 5 -- op tracker -- seq: 2982, time: 2014-11-03 11:37:51.479215, event: done, op: osd_sub_op(client.7868798.0:25415450 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108019 snapset=0=[]:[] snapc=0=[]) -5950> 2014-11-03 11:37:51.479263 7f2911d06700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.479263, event: commit_sent, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5949> 2014-11-03 11:37:51.479293 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- osd_sub_op_reply(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7840 con 0x230a92c0 -5948> 2014-11-03 11:37:51.479445 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 67 ==== osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1765891659 0 3511520646) 0x26dba000 con 0x22fbc840 -5947> 2014-11-03 11:37:51.479498 7f28f68f4700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479276, event: header_read, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5946> 2014-11-03 11:37:51.479536 7f28f68f4700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479277, event: throttled, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5945> 2014-11-03 11:37:51.479554 7f28f68f4700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479415, event: all_read, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5944> 2014-11-03 11:37:51.479555 7f2912507700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.479555, event: sub_op_applied, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5943> 2014-11-03 11:37:51.479569 7f28f68f4700 5 -- op tracker -- seq: 2984, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5942> 2014-11-03 11:37:51.479581 7f2912507700 5 -- op tracker -- seq: 2983, time: 2014-11-03 11:37:51.479581, event: done, op: osd_sub_op(client.7868798.0:25415451 6.75f b7c8f5f/rbd_data.5012e52ae8944a.00000000000000e5/head//6 [] v 105950'15108020 snapset=0=[]:[] snapc=0=[]) -5941> 2014-11-03 11:37:51.479665 7f290081c700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479664, event: reached_pg, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5940> 2014-11-03 11:37:51.479709 7f290081c700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479709, event: started, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5939> 2014-11-03 11:37:51.479834 7f290081c700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479834, event: started, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5938> 2014-11-03 11:37:51.479867 7f290081c700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479867, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5937> 2014-11-03 11:37:51.479930 7f2914d0c700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.479930, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5936> 2014-11-03 11:37:51.480166 7f291450b700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.480166, event: journaled_completion_queued, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5935> 2014-11-03 11:37:51.480206 7f2911d06700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.480205, event: commit_sent, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5934> 2014-11-03 11:37:51.480243 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f8c0 con 0x22fbc840 -5933> 2014-11-03 11:37:51.480414 7f2912507700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.480414, event: sub_op_applied, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5932> 2014-11-03 11:37:51.480459 7f2912507700 5 -- op tracker -- seq: 2984, time: 2014-11-03 11:37:51.480459, event: done, op: osd_sub_op(client.7867856.0:1671200 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195962 snapset=0=[]:[] snapc=0=[]) -5931> 2014-11-03 11:37:51.485560 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 68 ==== osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3397985197 0 3511520646) 0x2cff4800 con 0x22fbc840 -5930> 2014-11-03 11:37:51.485607 7f28f68f4700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.485432, event: header_read, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5929> 2014-11-03 11:37:51.485639 7f28f68f4700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.485434, event: throttled, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5928> 2014-11-03 11:37:51.485670 7f28f68f4700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.485544, event: all_read, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5927> 2014-11-03 11:37:51.485684 7f28f68f4700 5 -- op tracker -- seq: 2985, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5926> 2014-11-03 11:37:51.485768 7f28fe017700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.485768, event: reached_pg, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5925> 2014-11-03 11:37:51.485820 7f28fe017700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.485820, event: started, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5924> 2014-11-03 11:37:51.485956 7f28fe017700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.485956, event: started, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5923> 2014-11-03 11:37:51.486006 7f28fe017700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.486006, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5922> 2014-11-03 11:37:51.486125 7f2914d0c700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.486125, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5921> 2014-11-03 11:37:51.486490 7f291450b700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.486490, event: journaled_completion_queued, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5920> 2014-11-03 11:37:51.486566 7f2911d06700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.486566, event: commit_sent, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5919> 2014-11-03 11:37:51.486620 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f080 con 0x22fbc840 -5918> 2014-11-03 11:37:51.486825 7f2912507700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.486825, event: sub_op_applied, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5917> 2014-11-03 11:37:51.486856 7f2912507700 5 -- op tracker -- seq: 2985, time: 2014-11-03 11:37:51.486856, event: done, op: osd_sub_op(client.7867856.0:1671204 6.a8a 79051a8a/rbd_data.735f372ae8944a.00000000000002f9/head//6 [] v 105950'8195963 snapset=0=[]:[] snapc=0=[]) -5916> 2014-11-03 11:37:51.493829 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.1 192.168.5.1:0/11054 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.493552) v2 ==== 47+0+0 (3400606904 0 0) 0x2a25b800 con 0x2327a580 -5915> 2014-11-03 11:37:51.493890 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/11054 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.493552) v2 -- ?+0 0xcd57000 con 0x2327a580 -5914> 2014-11-03 11:37:51.494210 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.1 192.168.5.1:0/11054 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.493552) v2 ==== 47+0+0 (3400606904 0 0) 0x2c390a00 con 0x2327a2c0 -5913> 2014-11-03 11:37:51.494256 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/11054 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.493552) v2 -- ?+0 0x252d7000 con 0x2327a2c0 -5912> 2014-11-03 11:37:51.518098 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 69 ==== osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (940277386 0 3511520646) 0x27813400 con 0x22fbc840 -5911> 2014-11-03 11:37:51.518173 7f28f68f4700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.517953, event: header_read, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5910> 2014-11-03 11:37:51.518216 7f28f68f4700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.517955, event: throttled, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5909> 2014-11-03 11:37:51.518244 7f28f68f4700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518083, event: all_read, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5908> 2014-11-03 11:37:51.518270 7f28f68f4700 5 -- op tracker -- seq: 2986, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5907> 2014-11-03 11:37:51.518358 7f290081c700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518358, event: reached_pg, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5906> 2014-11-03 11:37:51.518396 7f290081c700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518396, event: started, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5905> 2014-11-03 11:37:51.518501 7f290081c700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518501, event: started, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5904> 2014-11-03 11:37:51.518534 7f290081c700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518534, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5903> 2014-11-03 11:37:51.518604 7f2914d0c700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518604, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5902> 2014-11-03 11:37:51.518866 7f291450b700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518866, event: journaled_completion_queued, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5901> 2014-11-03 11:37:51.518875 7f2911d06700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.518875, event: commit_sent, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5900> 2014-11-03 11:37:51.518959 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f600 con 0x22fbc840 -5899> 2014-11-03 11:37:51.519364 7f2912507700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.519363, event: sub_op_applied, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5898> 2014-11-03 11:37:51.519430 7f2912507700 5 -- op tracker -- seq: 2986, time: 2014-11-03 11:37:51.519430, event: done, op: osd_sub_op(client.7867945.0:15039368 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195964 snapset=0=[]:[] snapc=0=[]) -5897> 2014-11-03 11:37:51.519679 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 70 ==== osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3974341775 0 3511520646) 0x27811000 con 0x22fbc840 -5896> 2014-11-03 11:37:51.519721 7f28f68f4700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.519567, event: header_read, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5895> 2014-11-03 11:37:51.519742 7f28f68f4700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.519569, event: throttled, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5894> 2014-11-03 11:37:51.519756 7f28f68f4700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.519664, event: all_read, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5893> 2014-11-03 11:37:51.519779 7f28f68f4700 5 -- op tracker -- seq: 2987, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5892> 2014-11-03 11:37:51.519988 7f28fe017700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.519987, event: reached_pg, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5891> 2014-11-03 11:37:51.520016 7f28fe017700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520016, event: started, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5890> 2014-11-03 11:37:51.520114 7f28fe017700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520113, event: started, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5889> 2014-11-03 11:37:51.520148 7f28fe017700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520148, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5888> 2014-11-03 11:37:51.520215 7f2914d0c700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520215, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5887> 2014-11-03 11:37:51.520439 7f291450b700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520438, event: journaled_completion_queued, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5886> 2014-11-03 11:37:51.520482 7f2911d06700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520482, event: commit_sent, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5885> 2014-11-03 11:37:51.520518 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28229b80 con 0x22fbc840 -5884> 2014-11-03 11:37:51.520848 7f2912507700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520848, event: sub_op_applied, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5883> 2014-11-03 11:37:51.520902 7f2912507700 5 -- op tracker -- seq: 2987, time: 2014-11-03 11:37:51.520902, event: done, op: osd_sub_op(client.7867945.0:15039369 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195965 snapset=0=[]:[] snapc=0=[]) -5882> 2014-11-03 11:37:51.521747 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.70 192.168.5.6:0/3622 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.518924) v2 ==== 47+0+0 (2520580111 0 0) 0x2a9d3a00 con 0x239f8b00 -5881> 2014-11-03 11:37:51.521803 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/3622 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.518924) v2 -- ?+0 0x2c390a00 con 0x239f8b00 -5880> 2014-11-03 11:37:51.522486 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.70 192.168.5.6:0/3622 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.518924) v2 ==== 47+0+0 (2520580111 0 0) 0x27256400 con 0x238b8c60 -5879> 2014-11-03 11:37:51.522532 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/3622 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.518924) v2 -- ?+0 0x2a25b800 con 0x238b8c60 -5878> 2014-11-03 11:37:51.526886 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.99 192.168.5.8:0/26930 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.525731) v2 ==== 47+0+0 (1482853611 0 0) 0x2be31c00 con 0x239f8160 -5877> 2014-11-03 11:37:51.526928 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.525731) v2 -- ?+0 0x2a9d3a00 con 0x239f8160 -5876> 2014-11-03 11:37:51.526945 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.99 192.168.5.8:0/26930 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.525731) v2 ==== 47+0+0 (1482853611 0 0) 0x2369dc00 con 0x24205580 -5875> 2014-11-03 11:37:51.526988 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.525731) v2 -- ?+0 0x27256400 con 0x24205580 -5874> 2014-11-03 11:37:51.544656 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 93 ==== MOSDPGPush(6.58e 105950 [PushOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, version: 105946'11622292, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1437558204 0 0) 0x26a9f000 con 0x22fbc420 -5873> 2014-11-03 11:37:51.544723 7f28f72fe700 5 -- op tracker -- seq: 2988, time: 2014-11-03 11:37:51.527029, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, version: 105946'11622292, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5872> 2014-11-03 11:37:51.544769 7f28f72fe700 5 -- op tracker -- seq: 2988, time: 2014-11-03 11:37:51.527030, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, version: 105946'11622292, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5871> 2014-11-03 11:37:51.544795 7f28f72fe700 5 -- op tracker -- seq: 2988, time: 2014-11-03 11:37:51.544640, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, version: 105946'11622292, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5870> 2014-11-03 11:37:51.544822 7f28f72fe700 5 -- op tracker -- seq: 2988, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, version: 105946'11622292, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5869> 2014-11-03 11:37:51.544897 7f28fe818700 5 -- op tracker -- seq: 2988, time: 2014-11-03 11:37:51.544897, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, version: 105946'11622292, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5868> 2014-11-03 11:37:51.545131 7f28fe818700 5 -- op tracker -- seq: 2988, time: 2014-11-03 11:37:51.545130, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6, version: 105946'11622292, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f71c3d8e/rbd_data.733ef12ae8944a.0000000000001e79/head//6@105946'11622292, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5867> 2014-11-03 11:37:51.553515 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.77 192.168.5.6:0/32590 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.552256) v2 ==== 47+0+0 (1507310978 0 0) 0x23fdfc00 con 0x2484a420 -5866> 2014-11-03 11:37:51.553592 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/32590 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.552256) v2 -- ?+0 0x2be31c00 con 0x2484a420 -5865> 2014-11-03 11:37:51.553603 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.77 192.168.5.6:0/32590 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.552256) v2 ==== 47+0+0 (1507310978 0 0) 0x1a327a00 con 0x24205840 -5864> 2014-11-03 11:37:51.553665 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/32590 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.552256) v2 -- ?+0 0x2369dc00 con 0x24205840 -5863> 2014-11-03 11:37:51.558390 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 71 ==== osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3655428737 0 3511520646) 0x2ce3b000 con 0x22fbc840 -5862> 2014-11-03 11:37:51.558443 7f28f68f4700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.558196, event: header_read, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5861> 2014-11-03 11:37:51.558470 7f28f68f4700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.558198, event: throttled, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5860> 2014-11-03 11:37:51.558490 7f28f68f4700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.558373, event: all_read, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5859> 2014-11-03 11:37:51.558518 7f28f68f4700 5 -- op tracker -- seq: 2989, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5858> 2014-11-03 11:37:51.558573 7f290081c700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.558573, event: reached_pg, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5857> 2014-11-03 11:37:51.558608 7f290081c700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.558608, event: started, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5856> 2014-11-03 11:37:51.558726 7f290081c700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.558726, event: started, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5855> 2014-11-03 11:37:51.558758 7f290081c700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.558758, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5854> 2014-11-03 11:37:51.571908 7f28f18a4700 1 -- 192.168.5.2:6803/7606 <== osd.60 192.168.5.5:6826/5983 36 ==== osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (244042469 0 3511520646) 0x2775a400 con 0x22ff3840 -5853> 2014-11-03 11:37:51.571957 7f28f18a4700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.571784, event: header_read, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5852> 2014-11-03 11:37:51.571974 7f28f18a4700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.571787, event: throttled, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5851> 2014-11-03 11:37:51.571988 7f28f18a4700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.571893, event: all_read, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5850> 2014-11-03 11:37:51.572000 7f28f18a4700 5 -- op tracker -- seq: 2990, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5849> 2014-11-03 11:37:51.572076 7f28ff019700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.572076, event: reached_pg, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5848> 2014-11-03 11:37:51.572134 7f28ff019700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.572134, event: started, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5847> 2014-11-03 11:37:51.572300 7f28ff019700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.572299, event: started, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5846> 2014-11-03 11:37:51.572349 7f28ff019700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.572349, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5845> 2014-11-03 11:37:51.575902 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 84 ==== osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3308919377 0 3511520646) 0x26153400 con 0x22ff3b00 -5844> 2014-11-03 11:37:51.575936 7f28f0d99700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.575802, event: header_read, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5843> 2014-11-03 11:37:51.575951 7f28f0d99700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.575804, event: throttled, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5842> 2014-11-03 11:37:51.575964 7f28f0d99700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.575892, event: all_read, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5841> 2014-11-03 11:37:51.575974 7f28f0d99700 5 -- op tracker -- seq: 2991, time: 0.000000, event: dispatched, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5840> 2014-11-03 11:37:51.576053 7f290101d700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.576053, event: reached_pg, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5839> 2014-11-03 11:37:51.576090 7f290101d700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.576090, event: started, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5838> 2014-11-03 11:37:51.576236 7f290101d700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.576235, event: started, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5837> 2014-11-03 11:37:51.576279 7f290101d700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.576279, event: commit_queued_for_journal_write, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5836> 2014-11-03 11:37:51.577892 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 85 ==== osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2557707979 0 3511520646) 0x24bec800 con 0x22ff3b00 -5835> 2014-11-03 11:37:51.577924 7f28f0d99700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.577794, event: header_read, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5834> 2014-11-03 11:37:51.577942 7f28f0d99700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.577796, event: throttled, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5833> 2014-11-03 11:37:51.577956 7f28f0d99700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.577882, event: all_read, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5832> 2014-11-03 11:37:51.577976 7f28f0d99700 5 -- op tracker -- seq: 2992, time: 0.000000, event: dispatched, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5831> 2014-11-03 11:37:51.578039 7f28fe818700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.578039, event: reached_pg, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5830> 2014-11-03 11:37:51.578070 7f28fe818700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.578070, event: started, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5829> 2014-11-03 11:37:51.578243 7f28fe818700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.578243, event: started, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5828> 2014-11-03 11:37:51.578284 7f28fe818700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.578284, event: commit_queued_for_journal_write, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5827> 2014-11-03 11:37:51.579775 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 86 ==== osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (686517973 0 3511520646) 0x2bd4e000 con 0x22ff3b00 -5826> 2014-11-03 11:37:51.579807 7f28f0d99700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.579616, event: header_read, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5825> 2014-11-03 11:37:51.579824 7f28f0d99700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.579617, event: throttled, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5824> 2014-11-03 11:37:51.579833 7f28f0d99700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.579766, event: all_read, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5823> 2014-11-03 11:37:51.579844 7f28f0d99700 5 -- op tracker -- seq: 2993, time: 0.000000, event: dispatched, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5822> 2014-11-03 11:37:51.579898 7f290101d700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.579898, event: reached_pg, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5821> 2014-11-03 11:37:51.579924 7f290101d700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.579924, event: started, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5820> 2014-11-03 11:37:51.580015 7f290101d700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.580015, event: started, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5819> 2014-11-03 11:37:51.580044 7f290101d700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.580044, event: commit_queued_for_journal_write, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5818> 2014-11-03 11:37:51.581669 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.61 192.168.5.5:0/12481 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.580260) v2 ==== 47+0+0 (572636735 0 0) 0x3353400 con 0x2445c840 -5817> 2014-11-03 11:37:51.581718 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/12481 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.580260) v2 -- ?+0 0x23fdfc00 con 0x2445c840 -5816> 2014-11-03 11:37:51.581804 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.61 192.168.5.5:0/12481 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.580260) v2 ==== 47+0+0 (572636735 0 0) 0x2be31600 con 0x23ea49a0 -5815> 2014-11-03 11:37:51.581846 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/12481 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.580260) v2 -- ?+0 0x1a327a00 con 0x23ea49a0 -5814> 2014-11-03 11:37:51.586600 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.97 192.168.5.7:0/21285 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.584194) v2 ==== 47+0+0 (1923844063 0 0) 0x273fd400 con 0x24537c60 -5813> 2014-11-03 11:37:51.586606 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.97 192.168.5.7:0/21285 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.584194) v2 ==== 47+0+0 (1923844063 0 0) 0x6352800 con 0x2484a2c0 -5812> 2014-11-03 11:37:51.586662 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/21285 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.584194) v2 -- ?+0 0x2be31600 con 0x24537c60 -5811> 2014-11-03 11:37:51.586772 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/21285 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.584194) v2 -- ?+0 0x3353400 con 0x2484a2c0 -5810> 2014-11-03 11:37:51.588518 7f2914d0c700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.588518, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5809> 2014-11-03 11:37:51.588568 7f2914d0c700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.588568, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5808> 2014-11-03 11:37:51.588599 7f2914d0c700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.588599, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5807> 2014-11-03 11:37:51.588636 7f2914d0c700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.588636, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5806> 2014-11-03 11:37:51.588683 7f2914d0c700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.588683, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5805> 2014-11-03 11:37:51.589034 7f291450b700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.589034, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5804> 2014-11-03 11:37:51.589089 7f2911d06700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.589088, event: commit_sent, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5803> 2014-11-03 11:37:51.589095 7f291450b700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.589095, event: journaled_completion_queued, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5802> 2014-11-03 11:37:51.589134 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcad080 con 0x22fbc840 -5801> 2014-11-03 11:37:51.589166 7f291450b700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.589166, event: journaled_completion_queued, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5800> 2014-11-03 11:37:51.589183 7f2911d06700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.589183, event: commit_sent, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5799> 2014-11-03 11:37:51.589197 7f291450b700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.589197, event: journaled_completion_queued, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5798> 2014-11-03 11:37:51.589217 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6826/5983 -- osd_sub_op_reply(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] ondisk, result = 0) v2 -- ?+0 0x316d1000 con 0x22ff3840 -5797> 2014-11-03 11:37:51.589231 7f291450b700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.589231, event: journaled_completion_queued, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5796> 2014-11-03 11:37:51.589258 7f2911d06700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.589258, event: commit_sent, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5795> 2014-11-03 11:37:51.589326 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9b080 con 0x22ff3b00 -5794> 2014-11-03 11:37:51.589429 7f2911d06700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.589429, event: commit_sent, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5793> 2014-11-03 11:37:51.589566 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9a000 con 0x22ff3b00 -5792> 2014-11-03 11:37:51.589568 7f2912507700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.589568, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5791> 2014-11-03 11:37:51.589602 7f2911d06700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.589602, event: commit_sent, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5790> 2014-11-03 11:37:51.589619 7f2912507700 5 -- op tracker -- seq: 2989, time: 2014-11-03 11:37:51.589619, event: done, op: osd_sub_op(client.8236383.0:16630737 6.a8a a6e02a8a/rbd_data.7e655d2ae8944a.0000000000004992/head//6 [] v 105950'8195966 snapset=0=[]:[] snapc=0=[]) -5789> 2014-11-03 11:37:51.589651 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9b8c0 con 0x22ff3b00 -5788> 2014-11-03 11:37:51.589866 7f2912507700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.589865, event: sub_op_applied, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5787> 2014-11-03 11:37:51.589929 7f2912507700 5 -- op tracker -- seq: 2990, time: 2014-11-03 11:37:51.589929, event: done, op: osd_sub_op(client.7869695.0:26479428 6.597 3f38c597/rbd_data.52c5492ae8944a.00000000000016ef/head//6 [] v 105950'11526502 snapset=0=[]:[] snapc=0=[]) -5786> 2014-11-03 11:37:51.590265 7f2912507700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.590265, event: sub_op_applied, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5785> 2014-11-03 11:37:51.590331 7f2912507700 5 -- op tracker -- seq: 2991, time: 2014-11-03 11:37:51.590331, event: done, op: osd_sub_op(client.7910324.0:3584136 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795121 snapset=0=[]:[] snapc=0=[]) -5784> 2014-11-03 11:37:51.590606 7f2912507700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.590606, event: sub_op_applied, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5783> 2014-11-03 11:37:51.590691 7f2912507700 5 -- op tracker -- seq: 2992, time: 2014-11-03 11:37:51.590690, event: done, op: osd_sub_op(client.7910324.0:3584140 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795122 snapset=0=[]:[] snapc=0=[]) -5782> 2014-11-03 11:37:51.590895 7f2912507700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.590895, event: sub_op_applied, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5781> 2014-11-03 11:37:51.590947 7f2912507700 5 -- op tracker -- seq: 2993, time: 2014-11-03 11:37:51.590947, event: done, op: osd_sub_op(client.7910324.0:3584141 6.3b3 a734ebb3/rbd_data.48611220f9ae32.0000000000000337/head//6 [] v 105950'11795123 snapset=0=[]:[] snapc=0=[]) -5780> 2014-11-03 11:37:51.606389 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.85 192.168.5.7:0/27520 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.604946) v2 ==== 47+0+0 (425125628 0 0) 0x2be31400 con 0x233cb6e0 -5779> 2014-11-03 11:37:51.606461 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/27520 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.604946) v2 -- ?+0 0x273fd400 con 0x233cb6e0 -5778> 2014-11-03 11:37:51.606399 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.85 192.168.5.7:0/27520 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.604946) v2 ==== 47+0+0 (425125628 0 0) 0x22a08c00 con 0x233cb2c0 -5777> 2014-11-03 11:37:51.606526 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/27520 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.604946) v2 -- ?+0 0x6352800 con 0x233cb2c0 -5776> 2014-11-03 11:37:51.606756 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.110 192.168.5.8:0/29518 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.604787) v2 ==== 47+0+0 (1433643194 0 0) 0x29fcd800 con 0x239f82c0 -5775> 2014-11-03 11:37:51.606797 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/29518 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.604787) v2 -- ?+0 0x22a08c00 con 0x239f82c0 -5774> 2014-11-03 11:37:51.606796 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.110 192.168.5.8:0/29518 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.604787) v2 ==== 47+0+0 (1433643194 0 0) 0x2353b000 con 0x238b8420 -5773> 2014-11-03 11:37:51.606895 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/29518 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.604787) v2 -- ?+0 0x2be31400 con 0x238b8420 -5772> 2014-11-03 11:37:51.613935 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.7868162 192.168.1.71:0/2009911 7 ==== ping magic: 0 v1 ==== 0+0+0 (0 0 0) 0x32f7700 con 0x2311c840 -5771> 2014-11-03 11:37:51.633242 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 110 ==== osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1159477395 0 3511520646) 0x26942c00 con 0x22fb76e0 -5770> 2014-11-03 11:37:51.633293 7f28edd69700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633109, event: header_read, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5769> 2014-11-03 11:37:51.633310 7f28edd69700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633112, event: throttled, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5768> 2014-11-03 11:37:51.633325 7f28edd69700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633225, event: all_read, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5767> 2014-11-03 11:37:51.633337 7f28edd69700 5 -- op tracker -- seq: 2994, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5766> 2014-11-03 11:37:51.633397 7f28fd816700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633397, event: reached_pg, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5765> 2014-11-03 11:37:51.633440 7f28fd816700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633440, event: started, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5764> 2014-11-03 11:37:51.633550 7f28fd816700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633550, event: started, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5763> 2014-11-03 11:37:51.633593 7f28fd816700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633593, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5762> 2014-11-03 11:37:51.633667 7f2914d0c700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633667, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5761> 2014-11-03 11:37:51.633957 7f291450b700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.633957, event: journaled_completion_queued, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5760> 2014-11-03 11:37:51.634044 7f2911d06700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.634044, event: commit_sent, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5759> 2014-11-03 11:37:51.634083 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9b340 con 0x22fb76e0 -5758> 2014-11-03 11:37:51.634468 7f2912507700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.634468, event: sub_op_applied, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5757> 2014-11-03 11:37:51.634557 7f2912507700 5 -- op tracker -- seq: 2994, time: 2014-11-03 11:37:51.634557, event: done, op: osd_sub_op(client.7912219.0:1154961 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187823 snapset=0=[]:[] snapc=0=[]) -5756> 2014-11-03 11:37:51.642522 7f28c4099700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x248b78c0 sd=548 :6802 s=0 pgs=0 cs=0 l=0 c=0xa26c2c0).accept sd=548 192.168.1.83:48170/0 -5755> 2014-11-03 11:37:51.643443 7f28c4099700 1 -- 192.168.5.2:6802/7606 <== client.9031502 192.168.1.83:0/3009793 1 ==== osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) v4 ==== 191+0+0 (975712166 0 0) 0x78e6000 con 0xa26c2c0 -5754> 2014-11-03 11:37:51.643494 7f28c4099700 5 -- op tracker -- seq: 2995, time: 2014-11-03 11:37:51.643325, event: header_read, op: osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) -5753> 2014-11-03 11:37:51.643526 7f28c4099700 5 -- op tracker -- seq: 2995, time: 2014-11-03 11:37:51.643332, event: throttled, op: osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) -5752> 2014-11-03 11:37:51.643540 7f28c4099700 5 -- op tracker -- seq: 2995, time: 2014-11-03 11:37:51.643423, event: all_read, op: osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) -5751> 2014-11-03 11:37:51.643552 7f28c4099700 5 -- op tracker -- seq: 2995, time: 0.000000, event: dispatched, op: osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) -5750> 2014-11-03 11:37:51.643751 7f290001b700 5 -- op tracker -- seq: 2995, time: 2014-11-03 11:37:51.643751, event: reached_pg, op: osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) -5749> 2014-11-03 11:37:51.644041 7f290001b700 5 -- op tracker -- seq: 2995, time: 2014-11-03 11:37:51.644040, event: started, op: osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) -5748> 2014-11-03 11:37:51.644112 7f290001b700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.83:0/3009793 -- osd_op_reply(167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] v0'0 uv14082900 ondisk = 0) v6 -- ?+0 0x270d2dc0 con 0xa26c2c0 -5747> 2014-11-03 11:37:51.644153 7f290001b700 5 -- op tracker -- seq: 2995, time: 2014-11-03 11:37:51.644153, event: done, op: osd_op(client.9031502.0:167557506 rbd_data.89cd895a911542.0000000000006254 [read 1114112~16384] 6.71b262b ack+read e105950) -5746> 2014-11-03 11:37:51.645444 7f28c4099700 1 -- 192.168.5.2:6802/7606 <== client.9031502 192.168.1.83:0/3009793 2 ==== osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) v4 ==== 191+0+0 (2493376864 0 0) 0x235dfc80 con 0xa26c2c0 -5745> 2014-11-03 11:37:51.645485 7f28c4099700 5 -- op tracker -- seq: 2996, time: 2014-11-03 11:37:51.645358, event: header_read, op: osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) -5744> 2014-11-03 11:37:51.645504 7f28c4099700 5 -- op tracker -- seq: 2996, time: 2014-11-03 11:37:51.645361, event: throttled, op: osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) -5743> 2014-11-03 11:37:51.645518 7f28c4099700 5 -- op tracker -- seq: 2996, time: 2014-11-03 11:37:51.645432, event: all_read, op: osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) -5742> 2014-11-03 11:37:51.645533 7f28c4099700 5 -- op tracker -- seq: 2996, time: 0.000000, event: dispatched, op: osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) -5741> 2014-11-03 11:37:51.645657 7f28fd816700 5 -- op tracker -- seq: 2996, time: 2014-11-03 11:37:51.645657, event: reached_pg, op: osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) -5740> 2014-11-03 11:37:51.645771 7f28fd816700 5 -- op tracker -- seq: 2996, time: 2014-11-03 11:37:51.645771, event: started, op: osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) -5739> 2014-11-03 11:37:51.645822 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.83:0/3009793 -- osd_op_reply(167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] v0'0 uv14082900 ondisk = 0) v6 -- ?+0 0x23c1d340 con 0xa26c2c0 -5738> 2014-11-03 11:37:51.645865 7f28fd816700 5 -- op tracker -- seq: 2996, time: 2014-11-03 11:37:51.645865, event: done, op: osd_op(client.9031502.0:167557507 rbd_data.89cd895a911542.0000000000006254 [read 1228800~4096] 6.71b262b ack+read e105950) -5737> 2014-11-03 11:37:51.660359 7f28f16a2700 1 -- 192.168.5.2:6803/7606 <== osd.110 192.168.5.8:6848/29518 49 ==== osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (134654931 0 3511520646) 0x246ff400 con 0x230009a0 -5736> 2014-11-03 11:37:51.660417 7f28f16a2700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.660201, event: header_read, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5735> 2014-11-03 11:37:51.660444 7f28f16a2700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.660205, event: throttled, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5734> 2014-11-03 11:37:51.660464 7f28f16a2700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.660338, event: all_read, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5733> 2014-11-03 11:37:51.660508 7f28f16a2700 5 -- op tracker -- seq: 2997, time: 0.000000, event: dispatched, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5732> 2014-11-03 11:37:51.660674 7f28ff81a700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.660674, event: reached_pg, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5731> 2014-11-03 11:37:51.660792 7f28ff81a700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.660791, event: started, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5730> 2014-11-03 11:37:51.660964 7f28ff81a700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.660964, event: started, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5729> 2014-11-03 11:37:51.661003 7f28ff81a700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.661002, event: commit_queued_for_journal_write, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5728> 2014-11-03 11:37:51.661091 7f2914d0c700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.661091, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5727> 2014-11-03 11:37:51.661553 7f291450b700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.661552, event: journaled_completion_queued, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5726> 2014-11-03 11:37:51.661600 7f2911d06700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.661599, event: commit_sent, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5725> 2014-11-03 11:37:51.661636 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6848/29518 -- osd_sub_op_reply(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2fc9ab00 con 0x230009a0 -5724> 2014-11-03 11:37:51.661710 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.102 192.168.5.8:0/21153 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.660087) v2 ==== 47+0+0 (362421117 0 0) 0x236c0400 con 0x233cb580 -5723> 2014-11-03 11:37:51.661747 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.660087) v2 -- ?+0 0x2353b000 con 0x233cb580 -5722> 2014-11-03 11:37:51.661714 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.102 192.168.5.8:0/21153 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.660087) v2 ==== 47+0+0 (362421117 0 0) 0x253e5400 con 0x233cb160 -5721> 2014-11-03 11:37:51.661849 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.660087) v2 -- ?+0 0x29fcd800 con 0x233cb160 -5720> 2014-11-03 11:37:51.661876 7f2912507700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.661875, event: sub_op_applied, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5719> 2014-11-03 11:37:51.661956 7f2912507700 5 -- op tracker -- seq: 2997, time: 2014-11-03 11:37:51.661956, event: done, op: osd_sub_op(client.8348783.0:3931952 24.636 d1bdbe36/rbd_data.48f8972ae8944a.0000000000001b2b/head//24 [] v 105950'204614 snapset=0=[]:[] snapc=0=[]) -5718> 2014-11-03 11:37:51.709558 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x2340d800 con 0x231abb00 -5717> 2014-11-03 11:37:51.744788 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 87 ==== osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1279247002 0 3511520646) 0x27813400 con 0x22ff3b00 -5716> 2014-11-03 11:37:51.744846 7f28f0d99700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.744436, event: header_read, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5715> 2014-11-03 11:37:51.744875 7f28f0d99700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.744439, event: throttled, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5714> 2014-11-03 11:37:51.744895 7f28f0d99700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.744769, event: all_read, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5713> 2014-11-03 11:37:51.744915 7f28f0d99700 5 -- op tracker -- seq: 2998, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5712> 2014-11-03 11:37:51.744982 7f290001b700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.744982, event: reached_pg, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5711> 2014-11-03 11:37:51.745020 7f290001b700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.745019, event: started, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5710> 2014-11-03 11:37:51.745145 7f290001b700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.745144, event: started, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5709> 2014-11-03 11:37:51.745177 7f290001b700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.745177, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5708> 2014-11-03 11:37:51.745263 7f2914d0c700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.745263, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5707> 2014-11-03 11:37:51.749707 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.108 192.168.5.8:0/21567 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.748210) v2 ==== 47+0+0 (4124397189 0 0) 0x286bb000 con 0x243212c0 -5706> 2014-11-03 11:37:51.749754 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.748210) v2 -- ?+0 0x236c0400 con 0x243212c0 -5705> 2014-11-03 11:37:51.749764 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.108 192.168.5.8:0/21567 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.748210) v2 ==== 47+0+0 (4124397189 0 0) 0x1106c200 con 0x2484a840 -5704> 2014-11-03 11:37:51.749918 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.748210) v2 -- ?+0 0x253e5400 con 0x2484a840 -5703> 2014-11-03 11:37:51.754708 7f291450b700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.754708, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5702> 2014-11-03 11:37:51.754851 7f2911d06700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.754839, event: commit_sent, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5701> 2014-11-03 11:37:51.754953 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x9fff580 con 0x22ff3b00 -5700> 2014-11-03 11:37:51.755319 7f2912507700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.755318, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5699> 2014-11-03 11:37:51.755398 7f2912507700 5 -- op tracker -- seq: 2998, time: 2014-11-03 11:37:51.755398, event: done, op: osd_sub_op(client.12146205.0:439303 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222066 snapset=0=[]:[] snapc=0=[]) -5698> 2014-11-03 11:37:51.762934 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.48 192.168.5.4:0/17727 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.762060) v2 ==== 47+0+0 (2412812811 0 0) 0x23482800 con 0x23335420 -5697> 2014-11-03 11:37:51.762941 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.48 192.168.5.4:0/17727 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.762060) v2 ==== 47+0+0 (2412812811 0 0) 0x23576200 con 0x234ed420 -5696> 2014-11-03 11:37:51.763051 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/17727 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.762060) v2 -- ?+0 0x1106c200 con 0x23335420 -5695> 2014-11-03 11:37:51.763126 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/17727 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.762060) v2 -- ?+0 0x286bb000 con 0x234ed420 -5694> 2014-11-03 11:37:51.767568 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 30 ==== MOSDPGPush(6.58e 105950 [PushOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, version: 105946'11622298, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (3906688086 0 0) 0x2340d800 con 0x231abb00 -5693> 2014-11-03 11:37:51.767615 7f28eb945700 5 -- op tracker -- seq: 2999, time: 2014-11-03 11:37:51.753647, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, version: 105946'11622298, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5692> 2014-11-03 11:37:51.767643 7f28eb945700 5 -- op tracker -- seq: 2999, time: 2014-11-03 11:37:51.753650, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, version: 105946'11622298, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5691> 2014-11-03 11:37:51.767662 7f28eb945700 5 -- op tracker -- seq: 2999, time: 2014-11-03 11:37:51.767554, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, version: 105946'11622298, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5690> 2014-11-03 11:37:51.767679 7f28eb945700 5 -- op tracker -- seq: 2999, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, version: 105946'11622298, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5689> 2014-11-03 11:37:51.767744 7f28fe818700 5 -- op tracker -- seq: 2999, time: 2014-11-03 11:37:51.767743, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, version: 105946'11622298, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5688> 2014-11-03 11:37:51.768024 7f28fe818700 5 -- op tracker -- seq: 2999, time: 2014-11-03 11:37:51.768024, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6, version: 105946'11622298, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6c44058e/rbd_data.4eeda241ca92c8.000000000000380e/head//6@105946'11622298, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5687> 2014-11-03 11:37:51.783495 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 97 ==== osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2601157268 0 3511520646) 0x2398e400 con 0x22fbcb00 -5686> 2014-11-03 11:37:51.783556 7f28f6cf8700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.783212, event: header_read, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5685> 2014-11-03 11:37:51.783599 7f28f6cf8700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.783215, event: throttled, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5684> 2014-11-03 11:37:51.783612 7f28f6cf8700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.783470, event: all_read, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5683> 2014-11-03 11:37:51.783625 7f28f6cf8700 5 -- op tracker -- seq: 3000, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5682> 2014-11-03 11:37:51.783683 7f28fe017700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.783683, event: reached_pg, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5681> 2014-11-03 11:37:51.783717 7f28fe017700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.783717, event: started, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5680> 2014-11-03 11:37:51.783823 7f28fe017700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.783823, event: started, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5679> 2014-11-03 11:37:51.783853 7f28fe017700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.783853, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5678> 2014-11-03 11:37:51.790572 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.92 192.168.5.7:0/5438 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.788508) v2 ==== 47+0+0 (2168545410 0 0) 0x24800200 con 0x239f8840 -5677> 2014-11-03 11:37:51.790626 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/5438 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.788508) v2 -- ?+0 0x23482800 con 0x239f8840 -5676> 2014-11-03 11:37:51.790603 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.92 192.168.5.7:0/5438 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.788508) v2 ==== 47+0+0 (2168545410 0 0) 0x2782f200 con 0x238b89a0 -5675> 2014-11-03 11:37:51.790663 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/5438 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.788508) v2 -- ?+0 0x23576200 con 0x238b89a0 -5674> 2014-11-03 11:37:51.791211 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 29 ==== osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (3101328694 0 3511520646) 0x26151c00 con 0x22fe0840 -5673> 2014-11-03 11:37:51.791242 7f28f4cd8700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.791094, event: header_read, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5672> 2014-11-03 11:37:51.791261 7f28f4cd8700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.791096, event: throttled, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5671> 2014-11-03 11:37:51.791273 7f28f4cd8700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.791201, event: all_read, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5670> 2014-11-03 11:37:51.791286 7f28f4cd8700 5 -- op tracker -- seq: 3001, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5669> 2014-11-03 11:37:51.791339 7f290081c700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.791339, event: reached_pg, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5668> 2014-11-03 11:37:51.791370 7f290081c700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.791370, event: started, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5667> 2014-11-03 11:37:51.791471 7f290081c700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.791470, event: started, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5666> 2014-11-03 11:37:51.791500 7f290081c700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.791500, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5665> 2014-11-03 11:37:51.808324 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 85 ==== osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2867550919 0 3511520646) 0x27806c00 con 0x22fef420 -5664> 2014-11-03 11:37:51.808377 7f28f41cd700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.808090, event: header_read, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5663> 2014-11-03 11:37:51.808399 7f28f41cd700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.808093, event: throttled, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5662> 2014-11-03 11:37:51.808412 7f28f41cd700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.808307, event: all_read, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5661> 2014-11-03 11:37:51.808421 7f28f41cd700 5 -- op tracker -- seq: 3002, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5660> 2014-11-03 11:37:51.808523 7f290101d700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.808523, event: reached_pg, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5659> 2014-11-03 11:37:51.808573 7f290101d700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.808572, event: started, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5658> 2014-11-03 11:37:51.808719 7f290101d700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.808719, event: started, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5657> 2014-11-03 11:37:51.808771 7f290101d700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.808771, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5656> 2014-11-03 11:37:51.809798 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 86 ==== osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (607821849 0 3511520646) 0x29e1f400 con 0x22fef420 -5655> 2014-11-03 11:37:51.809852 7f28f41cd700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.809649, event: header_read, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5654> 2014-11-03 11:37:51.809869 7f28f41cd700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.809651, event: throttled, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5653> 2014-11-03 11:37:51.809880 7f28f41cd700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.809784, event: all_read, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5652> 2014-11-03 11:37:51.809891 7f28f41cd700 5 -- op tracker -- seq: 3003, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5651> 2014-11-03 11:37:51.809932 7f28fe818700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.809931, event: reached_pg, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5650> 2014-11-03 11:37:51.809954 7f28fe818700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.809954, event: started, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5649> 2014-11-03 11:37:51.810032 7f28fe818700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.810032, event: started, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5648> 2014-11-03 11:37:51.810058 7f28fe818700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.810058, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5647> 2014-11-03 11:37:51.810749 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.10 192.168.5.1:0/15344 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.810190) v2 ==== 47+0+0 (3086252369 0 0) 0x253e5600 con 0x2469edc0 -5646> 2014-11-03 11:37:51.810788 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/15344 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.810190) v2 -- ?+0 0x2782f200 con 0x2469edc0 -5645> 2014-11-03 11:37:51.810814 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.10 192.168.5.1:0/15344 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.810190) v2 ==== 47+0+0 (3086252369 0 0) 0x23299e00 con 0x240ad000 -5644> 2014-11-03 11:37:51.810897 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/15344 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.810190) v2 -- ?+0 0x24800200 con 0x240ad000 -5643> 2014-11-03 11:37:51.823493 7f2914d0c700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.823493, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5642> 2014-11-03 11:37:51.823535 7f2914d0c700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.823535, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5641> 2014-11-03 11:37:51.823560 7f2914d0c700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.823560, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5640> 2014-11-03 11:37:51.823585 7f2914d0c700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.823585, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5639> 2014-11-03 11:37:51.823844 7f291450b700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.823844, event: journaled_completion_queued, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5638> 2014-11-03 11:37:51.823873 7f2911d06700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.823873, event: commit_sent, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5637> 2014-11-03 11:37:51.823879 7f291450b700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.823879, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5636> 2014-11-03 11:37:51.823917 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] ondisk, result = 0) v2 -- ?+0 0x9fff000 con 0x22fbcb00 -5635> 2014-11-03 11:37:51.823955 7f291450b700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.823955, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5634> 2014-11-03 11:37:51.823969 7f2911d06700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.823969, event: commit_sent, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5633> 2014-11-03 11:37:51.823979 7f291450b700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.823979, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5632> 2014-11-03 11:37:51.823987 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x9fff2c0 con 0x22fe0840 -5631> 2014-11-03 11:37:51.824020 7f2911d06700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.824020, event: commit_sent, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5630> 2014-11-03 11:37:51.824041 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] ondisk, result = 0) v2 -- ?+0 0x9fff840 con 0x22fef420 -5629> 2014-11-03 11:37:51.824063 7f2911d06700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.824063, event: commit_sent, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5628> 2014-11-03 11:37:51.824081 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa000b80 con 0x22fef420 -5627> 2014-11-03 11:37:51.824260 7f2912507700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.824260, event: sub_op_applied, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5626> 2014-11-03 11:37:51.824311 7f2912507700 5 -- op tracker -- seq: 3000, time: 2014-11-03 11:37:51.824310, event: done, op: osd_sub_op(client.7869446.0:2442769 6.576 27e6c576/rbd_data.4952b62ae8944a.0000000000000149/head//6 [] v 105950'17262140 snapset=0=[]:[] snapc=0=[]) -5625> 2014-11-03 11:37:51.824505 7f2912507700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.824505, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5624> 2014-11-03 11:37:51.824550 7f2912507700 5 -- op tracker -- seq: 3001, time: 2014-11-03 11:37:51.824550, event: done, op: osd_sub_op(client.7867942.0:137197710 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960079 snapset=0=[]:[] snapc=0=[]) -5623> 2014-11-03 11:37:51.824707 7f2912507700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.824707, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5622> 2014-11-03 11:37:51.824756 7f2912507700 5 -- op tracker -- seq: 3002, time: 2014-11-03 11:37:51.824756, event: done, op: osd_sub_op(client.7914641.0:33713517 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057565 snapset=0=[]:[] snapc=0=[]) -5621> 2014-11-03 11:37:51.825200 7f2912507700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.825200, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5620> 2014-11-03 11:37:51.825230 7f2912507700 5 -- op tracker -- seq: 3003, time: 2014-11-03 11:37:51.825230, event: done, op: osd_sub_op(client.7914641.0:33713518 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057566 snapset=0=[]:[] snapc=0=[]) -5619> 2014-11-03 11:37:51.829378 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x17cac800 con 0x22fbc420 -5618> 2014-11-03 11:37:51.835058 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.65 192.168.5.5:0/30042 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.833374) v2 ==== 47+0+0 (2747501129 0 0) 0x54a3e00 con 0x249256e0 -5617> 2014-11-03 11:37:51.835118 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/30042 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.833374) v2 -- ?+0 0x23299e00 con 0x249256e0 -5616> 2014-11-03 11:37:51.835155 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.65 192.168.5.5:0/30042 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.833374) v2 ==== 47+0+0 (2747501129 0 0) 0x24139600 con 0x24229420 -5615> 2014-11-03 11:37:51.835197 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/30042 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.833374) v2 -- ?+0 0x253e5600 con 0x24229420 -5614> 2014-11-03 11:37:51.865841 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 51 ==== osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3720366747 0 3511520646) 0x27812800 con 0x2301fb00 -5613> 2014-11-03 11:37:51.865893 7f28f31bd700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.865659, event: header_read, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5612> 2014-11-03 11:37:51.865908 7f28f31bd700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.865664, event: throttled, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5611> 2014-11-03 11:37:51.865922 7f28f31bd700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.865828, event: all_read, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5610> 2014-11-03 11:37:51.865939 7f28f31bd700 5 -- op tracker -- seq: 3004, time: 0.000000, event: dispatched, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5609> 2014-11-03 11:37:51.866015 7f290101d700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.866015, event: reached_pg, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5608> 2014-11-03 11:37:51.866051 7f290101d700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.866051, event: started, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5607> 2014-11-03 11:37:51.866257 7f290101d700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.866257, event: started, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5606> 2014-11-03 11:37:51.866336 7f290101d700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.866336, event: commit_queued_for_journal_write, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5605> 2014-11-03 11:37:51.866474 7f2914d0c700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.866474, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5604> 2014-11-03 11:37:51.867096 7f291450b700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.867096, event: journaled_completion_queued, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5603> 2014-11-03 11:37:51.867173 7f2911d06700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.867173, event: commit_sent, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5602> 2014-11-03 11:37:51.867200 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] ondisk, result = 0) v2 -- ?+0 0xa000340 con 0x2301fb00 -5601> 2014-11-03 11:37:51.867583 7f2912507700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.867582, event: sub_op_applied, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5600> 2014-11-03 11:37:51.867662 7f2912507700 5 -- op tracker -- seq: 3004, time: 2014-11-03 11:37:51.867661, event: done, op: osd_sub_op(client.8393371.0:2616620 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027708 snapset=0=[]:[] snapc=0=[]) -5599> 2014-11-03 11:37:51.871347 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 48 ==== osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2759102959 0 3511520646) 0x295a4400 con 0x230a9b00 -5598> 2014-11-03 11:37:51.871383 7f28f038f700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871226, event: header_read, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5597> 2014-11-03 11:37:51.871398 7f28f038f700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871228, event: throttled, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5596> 2014-11-03 11:37:51.871411 7f28f038f700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871332, event: all_read, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5595> 2014-11-03 11:37:51.871424 7f28f038f700 5 -- op tracker -- seq: 3005, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5594> 2014-11-03 11:37:51.871483 7f28fe818700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871483, event: reached_pg, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5593> 2014-11-03 11:37:51.871516 7f28fe818700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871516, event: started, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5592> 2014-11-03 11:37:51.871649 7f28fe818700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871649, event: started, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5591> 2014-11-03 11:37:51.871680 7f28fe818700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871680, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5590> 2014-11-03 11:37:51.871787 7f2914d0c700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.871787, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5589> 2014-11-03 11:37:51.873132 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 49 ==== osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2170185375 0 3511520646) 0x29e1e800 con 0x230a9b00 -5588> 2014-11-03 11:37:51.873178 7f28f038f700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.873043, event: header_read, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5587> 2014-11-03 11:37:51.873195 7f28f038f700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.873044, event: throttled, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5586> 2014-11-03 11:37:51.873203 7f28f038f700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.873123, event: all_read, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5585> 2014-11-03 11:37:51.873215 7f28f038f700 5 -- op tracker -- seq: 3006, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5584> 2014-11-03 11:37:51.873322 7f290101d700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.873322, event: reached_pg, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5583> 2014-11-03 11:37:51.873363 7f290101d700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.873363, event: started, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5582> 2014-11-03 11:37:51.873470 7f290101d700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.873470, event: started, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5581> 2014-11-03 11:37:51.873507 7f290101d700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.873506, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5580> 2014-11-03 11:37:51.874649 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 50 ==== osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2973356974 0 3511520646) 0x26151c00 con 0x230a9b00 -5579> 2014-11-03 11:37:51.874680 7f28f038f700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.874565, event: header_read, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5578> 2014-11-03 11:37:51.874694 7f28f038f700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.874570, event: throttled, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5577> 2014-11-03 11:37:51.874702 7f28f038f700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.874639, event: all_read, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5576> 2014-11-03 11:37:51.874710 7f28f038f700 5 -- op tracker -- seq: 3007, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5575> 2014-11-03 11:37:51.874756 7f28fe818700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.874756, event: reached_pg, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5574> 2014-11-03 11:37:51.874784 7f28fe818700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.874784, event: started, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5573> 2014-11-03 11:37:51.874872 7f28fe818700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.874872, event: started, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5572> 2014-11-03 11:37:51.874902 7f28fe818700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.874902, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5571> 2014-11-03 11:37:51.876114 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 51 ==== osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2495452740 0 3511520646) 0x2398e400 con 0x230a9b00 -5570> 2014-11-03 11:37:51.876150 7f28f038f700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.876026, event: header_read, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5569> 2014-11-03 11:37:51.876164 7f28f038f700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.876028, event: throttled, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5568> 2014-11-03 11:37:51.876172 7f28f038f700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.876104, event: all_read, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5567> 2014-11-03 11:37:51.876180 7f28f038f700 5 -- op tracker -- seq: 3008, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5566> 2014-11-03 11:37:51.876270 7f290101d700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.876270, event: reached_pg, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5565> 2014-11-03 11:37:51.876309 7f290101d700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.876309, event: started, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5564> 2014-11-03 11:37:51.876414 7f290101d700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.876414, event: started, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5563> 2014-11-03 11:37:51.876450 7f290101d700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.876450, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5562> 2014-11-03 11:37:51.877716 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 52 ==== osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (156204314 0 3511520646) 0x27813400 con 0x230a9b00 -5561> 2014-11-03 11:37:51.877747 7f28f038f700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.877620, event: header_read, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5560> 2014-11-03 11:37:51.877761 7f28f038f700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.877622, event: throttled, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5559> 2014-11-03 11:37:51.877773 7f28f038f700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.877706, event: all_read, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5558> 2014-11-03 11:37:51.877780 7f28f038f700 5 -- op tracker -- seq: 3009, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5557> 2014-11-03 11:37:51.877820 7f28fe818700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.877820, event: reached_pg, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5556> 2014-11-03 11:37:51.877852 7f28fe818700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.877852, event: started, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5555> 2014-11-03 11:37:51.877932 7f28fe818700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.877932, event: started, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5554> 2014-11-03 11:37:51.877965 7f28fe818700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.877965, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5553> 2014-11-03 11:37:51.878747 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.103 192.168.5.8:0/29722 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.877831) v2 ==== 47+0+0 (2674696895 0 0) 0x2d6ec400 con 0x239f8c60 -5552> 2014-11-03 11:37:51.878806 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/29722 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.877831) v2 -- ?+0 0x24139600 con 0x239f8c60 -5551> 2014-11-03 11:37:51.878803 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.103 192.168.5.8:0/29722 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:51.877831) v2 ==== 47+0+0 (2674696895 0 0) 0x2a2e0c00 con 0x23b1cc60 -5550> 2014-11-03 11:37:51.878860 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/29722 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:51.877831) v2 -- ?+0 0x54a3e00 con 0x23b1cc60 -5549> 2014-11-03 11:37:51.895434 7f291450b700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.895434, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5548> 2014-11-03 11:37:51.895505 7f2911d06700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.895505, event: commit_sent, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5547> 2014-11-03 11:37:51.895507 7f2914d0c700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.895506, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5546> 2014-11-03 11:37:51.895553 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa000600 con 0x230a9b00 -5545> 2014-11-03 11:37:51.895577 7f2914d0c700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.895577, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5544> 2014-11-03 11:37:51.895620 7f2914d0c700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.895620, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5543> 2014-11-03 11:37:51.895674 7f2914d0c700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.895674, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5542> 2014-11-03 11:37:51.896021 7f291450b700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.896020, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5541> 2014-11-03 11:37:51.896050 7f291450b700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.896050, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5540> 2014-11-03 11:37:51.896070 7f291450b700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.896070, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5539> 2014-11-03 11:37:51.896087 7f291450b700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.896087, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5538> 2014-11-03 11:37:51.896121 7f2911d06700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.896121, event: commit_sent, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5537> 2014-11-03 11:37:51.896156 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x236132c0 con 0x230a9b00 -5536> 2014-11-03 11:37:51.896180 7f2911d06700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.896180, event: commit_sent, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5535> 2014-11-03 11:37:51.896281 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23613b00 con 0x230a9b00 -5534> 2014-11-03 11:37:51.896328 7f2911d06700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.896327, event: commit_sent, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5533> 2014-11-03 11:37:51.896444 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23613580 con 0x230a9b00 -5532> 2014-11-03 11:37:51.896488 7f2911d06700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.896488, event: commit_sent, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5531> 2014-11-03 11:37:51.896586 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23613000 con 0x230a9b00 -5530> 2014-11-03 11:37:51.896701 7f2912507700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.896701, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5529> 2014-11-03 11:37:51.896769 7f2912507700 5 -- op tracker -- seq: 3005, time: 2014-11-03 11:37:51.896769, event: done, op: osd_sub_op(client.7869116.0:21661157 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850151 snapset=0=[]:[] snapc=0=[]) -5528> 2014-11-03 11:37:51.896846 7f2912507700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.896846, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5527> 2014-11-03 11:37:51.896869 7f2912507700 5 -- op tracker -- seq: 3006, time: 2014-11-03 11:37:51.896869, event: done, op: osd_sub_op(client.7869116.0:21661158 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850152 snapset=0=[]:[] snapc=0=[]) -5526> 2014-11-03 11:37:51.896942 7f2912507700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.896941, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5525> 2014-11-03 11:37:51.896978 7f2912507700 5 -- op tracker -- seq: 3007, time: 2014-11-03 11:37:51.896978, event: done, op: osd_sub_op(client.7869116.0:21661159 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850153 snapset=0=[]:[] snapc=0=[]) -5524> 2014-11-03 11:37:51.897072 7f2912507700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.897071, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5523> 2014-11-03 11:37:51.897122 7f2912507700 5 -- op tracker -- seq: 3008, time: 2014-11-03 11:37:51.897122, event: done, op: osd_sub_op(client.7869116.0:21661160 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850154 snapset=0=[]:[] snapc=0=[]) -5522> 2014-11-03 11:37:51.897369 7f2912507700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.897369, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5521> 2014-11-03 11:37:51.897441 7f2912507700 5 -- op tracker -- seq: 3009, time: 2014-11-03 11:37:51.897441, event: done, op: osd_sub_op(client.7869116.0:21661161 6.458 ab945c58/rbd_data.4fd1a75007e195.000000000000071c/head//6 [] v 105950'17850155 snapset=0=[]:[] snapc=0=[]) -5520> 2014-11-03 11:37:51.901782 7f290c950700 1 -- 192.168.5.2:6802/7606 <== client.8912789 192.168.1.93:0/1006831 2 ==== osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) v4 ==== 229+0+4096 (2184815059 0 1379546969) 0x235df000 con 0x28289160 -5519> 2014-11-03 11:37:51.901841 7f290c950700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.901620, event: header_read, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5518> 2014-11-03 11:37:51.901881 7f290c950700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.901626, event: throttled, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5517> 2014-11-03 11:37:51.901896 7f290c950700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.901761, event: all_read, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5516> 2014-11-03 11:37:51.901919 7f290c950700 5 -- op tracker -- seq: 3010, time: 0.000000, event: dispatched, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5515> 2014-11-03 11:37:51.902106 7f290101d700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.902106, event: reached_pg, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5514> 2014-11-03 11:37:51.902558 7f290101d700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.902558, event: started, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5513> 2014-11-03 11:37:51.902619 7f290101d700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.902619, event: started, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5512> 2014-11-03 11:37:51.902732 7f290101d700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.902732, event: waiting for subops from 7,43, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5511> 2014-11-03 11:37:51.902814 7f290101d700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] v 105950'11622402 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4921 0x2d6e7800 con 0x22fbc420 -5510> 2014-11-03 11:37:51.902980 7f290101d700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- osd_sub_op(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] v 105950'11622402 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4921 0x2c337800 con 0x231abb00 -5509> 2014-11-03 11:37:51.903169 7f290101d700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.903169, event: commit_queued_for_journal_write, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5508> 2014-11-03 11:37:51.903362 7f2914d0c700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.903362, event: write_thread_in_journal_buffer, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5507> 2014-11-03 11:37:51.903688 7f291450b700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.903687, event: journaled_completion_queued, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5506> 2014-11-03 11:37:51.903824 7f2911d06700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.903824, event: op_commit, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5505> 2014-11-03 11:37:51.904206 7f2912507700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.904206, event: op_applied, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5504> 2014-11-03 11:37:51.905168 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 31 ==== osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) v2 ==== 178+0+0 (3417515817 0 0) 0x279fb000 con 0x231abb00 -5503> 2014-11-03 11:37:51.905218 7f28eb945700 5 -- op tracker -- seq: 3011, time: 2014-11-03 11:37:51.905033, event: header_read, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5502> 2014-11-03 11:37:51.905267 7f28eb945700 5 -- op tracker -- seq: 3011, time: 2014-11-03 11:37:51.905034, event: throttled, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5501> 2014-11-03 11:37:51.905281 7f28eb945700 5 -- op tracker -- seq: 3011, time: 2014-11-03 11:37:51.905149, event: all_read, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5500> 2014-11-03 11:37:51.905289 7f28eb945700 5 -- op tracker -- seq: 3011, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5499> 2014-11-03 11:37:51.905409 7f28fe818700 5 -- op tracker -- seq: 3011, time: 2014-11-03 11:37:51.905409, event: reached_pg, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5498> 2014-11-03 11:37:51.905457 7f28fe818700 5 -- op tracker -- seq: 3011, time: 2014-11-03 11:37:51.905457, event: started, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5497> 2014-11-03 11:37:51.905481 7f28fe818700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.905481, event: sub_op_commit_rec, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5496> 2014-11-03 11:37:51.905518 7f28fe818700 5 -- op tracker -- seq: 3011, time: 2014-11-03 11:37:51.905518, event: done, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5495> 2014-11-03 11:37:51.941647 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 94 ==== MOSDPGPush(6.58e 105950 [PushOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, version: 105946'11622314, data_included: [0~3472896], data_size: 3472896, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~3472896], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3472896, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 3473802+0+0 (3425661156 0 0) 0x17cac800 con 0x22fbc420 -5494> 2014-11-03 11:37:51.941694 7f28f72fe700 5 -- op tracker -- seq: 3012, time: 2014-11-03 11:37:51.931308, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, version: 105946'11622314, data_included: [0~3472896], data_size: 3472896, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~3472896], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3472896, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5493> 2014-11-03 11:37:51.941715 7f28f72fe700 5 -- op tracker -- seq: 3012, time: 2014-11-03 11:37:51.931309, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, version: 105946'11622314, data_included: [0~3472896], data_size: 3472896, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~3472896], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3472896, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5492> 2014-11-03 11:37:51.941730 7f28f72fe700 5 -- op tracker -- seq: 3012, time: 2014-11-03 11:37:51.941631, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, version: 105946'11622314, data_included: [0~3472896], data_size: 3472896, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~3472896], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3472896, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5491> 2014-11-03 11:37:51.941759 7f28f72fe700 5 -- op tracker -- seq: 3012, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, version: 105946'11622314, data_included: [0~3472896], data_size: 3472896, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~3472896], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3472896, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5490> 2014-11-03 11:37:51.941860 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 95 ==== osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) v2 ==== 178+0+0 (2629050060 0 0) 0x2dfbadc0 con 0x22fbc420 -5489> 2014-11-03 11:37:51.941884 7f28f72fe700 5 -- op tracker -- seq: 3013, time: 2014-11-03 11:37:51.941805, event: header_read, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5488> 2014-11-03 11:37:51.941896 7f28f72fe700 5 -- op tracker -- seq: 3013, time: 2014-11-03 11:37:51.941806, event: throttled, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5487> 2014-11-03 11:37:51.941903 7f28f72fe700 5 -- op tracker -- seq: 3013, time: 2014-11-03 11:37:51.941852, event: all_read, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5486> 2014-11-03 11:37:51.941923 7f28f72fe700 5 -- op tracker -- seq: 3013, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5485> 2014-11-03 11:37:51.941863 7f290101d700 5 -- op tracker -- seq: 3012, time: 2014-11-03 11:37:51.941862, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, version: 105946'11622314, data_included: [0~3472896], data_size: 3472896, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~3472896], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3472896, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5484> 2014-11-03 11:37:51.942197 7f28fe818700 5 -- op tracker -- seq: 3013, time: 2014-11-03 11:37:51.942197, event: reached_pg, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5483> 2014-11-03 11:37:51.942237 7f28fe818700 5 -- op tracker -- seq: 3013, time: 2014-11-03 11:37:51.942237, event: started, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5482> 2014-11-03 11:37:51.942176 7f290101d700 5 -- op tracker -- seq: 3012, time: 2014-11-03 11:37:51.942176, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6, version: 105946'11622314, data_included: [0~3472896], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b53a658e/rbd_data.87fc242ae8944a.00000000000009ca/head//6@105946'11622314, copy_subset: [0~3472896], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3472896, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5481> 2014-11-03 11:37:51.942268 7f28fe818700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.942268, event: sub_op_commit_rec, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5480> 2014-11-03 11:37:51.942291 7f28fe818700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.93:0/1006831 -- osd_op_reply(665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] v105950'11622402 uv11622402 ack = 0) v6 -- ?+0 0x3110d580 con 0x28289160 -5479> 2014-11-03 11:37:51.942343 7f28fe818700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.93:0/1006831 -- osd_op_reply(665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] v105950'11622402 uv11622402 ondisk = 0) v6 -- ?+0 0x279fb000 con 0x28289160 -5478> 2014-11-03 11:37:51.942476 7f28fe818700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.942476, event: commit_sent, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5477> 2014-11-03 11:37:51.942532 7f28fe818700 5 -- op tracker -- seq: 3010, time: 2014-11-03 11:37:51.942532, event: done, op: osd_op(client.8912789.0:665742 rbd_data.87a6a72ae8944a.000000000000009f [set-alloc-hint object_size 4194304 write_size 4194304,write 1265152~4096] 6.88cec58e ack+ondisk+write e105950) -5476> 2014-11-03 11:37:51.942569 7f28fe818700 5 -- op tracker -- seq: 3013, time: 2014-11-03 11:37:51.942569, event: done, op: osd_sub_op_reply(client.8912789.0:665742 6.58e 88cec58e/rbd_data.87a6a72ae8944a.000000000000009f/head//6 [] ondisk, result = 0) -5475> 2014-11-03 11:37:51.994348 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x316d2b80 sd=239 :6802 s=0 pgs=0 cs=0 l=0 c=0xa26c160).accept sd=239 192.168.5.2:42904/0 -5474> 2014-11-03 11:37:51.995628 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450728 192.168.5.2:0/1026484 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x24800200 con 0xa26c160 -5473> 2014-11-03 11:37:51.996804 7f28fb812700 0 osd.21 105950 do_command r=0 -5472> 2014-11-03 11:37:51.996833 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1026484 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x2dcb0c40 con 0xa26c160 -5471> 2014-11-03 11:37:51.999512 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 77 ==== osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (446587309 0 3511520646) 0x191b7400 con 0x230c2000 -5470> 2014-11-03 11:37:51.999547 7f28ebc48700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999333, event: header_read, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5469> 2014-11-03 11:37:51.999562 7f28ebc48700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999336, event: throttled, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5468> 2014-11-03 11:37:51.999571 7f28ebc48700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999496, event: all_read, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5467> 2014-11-03 11:37:51.999581 7f28ebc48700 5 -- op tracker -- seq: 3014, time: 0.000000, event: dispatched, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5466> 2014-11-03 11:37:51.999657 7f290101d700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999657, event: reached_pg, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5465> 2014-11-03 11:37:51.999699 7f290101d700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999699, event: started, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5464> 2014-11-03 11:37:51.999811 7f290101d700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999811, event: started, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5463> 2014-11-03 11:37:51.999848 7f290101d700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999848, event: commit_queued_for_journal_write, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5462> 2014-11-03 11:37:51.999956 7f2914d0c700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:51.999956, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5461> 2014-11-03 11:37:52.000462 7f291450b700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:52.000462, event: journaled_completion_queued, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5460> 2014-11-03 11:37:52.000547 7f2911d06700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:52.000547, event: commit_sent, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5459> 2014-11-03 11:37:52.000579 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op_reply(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23613840 con 0x230c2000 -5458> 2014-11-03 11:37:52.001154 7f2912507700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:52.001154, event: sub_op_applied, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5457> 2014-11-03 11:37:52.001217 7f2912507700 5 -- op tracker -- seq: 3014, time: 2014-11-03 11:37:52.001217, event: done, op: osd_sub_op(client.8421911.0:15704400 6.773 82cf2773/rbd_data.805e322ae8944a.0000000000002125/head//6 [] v 105950'12166719 snapset=0=[]:[] snapc=0=[]) -5456> 2014-11-03 11:37:52.001345 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.35 192.168.5.3:0/6455 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.000651) v2 ==== 47+0+0 (1896749194 0 0) 0x2be31600 con 0x24321160 -5455> 2014-11-03 11:37:52.001398 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.000651) v2 -- ?+0 0x2d6ec400 con 0x24321160 -5454> 2014-11-03 11:37:52.001557 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.35 192.168.5.3:0/6455 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.000651) v2 ==== 47+0+0 (1896749194 0 0) 0x229e7800 con 0x2484a6e0 -5453> 2014-11-03 11:37:52.001616 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.000651) v2 -- ?+0 0x2a2e0c00 con 0x2484a6e0 -5452> 2014-11-03 11:37:52.008225 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450728 192.168.5.2:0/1026484 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x259f3400 con 0xa26c160 -5451> 2014-11-03 11:37:52.009086 7f28fb812700 0 osd.21 105950 do_command r=0 -5450> 2014-11-03 11:37:52.009103 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1026484 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0xfc73000 con 0xa26c160 -5449> 2014-11-03 11:37:52.016583 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1026484 pipe(0x316d2b80 sd=239 :6802 s=2 pgs=3 cs=1 l=1 c=0xa26c160).reader couldn't read tag, (0) Success -5448> 2014-11-03 11:37:52.016632 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1026484 pipe(0x316d2b80 sd=239 :6802 s=2 pgs=3 cs=1 l=1 c=0xa26c160).fault (0) Success -5447> 2014-11-03 11:37:52.016721 7f290c834700 1 osd.21 105950 ms_handle_reset con 0xa26c160 session 0x2439aa00 -5446> 2014-11-03 11:37:52.016849 7f28eca56700 1 -- 192.168.5.2:6803/7606 <== osd.104 192.168.5.8:6828/2576 48 ==== osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1479539589 0 3511520646) 0x2ce3d400 con 0x230c2dc0 -5445> 2014-11-03 11:37:52.016890 7f28eca56700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.016707, event: header_read, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5444> 2014-11-03 11:37:52.016908 7f28eca56700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.016710, event: throttled, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5443> 2014-11-03 11:37:52.016923 7f28eca56700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.016834, event: all_read, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5442> 2014-11-03 11:37:52.016936 7f28eca56700 5 -- op tracker -- seq: 3015, time: 0.000000, event: dispatched, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5441> 2014-11-03 11:37:52.017014 7f290181e700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.017014, event: reached_pg, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5440> 2014-11-03 11:37:52.017052 7f290181e700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.017052, event: started, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5439> 2014-11-03 11:37:52.017173 7f290181e700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.017173, event: started, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5438> 2014-11-03 11:37:52.017216 7f290181e700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.017216, event: commit_queued_for_journal_write, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5437> 2014-11-03 11:37:52.017295 7f2914d0c700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.017295, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5436> 2014-11-03 11:37:52.017560 7f291450b700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.017560, event: journaled_completion_queued, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5435> 2014-11-03 11:37:52.017602 7f2911d06700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.017602, event: commit_sent, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5434> 2014-11-03 11:37:52.017644 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6828/2576 -- osd_sub_op_reply(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23613dc0 con 0x230c2dc0 -5433> 2014-11-03 11:37:52.018019 7f2912507700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.018018, event: sub_op_applied, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5432> 2014-11-03 11:37:52.018061 7f2912507700 5 -- op tracker -- seq: 3015, time: 2014-11-03 11:37:52.018061, event: done, op: osd_sub_op(client.8032023.0:8609737 6.69b 825a3e9b/rbd_data.1a66b611eb903e.0000000000001160/head//6 [] v 105950'21540857 snapset=0=[]:[] snapc=0=[]) -5431> 2014-11-03 11:37:52.021055 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 41 ==== osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3724817504 0 3511520646) 0x29e11000 con 0x231ab580 -5430> 2014-11-03 11:37:52.021087 7f28ea430700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.020920, event: header_read, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5429> 2014-11-03 11:37:52.021099 7f28ea430700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.020922, event: throttled, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5428> 2014-11-03 11:37:52.021112 7f28ea430700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021041, event: all_read, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5427> 2014-11-03 11:37:52.021131 7f28ea430700 5 -- op tracker -- seq: 3016, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5426> 2014-11-03 11:37:52.021186 7f28ff019700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021186, event: reached_pg, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5425> 2014-11-03 11:37:52.021219 7f28ff019700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021219, event: started, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5424> 2014-11-03 11:37:52.021312 7f28ff019700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021312, event: started, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5423> 2014-11-03 11:37:52.021345 7f28ff019700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021345, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5422> 2014-11-03 11:37:52.021397 7f2914d0c700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021396, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5421> 2014-11-03 11:37:52.021616 7f291450b700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021616, event: journaled_completion_queued, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5420> 2014-11-03 11:37:52.021653 7f2911d06700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021652, event: commit_sent, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5419> 2014-11-03 11:37:52.021684 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7d600 con 0x231ab580 -5418> 2014-11-03 11:37:52.021875 7f2912507700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021875, event: sub_op_applied, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5417> 2014-11-03 11:37:52.021914 7f2912507700 5 -- op tracker -- seq: 3016, time: 2014-11-03 11:37:52.021914, event: done, op: osd_sub_op(client.7869893.0:738134 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629963 snapset=0=[]:[] snapc=0=[]) -5416> 2014-11-03 11:37:52.031655 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.80 192.168.5.6:0/21329 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.030926) v2 ==== 47+0+0 (1718850742 0 0) 0x23fdf600 con 0x249252c0 -5415> 2014-11-03 11:37:52.031709 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/21329 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.030926) v2 -- ?+0 0x229e7800 con 0x249252c0 -5414> 2014-11-03 11:37:52.031813 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.80 192.168.5.6:0/21329 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.030926) v2 ==== 47+0+0 (1718850742 0 0) 0x2b1ff400 con 0x24229000 -5413> 2014-11-03 11:37:52.031868 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/21329 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.030926) v2 -- ?+0 0x2be31600 con 0x24229000 -5412> 2014-11-03 11:37:52.056481 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.4 192.168.5.1:0/31636 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.055805) v2 ==== 47+0+0 (623072289 0 0) 0x8714200 con 0x2389f580 -5411> 2014-11-03 11:37:52.056493 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.4 192.168.5.1:0/31636 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.055805) v2 ==== 47+0+0 (623072289 0 0) 0x9591800 con 0x31a9580 -5410> 2014-11-03 11:37:52.056599 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/31636 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.055805) v2 -- ?+0 0x2b1ff400 con 0x2389f580 -5409> 2014-11-03 11:37:52.056742 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/31636 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.055805) v2 -- ?+0 0x23fdf600 con 0x31a9580 -5408> 2014-11-03 11:37:52.091993 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x23a9e400 con 0x22fbc420 -5407> 2014-11-03 11:37:52.112790 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 96 ==== MOSDPGPush(6.58e 105950 [PushOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, version: 105946'11622353, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (603631077 0 0) 0x23a9e400 con 0x22fbc420 -5406> 2014-11-03 11:37:52.112845 7f28f72fe700 5 -- op tracker -- seq: 3017, time: 2014-11-03 11:37:52.096066, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, version: 105946'11622353, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5405> 2014-11-03 11:37:52.112867 7f28f72fe700 5 -- op tracker -- seq: 3017, time: 2014-11-03 11:37:52.096068, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, version: 105946'11622353, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5404> 2014-11-03 11:37:52.112931 7f28f72fe700 5 -- op tracker -- seq: 3017, time: 2014-11-03 11:37:52.112776, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, version: 105946'11622353, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5403> 2014-11-03 11:37:52.112954 7f28f72fe700 5 -- op tracker -- seq: 3017, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, version: 105946'11622353, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5402> 2014-11-03 11:37:52.113052 7f28fe818700 5 -- op tracker -- seq: 3017, time: 2014-11-03 11:37:52.113052, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, version: 105946'11622353, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5401> 2014-11-03 11:37:52.113284 7f28fe818700 5 -- op tracker -- seq: 3017, time: 2014-11-03 11:37:52.113283, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6, version: 105946'11622353, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e0dafd8e/rbd_data.7986317d92ecf2.0000000000004a67/head//6@105946'11622353, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5400> 2014-11-03 11:37:52.119358 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x23614340 sd=534 :6802 s=0 pgs=0 cs=0 l=0 c=0x975e840).accept sd=534 192.168.5.201:58666/0 -5399> 2014-11-03 11:37:52.120280 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419922 192.168.5.201:0/1029156 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2782fe00 con 0x975e840 -5398> 2014-11-03 11:37:52.121071 7f28fb812700 0 osd.21 105950 do_command r=0 -5397> 2014-11-03 11:37:52.121118 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029156 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xccda000 con 0x975e840 -5396> 2014-11-03 11:37:52.124872 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x1adc4600 con 0x231abb00 -5395> 2014-11-03 11:37:52.129814 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419922 192.168.5.201:0/1029156 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23510c00 con 0x975e840 -5394> 2014-11-03 11:37:52.130589 7f28fb812700 0 osd.21 105950 do_command r=0 -5393> 2014-11-03 11:37:52.130603 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029156 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x234168c0 con 0x975e840 -5392> 2014-11-03 11:37:52.133921 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.104 192.168.5.8:0/2576 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.132645) v2 ==== 47+0+0 (473433572 0 0) 0x5f22600 con 0x239f8000 -5391> 2014-11-03 11:37:52.133936 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.104 192.168.5.8:0/2576 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.132645) v2 ==== 47+0+0 (473433572 0 0) 0x23414800 con 0x239f8dc0 -5390> 2014-11-03 11:37:52.133961 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.132645) v2 -- ?+0 0x9591800 con 0x239f8000 -5389> 2014-11-03 11:37:52.134011 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.132645) v2 -- ?+0 0x8714200 con 0x239f8dc0 -5388> 2014-11-03 11:37:52.136735 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029156 pipe(0x23614340 sd=534 :6802 s=2 pgs=2 cs=1 l=1 c=0x975e840).reader couldn't read tag, (0) Success -5387> 2014-11-03 11:37:52.136802 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029156 pipe(0x23614340 sd=534 :6802 s=2 pgs=2 cs=1 l=1 c=0x975e840).fault (0) Success -5386> 2014-11-03 11:37:52.136906 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x975e840 session 0x2439aa00 -5385> 2014-11-03 11:37:52.157028 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.45 192.168.5.4:0/1781 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.154823) v2 ==== 47+0+0 (122140610 0 0) 0x1106ce00 con 0x2445c160 -5384> 2014-11-03 11:37:52.157059 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.45 192.168.5.4:0/1781 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.154823) v2 ==== 47+0+0 (122140610 0 0) 0xde23c00 con 0x23ea42c0 -5383> 2014-11-03 11:37:52.157087 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.154823) v2 -- ?+0 0x5f22600 con 0x2445c160 -5382> 2014-11-03 11:37:52.157133 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.154823) v2 -- ?+0 0x23414800 con 0x23ea42c0 -5381> 2014-11-03 11:37:52.157683 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 30 ==== osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (2194690030 0 3511520646) 0x26151000 con 0x22fe0840 -5380> 2014-11-03 11:37:52.157717 7f28f4cd8700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.157569, event: header_read, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5379> 2014-11-03 11:37:52.157736 7f28f4cd8700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.157571, event: throttled, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5378> 2014-11-03 11:37:52.157747 7f28f4cd8700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.157669, event: all_read, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5377> 2014-11-03 11:37:52.157765 7f28f4cd8700 5 -- op tracker -- seq: 3018, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5376> 2014-11-03 11:37:52.157824 7f28fe017700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.157824, event: reached_pg, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5375> 2014-11-03 11:37:52.157856 7f28fe017700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.157856, event: started, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5374> 2014-11-03 11:37:52.157944 7f28fe017700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.157943, event: started, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5373> 2014-11-03 11:37:52.158023 7f28fe017700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.158023, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5372> 2014-11-03 11:37:52.158083 7f2914d0c700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.158083, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5371> 2014-11-03 11:37:52.158420 7f291450b700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.158419, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5370> 2014-11-03 11:37:52.158476 7f2911d06700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.158476, event: commit_sent, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5369> 2014-11-03 11:37:52.158504 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7c000 con 0x22fe0840 -5368> 2014-11-03 11:37:52.158762 7f2912507700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.158761, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5367> 2014-11-03 11:37:52.158822 7f2912507700 5 -- op tracker -- seq: 3018, time: 2014-11-03 11:37:52.158822, event: done, op: osd_sub_op(client.7867942.0:137197711 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960080 snapset=0=[]:[] snapc=0=[]) -5366> 2014-11-03 11:37:52.161862 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0xa0008c0 sd=205 :6802 s=0 pgs=0 cs=0 l=0 c=0x9fc42c0).accept sd=205 192.168.5.2:42909/0 -5365> 2014-11-03 11:37:52.162830 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450734 192.168.5.2:0/1032290 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23510200 con 0x9fc42c0 -5364> 2014-11-03 11:37:52.163484 7f28fb812700 0 osd.21 105950 do_command r=0 -5363> 2014-11-03 11:37:52.163517 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1032290 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xccda8c0 con 0x9fc42c0 -5362> 2014-11-03 11:37:52.168174 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.79 192.168.5.6:0/12953 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.166784) v2 ==== 47+0+0 (3187455090 0 0) 0x23296a00 con 0x23b1c420 -5361> 2014-11-03 11:37:52.168217 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/12953 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.166784) v2 -- ?+0 0x1106ce00 con 0x23b1c420 -5360> 2014-11-03 11:37:52.168238 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.79 192.168.5.6:0/12953 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.166784) v2 ==== 47+0+0 (3187455090 0 0) 0x23408c00 con 0x242052c0 -5359> 2014-11-03 11:37:52.168302 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/12953 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.166784) v2 -- ?+0 0xde23c00 con 0x242052c0 -5358> 2014-11-03 11:37:52.170371 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 31 ==== osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (3785359093 0 3511520646) 0x26153400 con 0x22fe0840 -5357> 2014-11-03 11:37:52.170408 7f28f4cd8700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170290, event: header_read, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5356> 2014-11-03 11:37:52.170426 7f28f4cd8700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170291, event: throttled, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5355> 2014-11-03 11:37:52.170447 7f28f4cd8700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170358, event: all_read, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5354> 2014-11-03 11:37:52.170463 7f28f4cd8700 5 -- op tracker -- seq: 3019, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5353> 2014-11-03 11:37:52.170543 7f290081c700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170543, event: reached_pg, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5352> 2014-11-03 11:37:52.170575 7f290081c700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170575, event: started, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5351> 2014-11-03 11:37:52.170669 7f290081c700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170668, event: started, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5350> 2014-11-03 11:37:52.170698 7f290081c700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170698, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5349> 2014-11-03 11:37:52.170767 7f2914d0c700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.170767, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5348> 2014-11-03 11:37:52.171189 7f291450b700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.171189, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5347> 2014-11-03 11:37:52.171277 7f2911d06700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.171277, event: commit_sent, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5346> 2014-11-03 11:37:52.171309 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7c840 con 0x22fe0840 -5345> 2014-11-03 11:37:52.171651 7f2912507700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.171651, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5344> 2014-11-03 11:37:52.171755 7f2912507700 5 -- op tracker -- seq: 3019, time: 2014-11-03 11:37:52.171755, event: done, op: osd_sub_op(client.7867942.0:137197712 6.35a a85e1b5a/rbd_data.43c40916a10.00000000000005e9/head//6 [] v 105950'14960081 snapset=0=[]:[] snapc=0=[]) -5343> 2014-11-03 11:37:52.174059 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450734 192.168.5.2:0/1032290 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x27830600 con 0x9fc42c0 -5342> 2014-11-03 11:37:52.175252 7f28fb812700 0 osd.21 105950 do_command r=0 -5341> 2014-11-03 11:37:52.175276 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1032290 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x23416c40 con 0x9fc42c0 -5340> 2014-11-03 11:37:52.180617 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.57 192.168.5.5:0/16411 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.179082) v2 ==== 47+0+0 (1319518129 0 0) 0x896d600 con 0x2469e420 -5339> 2014-11-03 11:37:52.180675 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.179082) v2 -- ?+0 0x23408c00 con 0x2469e420 -5338> 2014-11-03 11:37:52.182221 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.57 192.168.5.5:0/16411 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.179082) v2 ==== 47+0+0 (1319518129 0 0) 0x896d000 con 0x242c0160 -5337> 2014-11-03 11:37:52.182272 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.179082) v2 -- ?+0 0x23296a00 con 0x242c0160 -5336> 2014-11-03 11:37:52.182910 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1032290 pipe(0xa0008c0 sd=205 :6802 s=2 pgs=2 cs=1 l=1 c=0x9fc42c0).reader couldn't read tag, (0) Success -5335> 2014-11-03 11:37:52.182988 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1032290 pipe(0xa0008c0 sd=205 :6802 s=2 pgs=2 cs=1 l=1 c=0x9fc42c0).fault (0) Success -5334> 2014-11-03 11:37:52.183134 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x9fc42c0 session 0x2439aa00 -5333> 2014-11-03 11:37:52.188834 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.114 192.168.5.9:0/17408 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.187354) v2 ==== 47+0+0 (4093058512 0 0) 0x5d49800 con 0x2484a000 -5332> 2014-11-03 11:37:52.188853 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.114 192.168.5.9:0/17408 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.187354) v2 ==== 47+0+0 (4093058512 0 0) 0x2295a800 con 0x2469e580 -5331> 2014-11-03 11:37:52.188876 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/17408 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.187354) v2 -- ?+0 0x896d000 con 0x2484a000 -5330> 2014-11-03 11:37:52.188960 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/17408 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.187354) v2 -- ?+0 0x896d600 con 0x2469e580 -5329> 2014-11-03 11:37:52.193814 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 48 ==== MOSDPGPush(6.5ca 105950 [PushOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6, version: 105946'9539459, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6@105946'9539459, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1136181489 0 0) 0x27b01600 con 0x22fbc160 -5328> 2014-11-03 11:37:52.193883 7f28f6df9700 5 -- op tracker -- seq: 3020, time: 2014-11-03 11:37:52.181203, event: header_read, op: MOSDPGPush(6.5ca 105950 [PushOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6, version: 105946'9539459, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6@105946'9539459, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5327> 2014-11-03 11:37:52.194096 7f28f6df9700 5 -- op tracker -- seq: 3020, time: 2014-11-03 11:37:52.181206, event: throttled, op: MOSDPGPush(6.5ca 105950 [PushOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6, version: 105946'9539459, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6@105946'9539459, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5326> 2014-11-03 11:37:52.194176 7f28f6df9700 5 -- op tracker -- seq: 3020, time: 2014-11-03 11:37:52.193794, event: all_read, op: MOSDPGPush(6.5ca 105950 [PushOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6, version: 105946'9539459, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6@105946'9539459, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5325> 2014-11-03 11:37:52.194230 7f28f6df9700 5 -- op tracker -- seq: 3020, time: 0.000000, event: dispatched, op: MOSDPGPush(6.5ca 105950 [PushOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6, version: 105946'9539459, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6@105946'9539459, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5324> 2014-11-03 11:37:52.194427 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 49 ==== osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3689572978 0 3511520646) 0x28ea5400 con 0x22fbc160 -5323> 2014-11-03 11:37:52.194340 7f290101d700 5 -- op tracker -- seq: 3020, time: 2014-11-03 11:37:52.194340, event: reached_pg, op: MOSDPGPush(6.5ca 105950 [PushOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6, version: 105946'9539459, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6@105946'9539459, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5322> 2014-11-03 11:37:52.194473 7f28f6df9700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.194299, event: header_read, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5321> 2014-11-03 11:37:52.194516 7f28f6df9700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.194300, event: throttled, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5320> 2014-11-03 11:37:52.194538 7f28f6df9700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.194414, event: all_read, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5319> 2014-11-03 11:37:52.194561 7f28f6df9700 5 -- op tracker -- seq: 3021, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5318> 2014-11-03 11:37:52.194714 7f290101d700 5 -- op tracker -- seq: 3020, time: 2014-11-03 11:37:52.194714, event: done, op: MOSDPGPush(6.5ca 105950 [PushOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6, version: 105946'9539459, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6@105946'9539459, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5317> 2014-11-03 11:37:52.194737 7f28fe818700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.194737, event: reached_pg, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5316> 2014-11-03 11:37:52.194802 7f28fe818700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.194802, event: started, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5315> 2014-11-03 11:37:52.194971 7f28fe818700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.194971, event: started, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5314> 2014-11-03 11:37:52.195020 7f28fe818700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.195020, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5313> 2014-11-03 11:37:52.195941 7f2914d0c700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.195941, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5312> 2014-11-03 11:37:52.200935 7f291450b700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.200935, event: journaled_completion_queued, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5311> 2014-11-03 11:37:52.201055 7f2911d06700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.201055, event: commit_sent, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5310> 2014-11-03 11:37:52.201115 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- osd_sub_op_reply(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7c580 con 0x22fbc160 -5309> 2014-11-03 11:37:52.220970 7f2918d14700 5 osd.21 105950 tick -5308> 2014-11-03 11:37:52.232610 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- MOSDPGPushReply(6.5ca 105950 [PushReplyOp(1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6)]) v2 -- ?+0 0x24139600 con 0x22fbc160 -5307> 2014-11-03 11:37:52.232965 7f2912507700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.232965, event: sub_op_applied, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5306> 2014-11-03 11:37:52.233012 7f2912507700 5 -- op tracker -- seq: 3021, time: 2014-11-03 11:37:52.233012, event: done, op: osd_sub_op(client.7868017.0:1199802 6.5ca 47f1adca/rbd_data.56d605106cc7e4.0000000000000200/head//6 [] v 105950'9539477 snapset=0=[]:[] snapc=0=[]) -5305> 2014-11-03 11:37:52.234513 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 50 ==== osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+4921 (3212999336 0 1096458426) 0x24866c00 con 0x22fbc160 -5304> 2014-11-03 11:37:52.234551 7f28f6df9700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234402, event: header_read, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5303> 2014-11-03 11:37:52.234567 7f28f6df9700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234404, event: throttled, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5302> 2014-11-03 11:37:52.234588 7f28f6df9700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234500, event: all_read, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5301> 2014-11-03 11:37:52.234597 7f28f6df9700 5 -- op tracker -- seq: 3022, time: 0.000000, event: dispatched, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5300> 2014-11-03 11:37:52.234695 7f290101d700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234695, event: reached_pg, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5299> 2014-11-03 11:37:52.234731 7f290101d700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234730, event: started, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5298> 2014-11-03 11:37:52.234840 7f290101d700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234839, event: started, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5297> 2014-11-03 11:37:52.234871 7f290101d700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234871, event: commit_queued_for_journal_write, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5296> 2014-11-03 11:37:52.234941 7f2914d0c700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.234941, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5295> 2014-11-03 11:37:52.236135 7f291450b700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.236134, event: journaled_completion_queued, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5294> 2014-11-03 11:37:52.236250 7f2911d06700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.236250, event: commit_sent, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5293> 2014-11-03 11:37:52.236285 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- osd_sub_op_reply(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7cb00 con 0x22fbc160 -5292> 2014-11-03 11:37:52.236684 7f2912507700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.236683, event: sub_op_applied, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5291> 2014-11-03 11:37:52.236756 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2494b2c0 sd=202 :6802 s=0 pgs=0 cs=0 l=0 c=0x2d37ab00).accept sd=202 192.168.5.201:58669/0 -5290> 2014-11-03 11:37:52.236769 7f2912507700 5 -- op tracker -- seq: 3022, time: 2014-11-03 11:37:52.236768, event: done, op: osd_sub_op(client.8709331.0:2167933 6.5ca 1a3a45ca/rbd_data.7986317d92ecf2.0000000000005000/head//6 [] v 105950'9539478 snapset=0=[]:[] snapc=0=[]) -5289> 2014-11-03 11:37:52.237696 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419925 192.168.5.201:0/1029198 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.58e"}) v1 ==== 79+0+0 (3567090102 0 0) 0xada4200 con 0x2d37ab00 -5288> 2014-11-03 11:37:52.238427 7f28fb812700 0 osd.21 105950 do_command r=0 -5287> 2014-11-03 11:37:52.238458 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029198 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xa5f6380 con 0x2d37ab00 -5286> 2014-11-03 11:37:52.240973 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.50 192.168.5.4:0/26902 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.239541) v2 ==== 47+0+0 (4156983085 0 0) 0x2579f800 con 0x23335160 -5285> 2014-11-03 11:37:52.241030 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/26902 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.239541) v2 -- ?+0 0x5d49800 con 0x23335160 -5284> 2014-11-03 11:37:52.241320 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.50 192.168.5.4:0/26902 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.239541) v2 ==== 47+0+0 (4156983085 0 0) 0x23323200 con 0x234ed160 -5283> 2014-11-03 11:37:52.241373 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/26902 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.239541) v2 -- ?+0 0x2295a800 con 0x234ed160 -5282> 2014-11-03 11:37:52.247435 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419925 192.168.5.201:0/1029198 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.58e"}) v1 ==== 73+0+0 (125580214 0 0) 0x2295a800 con 0x2d37ab00 -5281> 2014-11-03 11:37:52.248520 7f28fb812700 0 osd.21 105950 do_command r=0 -5280> 2014-11-03 11:37:52.248536 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029198 -- command_reply(tid 2: 0 ) v1 -- ?+16765 0xa5f6e00 con 0x2d37ab00 -5279> 2014-11-03 11:37:52.252982 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029198 pipe(0x2494b2c0 sd=202 :6802 s=2 pgs=2 cs=1 l=1 c=0x2d37ab00).reader couldn't read tag, (0) Success -5278> 2014-11-03 11:37:52.253035 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029198 pipe(0x2494b2c0 sd=202 :6802 s=2 pgs=2 cs=1 l=1 c=0x2d37ab00).fault (0) Success -5277> 2014-11-03 11:37:52.253139 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2d37ab00 session 0x2439aa00 -5276> 2014-11-03 11:37:52.271757 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 87 ==== osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (652713035 0 3511520646) 0x24e85000 con 0x22fef420 -5275> 2014-11-03 11:37:52.271821 7f28f41cd700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.271609, event: header_read, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5274> 2014-11-03 11:37:52.271842 7f28f41cd700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.271612, event: throttled, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5273> 2014-11-03 11:37:52.271855 7f28f41cd700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.271735, event: all_read, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5272> 2014-11-03 11:37:52.271867 7f28f41cd700 5 -- op tracker -- seq: 3023, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5271> 2014-11-03 11:37:52.271930 7f28fe818700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.271930, event: reached_pg, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5270> 2014-11-03 11:37:52.271960 7f28fe818700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.271960, event: started, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5269> 2014-11-03 11:37:52.272097 7f28fe818700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.272097, event: started, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5268> 2014-11-03 11:37:52.272142 7f28fe818700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.272142, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5267> 2014-11-03 11:37:52.272230 7f2914d0c700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.272230, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5266> 2014-11-03 11:37:52.272602 7f291450b700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.272602, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5265> 2014-11-03 11:37:52.272720 7f2911d06700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.272720, event: commit_sent, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5264> 2014-11-03 11:37:52.272724 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 88 ==== osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2620958066 0 3511520646) 0x2c692400 con 0x22fef420 -5263> 2014-11-03 11:37:52.272805 7f28f41cd700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.272562, event: header_read, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5262> 2014-11-03 11:37:52.272824 7f28f41cd700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.272578, event: throttled, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5261> 2014-11-03 11:37:52.272837 7f28f41cd700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.272709, event: all_read, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5260> 2014-11-03 11:37:52.272826 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7d8c0 con 0x22fef420 -5259> 2014-11-03 11:37:52.272849 7f28f41cd700 5 -- op tracker -- seq: 3024, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5258> 2014-11-03 11:37:52.272978 7f290101d700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.272978, event: reached_pg, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5257> 2014-11-03 11:37:52.273018 7f290101d700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273018, event: started, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5256> 2014-11-03 11:37:52.273114 7f290101d700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273114, event: started, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5255> 2014-11-03 11:37:52.273142 7f290101d700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273142, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5254> 2014-11-03 11:37:52.273202 7f2914d0c700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273202, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5253> 2014-11-03 11:37:52.273218 7f2912507700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.273217, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5252> 2014-11-03 11:37:52.273264 7f2912507700 5 -- op tracker -- seq: 3023, time: 2014-11-03 11:37:52.273264, event: done, op: osd_sub_op(client.7914641.0:33713519 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057567 snapset=0=[]:[] snapc=0=[]) -5251> 2014-11-03 11:37:52.273411 7f291450b700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273411, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5250> 2014-11-03 11:37:52.273491 7f2911d06700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273490, event: commit_sent, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5249> 2014-11-03 11:37:52.273560 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272eb600 con 0x22fef420 -5248> 2014-11-03 11:37:52.273806 7f2912507700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273805, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5247> 2014-11-03 11:37:52.273840 7f2912507700 5 -- op tracker -- seq: 3024, time: 2014-11-03 11:37:52.273840, event: done, op: osd_sub_op(client.7914641.0:33713520 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057568 snapset=0=[]:[] snapc=0=[]) -5246> 2014-11-03 11:37:52.274208 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 89 ==== osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1701257238 0 3511520646) 0x2c690c00 con 0x22fef420 -5245> 2014-11-03 11:37:52.274236 7f28f41cd700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274129, event: header_read, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5244> 2014-11-03 11:37:52.274252 7f28f41cd700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274131, event: throttled, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5243> 2014-11-03 11:37:52.274270 7f28f41cd700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274199, event: all_read, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5242> 2014-11-03 11:37:52.274281 7f28f41cd700 5 -- op tracker -- seq: 3025, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5241> 2014-11-03 11:37:52.274338 7f28fe818700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274338, event: reached_pg, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5240> 2014-11-03 11:37:52.274362 7f28fe818700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274362, event: started, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5239> 2014-11-03 11:37:52.274433 7f28fe818700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274433, event: started, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5238> 2014-11-03 11:37:52.274456 7f28fe818700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274456, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5237> 2014-11-03 11:37:52.274520 7f2914d0c700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274519, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5236> 2014-11-03 11:37:52.274727 7f291450b700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274726, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5235> 2014-11-03 11:37:52.274813 7f2911d06700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274813, event: commit_sent, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5234> 2014-11-03 11:37:52.274851 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] ondisk, result = 0) v2 -- ?+0 0x272ea580 con 0x22fef420 -5233> 2014-11-03 11:37:52.274978 7f2912507700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.274978, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5232> 2014-11-03 11:37:52.275009 7f2912507700 5 -- op tracker -- seq: 3025, time: 2014-11-03 11:37:52.275009, event: done, op: osd_sub_op(client.7914641.0:33713521 6.755 76a16f55/rbd_data.49e9f52ae8944a.0000000000001212/head//6 [] v 105950'16057569 snapset=0=[]:[] snapc=0=[]) -5231> 2014-11-03 11:37:52.315383 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 43 ==== MOSDPGPush(6.91c 105950 [PushOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6, version: 105945'10710476, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6@105945'10710476, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6, version: 105946'10710818, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6@105946'10710818, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (495860616 0 0) 0x2b950c00 con 0x23000b00 -5230> 2014-11-03 11:37:52.315448 7f28f21ad700 5 -- op tracker -- seq: 3026, time: 2014-11-03 11:37:52.289891, event: header_read, op: MOSDPGPush(6.91c 105950 [PushOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6, version: 105945'10710476, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6@105945'10710476, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6, version: 105946'10710818, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6@105946'10710818, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5229> 2014-11-03 11:37:52.315549 7f28f21ad700 5 -- op tracker -- seq: 3026, time: 2014-11-03 11:37:52.289895, event: throttled, op: MOSDPGPush(6.91c 105950 [PushOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6, version: 105945'10710476, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6@105945'10710476, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6, version: 105946'10710818, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6@105946'10710818, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5228> 2014-11-03 11:37:52.315599 7f28f21ad700 5 -- op tracker -- seq: 3026, time: 2014-11-03 11:37:52.315365, event: all_read, op: MOSDPGPush(6.91c 105950 [PushOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6, version: 105945'10710476, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6@105945'10710476, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6, version: 105946'10710818, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6@105946'10710818, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5227> 2014-11-03 11:37:52.315636 7f28f21ad700 5 -- op tracker -- seq: 3026, time: 0.000000, event: dispatched, op: MOSDPGPush(6.91c 105950 [PushOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6, version: 105945'10710476, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6@105945'10710476, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6, version: 105946'10710818, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6@105946'10710818, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5226> 2014-11-03 11:37:52.315778 7f290101d700 5 -- op tracker -- seq: 3026, time: 2014-11-03 11:37:52.315778, event: reached_pg, op: MOSDPGPush(6.91c 105950 [PushOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6, version: 105945'10710476, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6@105945'10710476, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6, version: 105946'10710818, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6@105946'10710818, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5225> 2014-11-03 11:37:52.316283 7f290101d700 5 -- op tracker -- seq: 3026, time: 2014-11-03 11:37:52.316282, event: done, op: MOSDPGPush(6.91c 105950 [PushOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6, version: 105945'10710476, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6@105945'10710476, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6, version: 105946'10710818, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6@105946'10710818, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5224> 2014-11-03 11:37:52.317412 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 88 ==== osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4189561912 0 3511520646) 0x2cff4800 con 0x22ff3b00 -5223> 2014-11-03 11:37:52.317442 7f28f0d99700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.317302, event: header_read, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5222> 2014-11-03 11:37:52.317480 7f28f0d99700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.317303, event: throttled, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5221> 2014-11-03 11:37:52.317488 7f28f0d99700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.317404, event: all_read, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5220> 2014-11-03 11:37:52.317500 7f28f0d99700 5 -- op tracker -- seq: 3027, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5219> 2014-11-03 11:37:52.317594 7f28fd816700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.317593, event: reached_pg, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5218> 2014-11-03 11:37:52.317647 7f28fd816700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.317647, event: started, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5217> 2014-11-03 11:37:52.317789 7f28fd816700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.317788, event: started, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5216> 2014-11-03 11:37:52.317851 7f28fd816700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.317851, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5215> 2014-11-03 11:37:52.318318 7f2914d0c700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.318317, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5214> 2014-11-03 11:37:52.326728 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x28229b80 sd=202 :6802 s=0 pgs=0 cs=0 l=0 c=0x250a0420).accept sd=202 192.168.5.2:42912/0 -5213> 2014-11-03 11:37:52.327626 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450740 192.168.5.2:0/1032491 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x261a4800 con 0x250a0420 -5212> 2014-11-03 11:37:52.328388 7f28fb812700 0 osd.21 105950 do_command r=0 -5211> 2014-11-03 11:37:52.328408 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1032491 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f1c0 con 0x250a0420 -5210> 2014-11-03 11:37:52.330732 7f291450b700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.330732, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5209> 2014-11-03 11:37:52.330791 7f2911d06700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.330791, event: commit_sent, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5208> 2014-11-03 11:37:52.330834 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x272eb340 con 0x22ff3b00 -5207> 2014-11-03 11:37:52.331323 7f2912507700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.331323, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5206> 2014-11-03 11:37:52.331374 7f2912507700 5 -- op tracker -- seq: 3027, time: 2014-11-03 11:37:52.331374, event: done, op: osd_sub_op(client.12146205.0:439306 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222067 snapset=0=[]:[] snapc=0=[]) -5205> 2014-11-03 11:37:52.333501 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.84 192.168.5.7:0/19824 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.331720) v2 ==== 47+0+0 (720749448 0 0) 0x253e5c00 con 0x31a9420 -5204> 2014-11-03 11:37:52.333580 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/19824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.331720) v2 -- ?+0 0x2579f800 con 0x31a9420 -5203> 2014-11-03 11:37:52.333600 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.84 192.168.5.7:0/19824 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.331720) v2 ==== 47+0+0 (720749448 0 0) 0x2a347a00 con 0x2389f420 -5202> 2014-11-03 11:37:52.333644 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/19824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.331720) v2 -- ?+0 0x23323200 con 0x2389f420 -5201> 2014-11-03 11:37:52.337186 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450740 192.168.5.2:0/1032491 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x30db800 con 0x250a0420 -5200> 2014-11-03 11:37:52.338358 7f28fb812700 0 osd.21 105950 do_command r=0 -5199> 2014-11-03 11:37:52.338392 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1032491 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x272da700 con 0x250a0420 -5198> 2014-11-03 11:37:52.339168 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 165 ==== osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2411480403 0 3511520646) 0x96ea000 con 0x22fefdc0 -5197> 2014-11-03 11:37:52.339208 7f28f36c2700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339048, event: header_read, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5196> 2014-11-03 11:37:52.339223 7f28f36c2700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339050, event: throttled, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5195> 2014-11-03 11:37:52.339237 7f28f36c2700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339153, event: all_read, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5194> 2014-11-03 11:37:52.339247 7f28f36c2700 5 -- op tracker -- seq: 3028, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5193> 2014-11-03 11:37:52.339322 7f28fe818700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339322, event: reached_pg, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5192> 2014-11-03 11:37:52.339357 7f28fe818700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339357, event: started, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5191> 2014-11-03 11:37:52.339461 7f28fe818700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339460, event: started, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5190> 2014-11-03 11:37:52.339499 7f28fe818700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339499, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5189> 2014-11-03 11:37:52.339571 7f2914d0c700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.339571, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5188> 2014-11-03 11:37:52.340561 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 44 ==== MOSDPGPush(6.91c 105950 [PushOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6, version: 105946'10710822, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6@105946'10710822, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6, version: 105946'10710854, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6@105946'10710854, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390497+0+0 (451367275 0 0) 0x31a6200 con 0x23000b00 -5187> 2014-11-03 11:37:52.340610 7f28f21ad700 5 -- op tracker -- seq: 3029, time: 2014-11-03 11:37:52.315787, event: header_read, op: MOSDPGPush(6.91c 105950 [PushOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6, version: 105946'10710822, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6@105946'10710822, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6, version: 105946'10710854, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6@105946'10710854, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5186> 2014-11-03 11:37:52.340637 7f28f21ad700 5 -- op tracker -- seq: 3029, time: 2014-11-03 11:37:52.315788, event: throttled, op: MOSDPGPush(6.91c 105950 [PushOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6, version: 105946'10710822, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6@105946'10710822, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6, version: 105946'10710854, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6@105946'10710854, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5185> 2014-11-03 11:37:52.340664 7f28f21ad700 5 -- op tracker -- seq: 3029, time: 2014-11-03 11:37:52.340544, event: all_read, op: MOSDPGPush(6.91c 105950 [PushOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6, version: 105946'10710822, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6@105946'10710822, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6, version: 105946'10710854, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6@105946'10710854, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5184> 2014-11-03 11:37:52.340688 7f28f21ad700 5 -- op tracker -- seq: 3029, time: 0.000000, event: dispatched, op: MOSDPGPush(6.91c 105950 [PushOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6, version: 105946'10710822, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6@105946'10710822, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6, version: 105946'10710854, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6@105946'10710854, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5183> 2014-11-03 11:37:52.340762 7f290101d700 5 -- op tracker -- seq: 3029, time: 2014-11-03 11:37:52.340762, event: reached_pg, op: MOSDPGPush(6.91c 105950 [PushOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6, version: 105946'10710822, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6@105946'10710822, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6, version: 105946'10710854, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6@105946'10710854, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5182> 2014-11-03 11:37:52.341035 7f290101d700 5 -- op tracker -- seq: 3029, time: 2014-11-03 11:37:52.341034, event: done, op: MOSDPGPush(6.91c 105950 [PushOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6, version: 105946'10710822, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6@105946'10710822, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6, version: 105946'10710854, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6@105946'10710854, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5181> 2014-11-03 11:37:52.343792 7f28ea531700 1 -- 192.168.5.2:6803/7606 <== osd.92 192.168.5.7:6810/5438 20 ==== osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+17629 (2570672172 0 1645845580) 0x26c42c00 con 0x231ab6e0 -5180> 2014-11-03 11:37:52.343852 7f28ea531700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.343552, event: header_read, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5179> 2014-11-03 11:37:52.343874 7f28ea531700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.343555, event: throttled, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5178> 2014-11-03 11:37:52.343904 7f28ea531700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.343756, event: all_read, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5177> 2014-11-03 11:37:52.343922 7f28ea531700 5 -- op tracker -- seq: 3030, time: 0.000000, event: dispatched, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5176> 2014-11-03 11:37:52.344035 7f290181e700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.344034, event: reached_pg, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5175> 2014-11-03 11:37:52.344110 7f290181e700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.344110, event: started, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5174> 2014-11-03 11:37:52.344361 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1032491 pipe(0x28229b80 sd=202 :6802 s=2 pgs=2 cs=1 l=1 c=0x250a0420).reader couldn't read tag, (0) Success -5173> 2014-11-03 11:37:52.344471 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1032491 pipe(0x28229b80 sd=202 :6802 s=2 pgs=2 cs=1 l=1 c=0x250a0420).fault (0) Success -5172> 2014-11-03 11:37:52.344633 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x250a0420 session 0x2439aa00 -5171> 2014-11-03 11:37:52.348614 7f290181e700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.348614, event: started, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5170> 2014-11-03 11:37:52.348721 7f290181e700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.348720, event: commit_queued_for_journal_write, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5169> 2014-11-03 11:37:52.354962 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 45 ==== MOSDPGPush(6.91c 105950 [PushOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6, version: 105946'10710931, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6@105946'10710931, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (4167313380 0 0) 0x2369f000 con 0x23000b00 -5168> 2014-11-03 11:37:52.355064 7f28f21ad700 5 -- op tracker -- seq: 3031, time: 2014-11-03 11:37:52.340750, event: header_read, op: MOSDPGPush(6.91c 105950 [PushOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6, version: 105946'10710931, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6@105946'10710931, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5167> 2014-11-03 11:37:52.355150 7f28f21ad700 5 -- op tracker -- seq: 3031, time: 2014-11-03 11:37:52.340751, event: throttled, op: MOSDPGPush(6.91c 105950 [PushOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6, version: 105946'10710931, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6@105946'10710931, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5166> 2014-11-03 11:37:52.355233 7f28f21ad700 5 -- op tracker -- seq: 3031, time: 2014-11-03 11:37:52.354940, event: all_read, op: MOSDPGPush(6.91c 105950 [PushOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6, version: 105946'10710931, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6@105946'10710931, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5165> 2014-11-03 11:37:52.355285 7f28f21ad700 5 -- op tracker -- seq: 3031, time: 0.000000, event: dispatched, op: MOSDPGPush(6.91c 105950 [PushOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6, version: 105946'10710931, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6@105946'10710931, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5164> 2014-11-03 11:37:52.355372 7f28fe818700 5 -- op tracker -- seq: 3031, time: 2014-11-03 11:37:52.355371, event: reached_pg, op: MOSDPGPush(6.91c 105950 [PushOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6, version: 105946'10710931, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6@105946'10710931, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5163> 2014-11-03 11:37:52.355591 7f28fe818700 5 -- op tracker -- seq: 3031, time: 2014-11-03 11:37:52.355591, event: done, op: MOSDPGPush(6.91c 105950 [PushOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6, version: 105946'10710931, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6@105946'10710931, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -5162> 2014-11-03 11:37:52.368864 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 52 ==== osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2619132871 0 3511520646) 0x27eda400 con 0x23000840 -5161> 2014-11-03 11:37:52.368901 7f28f14a0700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.368718, event: header_read, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5160> 2014-11-03 11:37:52.368916 7f28f14a0700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.368720, event: throttled, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5159> 2014-11-03 11:37:52.368946 7f28f14a0700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.368852, event: all_read, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5158> 2014-11-03 11:37:52.368955 7f28f14a0700 5 -- op tracker -- seq: 3032, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5157> 2014-11-03 11:37:52.369028 7f290201f700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.369028, event: reached_pg, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5156> 2014-11-03 11:37:52.369061 7f290201f700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.369061, event: started, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5155> 2014-11-03 11:37:52.369171 7f290201f700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.369171, event: started, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5154> 2014-11-03 11:37:52.369205 7f290201f700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.369205, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5153> 2014-11-03 11:37:52.376989 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 53 ==== osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1954319881 0 3511520646) 0x27071800 con 0x23000840 -5152> 2014-11-03 11:37:52.377047 7f28f14a0700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.376880, event: header_read, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5151> 2014-11-03 11:37:52.377087 7f28f14a0700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.376889, event: throttled, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5150> 2014-11-03 11:37:52.377109 7f28f14a0700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.376976, event: all_read, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5149> 2014-11-03 11:37:52.377132 7f28f14a0700 5 -- op tracker -- seq: 3033, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5148> 2014-11-03 11:37:52.377194 7f28ff81a700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.377194, event: reached_pg, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5147> 2014-11-03 11:37:52.377278 7f28ff81a700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.377278, event: started, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5146> 2014-11-03 11:37:52.377496 7f28ff81a700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.377496, event: started, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5145> 2014-11-03 11:37:52.377580 7f28ff81a700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.377579, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5144> 2014-11-03 11:37:52.380825 7f290e7e2700 1 -- 192.168.5.2:6802/7606 <== client.7868459 192.168.1.91:0/2032747 2 ==== osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) v4 ==== 228+0+16384 (908161653 0 1398773268) 0x32fb280 con 0x2dc1ddc0 -5143> 2014-11-03 11:37:52.380887 7f290e7e2700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.380564, event: header_read, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5142> 2014-11-03 11:37:52.380909 7f290e7e2700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.380568, event: throttled, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5141> 2014-11-03 11:37:52.380918 7f290e7e2700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.380807, event: all_read, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5140> 2014-11-03 11:37:52.380934 7f290e7e2700 5 -- op tracker -- seq: 3034, time: 0.000000, event: dispatched, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5139> 2014-11-03 11:37:52.381044 7f290001b700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.381044, event: reached_pg, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5138> 2014-11-03 11:37:52.381412 7f290001b700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.381412, event: started, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5137> 2014-11-03 11:37:52.381445 7f290001b700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.381445, event: started, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5136> 2014-11-03 11:37:52.381533 7f290001b700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.381533, event: waiting for subops from 103,114, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5135> 2014-11-03 11:37:52.381587 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] v 105950'14083247 snapset=0=[]:[] snapc=0=[]) v11 -- ?+17204 0x29e1f400 con 0x22fefdc0 -5134> 2014-11-03 11:37:52.381756 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- osd_sub_op(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] v 105950'14083247 snapset=0=[]:[] snapc=0=[]) v11 -- ?+17204 0x29e11000 con 0x230a92c0 -5133> 2014-11-03 11:37:52.381994 7f290001b700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.381993, event: commit_queued_for_journal_write, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5132> 2014-11-03 11:37:52.383719 7f28f0a96700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 94 ==== osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) v2 ==== 177+0+0 (1930734198 0 0) 0x2e4b7840 con 0x230a92c0 -5131> 2014-11-03 11:37:52.383798 7f28f0a96700 5 -- op tracker -- seq: 3035, time: 2014-11-03 11:37:52.383589, event: header_read, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5130> 2014-11-03 11:37:52.383824 7f28f0a96700 5 -- op tracker -- seq: 3035, time: 2014-11-03 11:37:52.383591, event: throttled, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5129> 2014-11-03 11:37:52.383874 7f28f0a96700 5 -- op tracker -- seq: 3035, time: 2014-11-03 11:37:52.383705, event: all_read, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5128> 2014-11-03 11:37:52.383889 7f28f0a96700 5 -- op tracker -- seq: 3035, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5127> 2014-11-03 11:37:52.384031 7f28fd816700 5 -- op tracker -- seq: 3035, time: 2014-11-03 11:37:52.384031, event: reached_pg, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5126> 2014-11-03 11:37:52.384116 7f28fd816700 5 -- op tracker -- seq: 3035, time: 2014-11-03 11:37:52.384115, event: started, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5125> 2014-11-03 11:37:52.384146 7f28fd816700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.384146, event: sub_op_commit_rec, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5124> 2014-11-03 11:37:52.384279 7f28fd816700 5 -- op tracker -- seq: 3035, time: 2014-11-03 11:37:52.384279, event: done, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5123> 2014-11-03 11:37:52.384879 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 166 ==== osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) v2 ==== 177+0+0 (4007511590 0 0) 0x2aa6c080 con 0x22fefdc0 -5122> 2014-11-03 11:37:52.384915 7f28f36c2700 5 -- op tracker -- seq: 3036, time: 2014-11-03 11:37:52.384704, event: header_read, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5121> 2014-11-03 11:37:52.384928 7f28f36c2700 5 -- op tracker -- seq: 3036, time: 2014-11-03 11:37:52.384707, event: throttled, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5120> 2014-11-03 11:37:52.384943 7f28f36c2700 5 -- op tracker -- seq: 3036, time: 2014-11-03 11:37:52.384828, event: all_read, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5119> 2014-11-03 11:37:52.384951 7f28f36c2700 5 -- op tracker -- seq: 3036, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5118> 2014-11-03 11:37:52.385047 7f290001b700 5 -- op tracker -- seq: 3036, time: 2014-11-03 11:37:52.385047, event: reached_pg, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5117> 2014-11-03 11:37:52.385080 7f290001b700 5 -- op tracker -- seq: 3036, time: 2014-11-03 11:37:52.385080, event: started, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5116> 2014-11-03 11:37:52.385103 7f290001b700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.385103, event: sub_op_commit_rec, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5115> 2014-11-03 11:37:52.385134 7f290001b700 5 -- op tracker -- seq: 3036, time: 2014-11-03 11:37:52.385134, event: done, op: osd_sub_op_reply(client.7868459.0:14680525 6.62b b2ffb62b/rbd_data.3f64e796ffdc2.00000000000101e8/head//6 [] ondisk, result = 0) -5114> 2014-11-03 11:37:52.385479 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 111 ==== osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3727205592 0 3511520646) 0x27072400 con 0x22fb76e0 -5113> 2014-11-03 11:37:52.385509 7f28edd69700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.385345, event: header_read, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5112> 2014-11-03 11:37:52.385525 7f28edd69700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.385347, event: throttled, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5111> 2014-11-03 11:37:52.385538 7f28edd69700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.385464, event: all_read, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5110> 2014-11-03 11:37:52.385550 7f28edd69700 5 -- op tracker -- seq: 3037, time: 0.000000, event: dispatched, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5109> 2014-11-03 11:37:52.385600 7f28fd816700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.385600, event: reached_pg, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5108> 2014-11-03 11:37:52.385629 7f28fd816700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.385628, event: started, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5107> 2014-11-03 11:37:52.385743 7f28fd816700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.385742, event: started, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5106> 2014-11-03 11:37:52.385782 7f28fd816700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.385782, event: commit_queued_for_journal_write, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5105> 2014-11-03 11:37:52.387035 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 94 ==== osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (546347704 0 3511520646) 0x2d6e6000 con 0x2308b840 -5104> 2014-11-03 11:37:52.387087 7f28ef985700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.386897, event: header_read, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5103> 2014-11-03 11:37:52.387107 7f28ef985700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.386900, event: throttled, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5102> 2014-11-03 11:37:52.387119 7f28ef985700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.387016, event: all_read, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5101> 2014-11-03 11:37:52.387137 7f28ef985700 5 -- op tracker -- seq: 3038, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5100> 2014-11-03 11:37:52.387196 7f290201f700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.387196, event: reached_pg, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5099> 2014-11-03 11:37:52.387229 7f290201f700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.387229, event: started, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5098> 2014-11-03 11:37:52.387340 7f290201f700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.387340, event: started, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5097> 2014-11-03 11:37:52.387370 7f290201f700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.387369, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5096> 2014-11-03 11:37:52.399609 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MOSDPGPushReply(6.91c 105950 [PushReplyOp(3b05791c/rbd_data.89cd895a911542.0000000000009af9/head//6),PushReplyOp(5045191c/rbd_data.89cd895a911542.000000000000466f/head//6)]) v2 -- ?+0 0x23a9e600 con 0x23000b00 -5095> 2014-11-03 11:37:52.400346 7f291450b700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.400345, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5094> 2014-11-03 11:37:52.400496 7f2911d06700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.400496, event: commit_sent, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5093> 2014-11-03 11:37:52.400542 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x922d340 con 0x22fefdc0 -5092> 2014-11-03 11:37:52.401114 7f2912507700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.401114, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5091> 2014-11-03 11:37:52.401239 7f2912507700 5 -- op tracker -- seq: 3028, time: 2014-11-03 11:37:52.401239, event: done, op: osd_sub_op(client.8703713.0:32254403 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953147 snapset=0=[]:[] snapc=0=[]) -5090> 2014-11-03 11:37:52.403219 7f2914d0c700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.403219, event: write_thread_in_journal_buffer, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5089> 2014-11-03 11:37:52.409369 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 167 ==== osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3558682269 0 3511520646) 0x96eb800 con 0x22fefdc0 -5088> 2014-11-03 11:37:52.409416 7f28f36c2700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.409074, event: header_read, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5087> 2014-11-03 11:37:52.409438 7f28f36c2700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.409077, event: throttled, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5086> 2014-11-03 11:37:52.409453 7f28f36c2700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.409351, event: all_read, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5085> 2014-11-03 11:37:52.409467 7f28f36c2700 5 -- op tracker -- seq: 3039, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5084> 2014-11-03 11:37:52.409579 7f290101d700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.409578, event: reached_pg, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5083> 2014-11-03 11:37:52.409646 7f290101d700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.409645, event: started, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5082> 2014-11-03 11:37:52.409774 7f290101d700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.409774, event: started, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5081> 2014-11-03 11:37:52.409814 7f290101d700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.409814, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5080> 2014-11-03 11:37:52.436455 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 50 ==== osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1605856734 0 3511520646) 0x2bd4f800 con 0x22ff3580 -5079> 2014-11-03 11:37:52.436517 7f28f17a3700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.436280, event: header_read, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5078> 2014-11-03 11:37:52.436568 7f28f17a3700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.436283, event: throttled, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5077> 2014-11-03 11:37:52.436598 7f28f17a3700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.436429, event: all_read, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5076> 2014-11-03 11:37:52.436625 7f28f17a3700 5 -- op tracker -- seq: 3040, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5075> 2014-11-03 11:37:52.436735 7f290001b700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.436735, event: reached_pg, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5074> 2014-11-03 11:37:52.436769 7f290001b700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.436769, event: started, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5073> 2014-11-03 11:37:52.436919 7f290001b700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.436918, event: started, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5072> 2014-11-03 11:37:52.437002 7f290001b700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.437002, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5071> 2014-11-03 11:37:52.442211 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 52 ==== osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (566408230 0 3511520646) 0x2fdf4000 con 0x2301fb00 -5070> 2014-11-03 11:37:52.442260 7f28f31bd700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.441995, event: header_read, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5069> 2014-11-03 11:37:52.442279 7f28f31bd700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.441998, event: throttled, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5068> 2014-11-03 11:37:52.442317 7f28f31bd700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.442188, event: all_read, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5067> 2014-11-03 11:37:52.442341 7f28f31bd700 5 -- op tracker -- seq: 3041, time: 0.000000, event: dispatched, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5066> 2014-11-03 11:37:52.442430 7f28fe818700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.442430, event: reached_pg, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5065> 2014-11-03 11:37:52.442463 7f28fe818700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.442463, event: started, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5064> 2014-11-03 11:37:52.442627 7f28fe818700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.442627, event: started, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5063> 2014-11-03 11:37:52.442675 7f28fe818700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.442674, event: commit_queued_for_journal_write, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5062> 2014-11-03 11:37:52.449299 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.58 192.168.5.5:0/24563 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.448258) v2 ==== 47+0+0 (1344466592 0 0) 0x2579ac00 con 0x2469eb00 -5061> 2014-11-03 11:37:52.449308 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.58 192.168.5.5:0/24563 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.448258) v2 ==== 47+0+0 (1344466592 0 0) 0x236c0a00 con 0x244cbdc0 -5060> 2014-11-03 11:37:52.449392 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/24563 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.448258) v2 -- ?+0 0x2a347a00 con 0x2469eb00 -5059> 2014-11-03 11:37:52.449443 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/24563 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.448258) v2 -- ?+0 0x253e5c00 con 0x244cbdc0 -5058> 2014-11-03 11:37:52.460191 7f291450b700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.460190, event: journaled_completion_queued, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5057> 2014-11-03 11:37:52.460431 7f2911d06700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.460431, event: commit_sent, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5056> 2014-11-03 11:37:52.460482 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6810/5438 -- osd_sub_op_reply(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] ondisk, result = 0) v2 -- ?+0 0x922c2c0 con 0x231ab6e0 -5055> 2014-11-03 11:37:52.460883 7f2914d0c700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.460883, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5054> 2014-11-03 11:37:52.465531 7f2912507700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.465530, event: sub_op_applied, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5053> 2014-11-03 11:37:52.465551 7f2914d0c700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.465551, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5052> 2014-11-03 11:37:52.465672 7f2912507700 5 -- op tracker -- seq: 3030, time: 2014-11-03 11:37:52.465672, event: done, op: osd_sub_op(client.10099736.0:11189804 24.a60 25eb8a60/rbd_data.5dd32f2ae8944a.0000000000001642/head//24 [] v 105950'190377 snapset=0=[]:[] snapc=0=[]) -5051> 2014-11-03 11:37:52.465776 7f2914d0c700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.465776, event: write_thread_in_journal_buffer, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5050> 2014-11-03 11:37:52.465843 7f2914d0c700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.465843, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5049> 2014-11-03 11:37:52.465891 7f2914d0c700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.465891, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5048> 2014-11-03 11:37:52.465940 7f2914d0c700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.465940, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5047> 2014-11-03 11:37:52.465987 7f2914d0c700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.465987, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5046> 2014-11-03 11:37:52.466031 7f2914d0c700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.466031, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5045> 2014-11-03 11:37:52.469605 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.11 192.168.5.1:0/18824 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.469108) v2 ==== 47+0+0 (3261458022 0 0) 0x244ab800 con 0x238b8dc0 -5044> 2014-11-03 11:37:52.469675 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/18824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.469108) v2 -- ?+0 0x2579ac00 con 0x238b8dc0 -5043> 2014-11-03 11:37:52.469767 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.11 192.168.5.1:0/18824 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.469108) v2 ==== 47+0+0 (3261458022 0 0) 0x26bdb000 con 0x244ce000 -5042> 2014-11-03 11:37:52.469823 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/18824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.469108) v2 -- ?+0 0x236c0a00 con 0x244ce000 -5041> 2014-11-03 11:37:52.501005 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2f94cdc0 sd=36 :6802 s=0 pgs=0 cs=0 l=0 c=0x250a02c0).accept sd=36 192.168.5.2:42915/0 -5040> 2014-11-03 11:37:52.501791 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450862 192.168.5.2:0/1032669 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23414400 con 0x250a02c0 -5039> 2014-11-03 11:37:52.502375 7f28fb812700 0 osd.21 105950 do_command r=0 -5038> 2014-11-03 11:37:52.502401 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1032669 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x29111c40 con 0x250a02c0 -5037> 2014-11-03 11:37:52.510805 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 51 ==== osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2785351189 0 3511520646) 0x2bd4ec00 con 0x22ff3580 -5036> 2014-11-03 11:37:52.510871 7f28f17a3700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.510577, event: header_read, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5035> 2014-11-03 11:37:52.510895 7f28f17a3700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.510583, event: throttled, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5034> 2014-11-03 11:37:52.510915 7f28f17a3700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.510776, event: all_read, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5033> 2014-11-03 11:37:52.510927 7f28f17a3700 5 -- op tracker -- seq: 3042, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5032> 2014-11-03 11:37:52.511075 7f28fd816700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.511075, event: reached_pg, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5031> 2014-11-03 11:37:52.511111 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.9 192.168.5.1:0/17062 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.510267) v2 ==== 47+0+0 (2264213740 0 0) 0x1adc4400 con 0x24321580 -5030> 2014-11-03 11:37:52.511173 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/17062 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.510267) v2 -- ?+0 0x244ab800 con 0x24321580 -5029> 2014-11-03 11:37:52.511260 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.9 192.168.5.1:0/17062 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.510267) v2 ==== 47+0+0 (2264213740 0 0) 0x5f22200 con 0x24292580 -5028> 2014-11-03 11:37:52.511134 7f28fd816700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.511134, event: started, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5027> 2014-11-03 11:37:52.511320 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/17062 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.510267) v2 -- ?+0 0x26bdb000 con 0x24292580 -5026> 2014-11-03 11:37:52.511463 7f28fd816700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.511462, event: started, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5025> 2014-11-03 11:37:52.511529 7f28fd816700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.511529, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5024> 2014-11-03 11:37:52.536853 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.80 192.168.5.6:0/21329 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.535830) v2 ==== 47+0+0 (882984161 0 0) 0x256b9a00 con 0x249252c0 -5023> 2014-11-03 11:37:52.536949 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.80 192.168.5.6:0/21329 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.535830) v2 ==== 47+0+0 (882984161 0 0) 0x26c59e00 con 0x24229000 -5022> 2014-11-03 11:37:52.536963 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/21329 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.535830) v2 -- ?+0 0x5f22200 con 0x249252c0 -5021> 2014-11-03 11:37:52.537193 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/21329 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.535830) v2 -- ?+0 0x1adc4400 con 0x24229000 -5020> 2014-11-03 11:37:52.554877 7f291450b700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.554876, event: journaled_completion_queued, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5019> 2014-11-03 11:37:52.554933 7f291450b700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.554933, event: journaled_completion_queued, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5018> 2014-11-03 11:37:52.554953 7f291450b700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.554953, event: journaled_completion_queued, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5017> 2014-11-03 11:37:52.554980 7f291450b700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.554979, event: journaled_completion_queued, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5016> 2014-11-03 11:37:52.554998 7f291450b700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.554998, event: journaled_completion_queued, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5015> 2014-11-03 11:37:52.555030 7f291450b700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.555030, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -5014> 2014-11-03 11:37:52.555066 7f291450b700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.555065, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -5013> 2014-11-03 11:37:52.555087 7f291450b700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.555087, event: journaled_completion_queued, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -5012> 2014-11-03 11:37:52.555174 7f2911d06700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.555174, event: commit_sent, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -5011> 2014-11-03 11:37:52.555210 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] ondisk, result = 0) v2 -- ?+0 0x922d080 con 0x23000840 -5010> 2014-11-03 11:37:52.555191 7f2914d0c700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.555191, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -5009> 2014-11-03 11:37:52.555255 7f2911d06700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.555254, event: commit_sent, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -5008> 2014-11-03 11:37:52.555282 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2dfba000 con 0x23000840 -5007> 2014-11-03 11:37:52.555486 7f2911d06700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.555486, event: op_commit, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5006> 2014-11-03 11:37:52.555538 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.91:0/2032747 -- osd_op_reply(14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] v105950'14083247 uv14083247 ondisk = 0) v6 -- ?+0 0x2b2e1080 con 0x2dc1ddc0 -5005> 2014-11-03 11:37:52.555582 7f2911d06700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.555582, event: commit_sent, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -5004> 2014-11-03 11:37:52.555607 7f2911d06700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.555607, event: commit_sent, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -5003> 2014-11-03 11:37:52.555633 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263aeb00 con 0x22fb76e0 -5002> 2014-11-03 11:37:52.555662 7f2911d06700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.555662, event: commit_sent, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -5001> 2014-11-03 11:37:52.555697 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23e43080 con 0x2308b840 -5000> 2014-11-03 11:37:52.555728 7f2911d06700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.555728, event: commit_sent, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -4999> 2014-11-03 11:37:52.555749 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23e42580 con 0x22fefdc0 -4998> 2014-11-03 11:37:52.555776 7f2911d06700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.555776, event: commit_sent, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -4997> 2014-11-03 11:37:52.555797 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23e42b00 con 0x22ff3580 -4996> 2014-11-03 11:37:52.555827 7f2911d06700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.555827, event: commit_sent, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -4995> 2014-11-03 11:37:52.555849 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23e422c0 con 0x2301fb00 -4994> 2014-11-03 11:37:52.566648 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 89 ==== osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1183814023 0 3511520646) 0x2ce3bc00 con 0x22ff3b00 -4993> 2014-11-03 11:37:52.566697 7f28f0d99700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.566484, event: header_read, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4992> 2014-11-03 11:37:52.566717 7f28f0d99700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.566488, event: throttled, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4991> 2014-11-03 11:37:52.566730 7f28f0d99700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.566629, event: all_read, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4990> 2014-11-03 11:37:52.566756 7f28f0d99700 5 -- op tracker -- seq: 3043, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4989> 2014-11-03 11:37:52.566841 7f290001b700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.566841, event: reached_pg, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4988> 2014-11-03 11:37:52.566901 7f290001b700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.566901, event: started, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4987> 2014-11-03 11:37:52.567032 7f290001b700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.567032, event: started, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4986> 2014-11-03 11:37:52.567070 7f290001b700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.567070, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4985> 2014-11-03 11:37:52.572802 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450862 192.168.5.2:0/1032669 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23640600 con 0x250a02c0 -4984> 2014-11-03 11:37:52.573820 7f28fb812700 0 osd.21 105950 do_command r=0 -4983> 2014-11-03 11:37:52.573853 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1032669 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x4e2a380 con 0x250a02c0 -4982> 2014-11-03 11:37:52.580693 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1032669 pipe(0x2f94cdc0 sd=36 :6802 s=2 pgs=2 cs=1 l=1 c=0x250a02c0).reader couldn't read tag, (0) Success -4981> 2014-11-03 11:37:52.580773 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1032669 pipe(0x2f94cdc0 sd=36 :6802 s=2 pgs=2 cs=1 l=1 c=0x250a02c0).fault (0) Success -4980> 2014-11-03 11:37:52.580911 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x250a02c0 session 0x78e6c80 -4979> 2014-11-03 11:37:52.581093 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 51 ==== osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1768902252 0 3511520646) 0x2cff3c00 con 0x22fbc160 -4978> 2014-11-03 11:37:52.581157 7f28f6df9700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.580908, event: header_read, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4977> 2014-11-03 11:37:52.581190 7f28f6df9700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.580912, event: throttled, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4976> 2014-11-03 11:37:52.581206 7f28f6df9700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.581068, event: all_read, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4975> 2014-11-03 11:37:52.581226 7f28f6df9700 5 -- op tracker -- seq: 3044, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4974> 2014-11-03 11:37:52.581312 7f290101d700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.581312, event: reached_pg, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4973> 2014-11-03 11:37:52.581365 7f290101d700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.581365, event: started, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4972> 2014-11-03 11:37:52.581535 7f290101d700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.581535, event: started, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4971> 2014-11-03 11:37:52.581586 7f290101d700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.581585, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4970> 2014-11-03 11:37:52.584785 7f291450b700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.584784, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -4969> 2014-11-03 11:37:52.584895 7f2911d06700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.584895, event: commit_sent, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -4968> 2014-11-03 11:37:52.584928 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23e42dc0 con 0x22ff3580 -4967> 2014-11-03 11:37:52.584916 7f2914d0c700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.584916, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4966> 2014-11-03 11:37:52.584974 7f2914d0c700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.584974, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4965> 2014-11-03 11:37:52.586550 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MOSDPGPushReply(6.91c 105950 [PushReplyOp(66bad91c/rbd_data.4eeda241ca92c8.00000000000036d7/head//6),PushReplyOp(d584391c/rbd_data.7b451f5a03917.0000000000000fb1/head//6)]) v2 -- ?+0 0x2b950c00 con 0x23000b00 -4964> 2014-11-03 11:37:52.590528 7f2912507700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.590528, event: sub_op_applied, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -4963> 2014-11-03 11:37:52.590602 7f2912507700 5 -- op tracker -- seq: 3032, time: 2014-11-03 11:37:52.590602, event: done, op: osd_sub_op(client.7912580.0:17283115 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931201 snapset=0=[]:[] snapc=0=[]) -4962> 2014-11-03 11:37:52.590703 7f2912507700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.590703, event: sub_op_applied, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -4961> 2014-11-03 11:37:52.590737 7f2912507700 5 -- op tracker -- seq: 3033, time: 2014-11-03 11:37:52.590736, event: done, op: osd_sub_op(client.7912580.0:17283116 6.a23 f3b86a23/rbd_data.74ae9c3be03aff.0000000000001087/head//6 [] v 105950'7931202 snapset=0=[]:[] snapc=0=[]) -4960> 2014-11-03 11:37:52.591102 7f2912507700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.591102, event: op_applied, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -4959> 2014-11-03 11:37:52.591181 7f2912507700 5 -- op tracker -- seq: 3034, time: 2014-11-03 11:37:52.591181, event: done, op: osd_op(client.7868459.0:14680525 rbd_data.3f64e796ffdc2.00000000000101e8 [set-alloc-hint object_size 4194304 write_size 4194304,write 1506816~16384] 6.b2ffb62b ack+ondisk+write e105950) -4958> 2014-11-03 11:37:52.591291 7f2912507700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.591291, event: sub_op_applied, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -4957> 2014-11-03 11:37:52.591337 7f2912507700 5 -- op tracker -- seq: 3037, time: 2014-11-03 11:37:52.591337, event: done, op: osd_sub_op(client.8036651.0:1068099 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288360 snapset=0=[]:[] snapc=0=[]) -4956> 2014-11-03 11:37:52.591494 7f2912507700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.591494, event: sub_op_applied, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -4955> 2014-11-03 11:37:52.591544 7f2912507700 5 -- op tracker -- seq: 3038, time: 2014-11-03 11:37:52.591544, event: done, op: osd_sub_op(client.7868671.0:3458113 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900012 snapset=0=[]:[] snapc=0=[]) -4954> 2014-11-03 11:37:52.591679 7f2912507700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.591679, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -4953> 2014-11-03 11:37:52.591728 7f2912507700 5 -- op tracker -- seq: 3039, time: 2014-11-03 11:37:52.591728, event: done, op: osd_sub_op(client.8703713.0:32254408 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953148 snapset=0=[]:[] snapc=0=[]) -4952> 2014-11-03 11:37:52.591935 7f2912507700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.591935, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -4951> 2014-11-03 11:37:52.591965 7f2912507700 5 -- op tracker -- seq: 3040, time: 2014-11-03 11:37:52.591965, event: done, op: osd_sub_op(client.7914641.0:33713598 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649554 snapset=0=[]:[] snapc=0=[]) -4950> 2014-11-03 11:37:52.592105 7f2912507700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.592105, event: sub_op_applied, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -4949> 2014-11-03 11:37:52.592134 7f2912507700 5 -- op tracker -- seq: 3041, time: 2014-11-03 11:37:52.592134, event: done, op: osd_sub_op(client.8393371.0:2616624 24.98a 6164a98a/rbd_data.534b342ae8944a.0000000000000042/head//24 [] v 105950'1027709 snapset=0=[]:[] snapc=0=[]) -4948> 2014-11-03 11:37:52.592268 7f2912507700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.592268, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -4947> 2014-11-03 11:37:52.592290 7f2912507700 5 -- op tracker -- seq: 3042, time: 2014-11-03 11:37:52.592290, event: done, op: osd_sub_op(client.7914641.0:33713616 6.ad6 d4d51ad6/rbd_data.49e9f52ae8944a.00000000000012bd/head//6 [] v 105950'8649555 snapset=0=[]:[] snapc=0=[]) -4946> 2014-11-03 11:37:52.604874 7f291450b700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.604874, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4945> 2014-11-03 11:37:52.604912 7f291450b700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.604912, event: journaled_completion_queued, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4944> 2014-11-03 11:37:52.604974 7f2911d06700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.604974, event: commit_sent, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4943> 2014-11-03 11:37:52.605037 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x27804340 con 0x22ff3b00 -4942> 2014-11-03 11:37:52.605132 7f2911d06700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.605132, event: commit_sent, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4941> 2014-11-03 11:37:52.605164 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- osd_sub_op_reply(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] ondisk, result = 0) v2 -- ?+0 0x278032c0 con 0x22fbc160 -4940> 2014-11-03 11:37:52.605389 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MOSDPGPushReply(6.91c 105950 [PushReplyOp(e330e91c/rbd_data.766a5456a2a64d.00000000000034f8/head//6)]) v2 -- ?+0 0x26a9f000 con 0x23000b00 -4939> 2014-11-03 11:37:52.605453 7f2912507700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.605453, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4938> 2014-11-03 11:37:52.605521 7f2912507700 5 -- op tracker -- seq: 3043, time: 2014-11-03 11:37:52.605521, event: done, op: osd_sub_op(client.12146205.0:439309 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222068 snapset=0=[]:[] snapc=0=[]) -4937> 2014-11-03 11:37:52.605657 7f2912507700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.605657, event: sub_op_applied, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4936> 2014-11-03 11:37:52.605706 7f2912507700 5 -- op tracker -- seq: 3044, time: 2014-11-03 11:37:52.605706, event: done, op: osd_sub_op(client.7869020.0:714675 6.5ca 5a673dca/rbd_data.4a387d2ae8944a.000000000000002b/head//6 [] v 105950'9539479 snapset=0=[]:[] snapc=0=[]) -4935> 2014-11-03 11:37:52.614049 7f28f57e3700 1 -- 192.168.5.2:6803/7606 <== osd.40 192.168.5.3:6868/25675 40 ==== osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3070290635 0 3511520646) 0x2dcb2800 con 0x22fea580 -4934> 2014-11-03 11:37:52.614087 7f28f57e3700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.613920, event: header_read, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4933> 2014-11-03 11:37:52.614102 7f28f57e3700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.613923, event: throttled, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4932> 2014-11-03 11:37:52.614141 7f28f57e3700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.614031, event: all_read, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4931> 2014-11-03 11:37:52.614154 7f28f57e3700 5 -- op tracker -- seq: 3045, time: 0.000000, event: dispatched, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4930> 2014-11-03 11:37:52.614238 7f28ff81a700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.614238, event: reached_pg, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4929> 2014-11-03 11:37:52.614272 7f28ff81a700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.614272, event: started, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4928> 2014-11-03 11:37:52.614375 7f28ff81a700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.614375, event: started, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4927> 2014-11-03 11:37:52.614410 7f28ff81a700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.614410, event: commit_queued_for_journal_write, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4926> 2014-11-03 11:37:52.614492 7f2914d0c700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.614492, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4925> 2014-11-03 11:37:52.618647 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.29 192.168.5.3:0/1192 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.617781) v2 ==== 47+0+0 (136265348 0 0) 0x1a327c00 con 0x23335000 -4924> 2014-11-03 11:37:52.618695 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/1192 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.617781) v2 -- ?+0 0x256b9a00 con 0x23335000 -4923> 2014-11-03 11:37:52.618889 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.29 192.168.5.3:0/1192 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.617781) v2 ==== 47+0+0 (136265348 0 0) 0x236a8000 con 0x234ed000 -4922> 2014-11-03 11:37:52.618927 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/1192 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.617781) v2 -- ?+0 0x26c59e00 con 0x234ed000 -4921> 2014-11-03 11:37:52.629651 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 46 ==== MOSDPGPush(6.91c 105950 [PushOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6, version: 105946'10710970, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6@105946'10710970, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6, version: 105946'10710983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6@105946'10710983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390505+0+0 (2856482097 0 0) 0x23a9e600 con 0x23000b00 -4920> 2014-11-03 11:37:52.629724 7f28f21ad700 5 -- op tracker -- seq: 3046, time: 2014-11-03 11:37:52.526912, event: header_read, op: MOSDPGPush(6.91c 105950 [PushOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6, version: 105946'10710970, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6@105946'10710970, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6, version: 105946'10710983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6@105946'10710983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4919> 2014-11-03 11:37:52.629762 7f28f21ad700 5 -- op tracker -- seq: 3046, time: 2014-11-03 11:37:52.526919, event: throttled, op: MOSDPGPush(6.91c 105950 [PushOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6, version: 105946'10710970, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6@105946'10710970, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6, version: 105946'10710983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6@105946'10710983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4918> 2014-11-03 11:37:52.629791 7f28f21ad700 5 -- op tracker -- seq: 3046, time: 2014-11-03 11:37:52.629628, event: all_read, op: MOSDPGPush(6.91c 105950 [PushOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6, version: 105946'10710970, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6@105946'10710970, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6, version: 105946'10710983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6@105946'10710983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4917> 2014-11-03 11:37:52.629814 7f28f21ad700 5 -- op tracker -- seq: 3046, time: 0.000000, event: dispatched, op: MOSDPGPush(6.91c 105950 [PushOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6, version: 105946'10710970, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6@105946'10710970, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6, version: 105946'10710983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6@105946'10710983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4916> 2014-11-03 11:37:52.629900 7f28fe818700 5 -- op tracker -- seq: 3046, time: 2014-11-03 11:37:52.629900, event: reached_pg, op: MOSDPGPush(6.91c 105950 [PushOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6, version: 105946'10710970, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6@105946'10710970, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6, version: 105946'10710983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6@105946'10710983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4915> 2014-11-03 11:37:52.630123 7f28fe818700 5 -- op tracker -- seq: 3046, time: 2014-11-03 11:37:52.630123, event: done, op: MOSDPGPush(6.91c 105950 [PushOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6, version: 105946'10710970, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6@105946'10710970, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6, version: 105946'10710983, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6@105946'10710983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4914> 2014-11-03 11:37:52.640812 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 98 ==== osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3197025786 0 3511520646) 0x2fe0c000 con 0x22fbcb00 -4913> 2014-11-03 11:37:52.640846 7f28f6cf8700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.640698, event: header_read, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4912> 2014-11-03 11:37:52.640866 7f28f6cf8700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.640700, event: throttled, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4911> 2014-11-03 11:37:52.640881 7f28f6cf8700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.640802, event: all_read, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4910> 2014-11-03 11:37:52.640895 7f28f6cf8700 5 -- op tracker -- seq: 3047, time: 0.000000, event: dispatched, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4909> 2014-11-03 11:37:52.640955 7f28fe017700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.640955, event: reached_pg, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4908> 2014-11-03 11:37:52.640983 7f28fe017700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.640983, event: started, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4907> 2014-11-03 11:37:52.641085 7f28fe017700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.641085, event: started, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4906> 2014-11-03 11:37:52.641114 7f28fe017700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.641114, event: commit_queued_for_journal_write, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4905> 2014-11-03 11:37:52.642547 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 99 ==== osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1667963501 0 3511520646) 0x2ce3b000 con 0x22fbcb00 -4904> 2014-11-03 11:37:52.642582 7f28f6cf8700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.642429, event: header_read, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4903> 2014-11-03 11:37:52.642599 7f28f6cf8700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.642431, event: throttled, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4902> 2014-11-03 11:37:52.642616 7f28f6cf8700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.642537, event: all_read, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4901> 2014-11-03 11:37:52.642624 7f28f6cf8700 5 -- op tracker -- seq: 3048, time: 0.000000, event: dispatched, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4900> 2014-11-03 11:37:52.642679 7f290081c700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.642678, event: reached_pg, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4899> 2014-11-03 11:37:52.642719 7f290081c700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.642719, event: started, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4898> 2014-11-03 11:37:52.642827 7f290081c700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.642827, event: started, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4897> 2014-11-03 11:37:52.642860 7f290081c700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.642860, event: commit_queued_for_journal_write, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4896> 2014-11-03 11:37:52.653513 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 47 ==== MOSDPGPush(6.91c 105950 [PushOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6, version: 105946'10710996, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6@105946'10710996, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1767091377 0 0) 0x26a9f000 con 0x23000b00 -4895> 2014-11-03 11:37:52.653581 7f28f21ad700 5 -- op tracker -- seq: 3049, time: 2014-11-03 11:37:52.629916, event: header_read, op: MOSDPGPush(6.91c 105950 [PushOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6, version: 105946'10710996, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6@105946'10710996, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4894> 2014-11-03 11:37:52.653619 7f28f21ad700 5 -- op tracker -- seq: 3049, time: 2014-11-03 11:37:52.629917, event: throttled, op: MOSDPGPush(6.91c 105950 [PushOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6, version: 105946'10710996, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6@105946'10710996, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4893> 2014-11-03 11:37:52.653652 7f28f21ad700 5 -- op tracker -- seq: 3049, time: 2014-11-03 11:37:52.653489, event: all_read, op: MOSDPGPush(6.91c 105950 [PushOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6, version: 105946'10710996, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6@105946'10710996, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4892> 2014-11-03 11:37:52.653680 7f28f21ad700 5 -- op tracker -- seq: 3049, time: 0.000000, event: dispatched, op: MOSDPGPush(6.91c 105950 [PushOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6, version: 105946'10710996, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6@105946'10710996, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4891> 2014-11-03 11:37:52.653787 7f290101d700 5 -- op tracker -- seq: 3049, time: 2014-11-03 11:37:52.653787, event: reached_pg, op: MOSDPGPush(6.91c 105950 [PushOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6, version: 105946'10710996, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6@105946'10710996, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4890> 2014-11-03 11:37:52.654008 7f290101d700 5 -- op tracker -- seq: 3049, time: 2014-11-03 11:37:52.654008, event: done, op: MOSDPGPush(6.91c 105950 [PushOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6, version: 105946'10710996, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6@105946'10710996, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4889> 2014-11-03 11:37:52.656040 7f291450b700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.656039, event: journaled_completion_queued, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4888> 2014-11-03 11:37:52.656127 7f2911d06700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.656127, event: commit_sent, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4887> 2014-11-03 11:37:52.656166 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6868/25675 -- osd_sub_op_reply(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] ondisk, result = 0) v2 -- ?+0 0x278048c0 con 0x22fea580 -4886> 2014-11-03 11:37:52.656414 7f2912507700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.656413, event: sub_op_applied, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4885> 2014-11-03 11:37:52.656461 7f2912507700 5 -- op tracker -- seq: 3045, time: 2014-11-03 11:37:52.656461, event: done, op: osd_sub_op(client.8689523.0:158110 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182997 snapset=0=[]:[] snapc=0=[]) -4884> 2014-11-03 11:37:52.658244 7f2914d0c700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.658244, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4883> 2014-11-03 11:37:52.658296 7f2914d0c700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.658296, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4882> 2014-11-03 11:37:52.706324 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.73 192.168.5.6:0/1260 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.705048) v2 ==== 47+0+0 (3007399208 0 0) 0x6425000 con 0x242c0dc0 -4881> 2014-11-03 11:37:52.706350 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.73 192.168.5.6:0/1260 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.705048) v2 ==== 47+0+0 (3007399208 0 0) 0x1a327400 con 0x23b1c6e0 -4880> 2014-11-03 11:37:52.706393 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/1260 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.705048) v2 -- ?+0 0x236a8000 con 0x242c0dc0 -4879> 2014-11-03 11:37:52.706489 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/1260 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.705048) v2 -- ?+0 0x1a327c00 con 0x23b1c6e0 -4878> 2014-11-03 11:37:52.724064 7f291450b700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.724064, event: journaled_completion_queued, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4877> 2014-11-03 11:37:52.724109 7f291450b700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.724108, event: journaled_completion_queued, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4876> 2014-11-03 11:37:52.724129 7f2911d06700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.724129, event: commit_sent, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4875> 2014-11-03 11:37:52.724159 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x27803000 con 0x22fbcb00 -4874> 2014-11-03 11:37:52.724188 7f2911d06700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.724188, event: commit_sent, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4873> 2014-11-03 11:37:52.724280 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258be080 con 0x22fbcb00 -4872> 2014-11-03 11:37:52.724314 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_trim(6.91c to 105950'10711003 e105950) v1 -- ?+0 0x2a6e8c00 con 0x23000b00 -4871> 2014-11-03 11:37:52.724689 7f2912507700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.724689, event: sub_op_applied, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4870> 2014-11-03 11:37:52.724751 7f2912507700 5 -- op tracker -- seq: 3047, time: 2014-11-03 11:37:52.724751, event: done, op: osd_sub_op(client.7928471.0:2008404 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134373 snapset=0=[]:[] snapc=0=[]) -4869> 2014-11-03 11:37:52.724863 7f2912507700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.724863, event: sub_op_applied, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4868> 2014-11-03 11:37:52.724915 7f2912507700 5 -- op tracker -- seq: 3048, time: 2014-11-03 11:37:52.724915, event: done, op: osd_sub_op(client.7928471.0:2008405 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134374 snapset=0=[]:[] snapc=0=[]) -4867> 2014-11-03 11:37:52.729754 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 32 ==== MOSDPGPush(6.58e 105950 [PushOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, version: 105946'11622356, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1948395030 0 0) 0x1adc4600 con 0x231abb00 -4866> 2014-11-03 11:37:52.729825 7f28eb945700 5 -- op tracker -- seq: 3050, time: 2014-11-03 11:37:52.714208, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, version: 105946'11622356, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4865> 2014-11-03 11:37:52.729865 7f28eb945700 5 -- op tracker -- seq: 3050, time: 2014-11-03 11:37:52.714210, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, version: 105946'11622356, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4864> 2014-11-03 11:37:52.729897 7f28eb945700 5 -- op tracker -- seq: 3050, time: 2014-11-03 11:37:52.729734, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, version: 105946'11622356, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4863> 2014-11-03 11:37:52.729929 7f28eb945700 5 -- op tracker -- seq: 3050, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, version: 105946'11622356, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4862> 2014-11-03 11:37:52.730053 7f28fe818700 5 -- op tracker -- seq: 3050, time: 2014-11-03 11:37:52.730053, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, version: 105946'11622356, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4861> 2014-11-03 11:37:52.730298 7f28fe818700 5 -- op tracker -- seq: 3050, time: 2014-11-03 11:37:52.730298, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6, version: 105946'11622356, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2c1c2d8e/rbd_data.78d1cb2ae8944a.0000000000000c4d/head//6@105946'11622356, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4860> 2014-11-03 11:37:52.767854 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.102 192.168.5.8:0/21153 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.766899) v2 ==== 47+0+0 (216167274 0 0) 0x2743a600 con 0x233cb160 -4859> 2014-11-03 11:37:52.767892 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.102 192.168.5.8:0/21153 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.766899) v2 ==== 47+0+0 (216167274 0 0) 0x236c0600 con 0x233cb580 -4858> 2014-11-03 11:37:52.767907 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.766899) v2 -- ?+0 0x1a327400 con 0x233cb160 -4857> 2014-11-03 11:37:52.768037 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21153 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.766899) v2 -- ?+0 0x6425000 con 0x233cb580 -4856> 2014-11-03 11:37:52.783553 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.75 192.168.5.6:0/12612 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.781934) v2 ==== 47+0+0 (3011845962 0 0) 0x1ae44e00 con 0x239f8420 -4855> 2014-11-03 11:37:52.783629 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/12612 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.781934) v2 -- ?+0 0x2743a600 con 0x239f8420 -4854> 2014-11-03 11:37:52.783556 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.75 192.168.5.6:0/12612 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.781934) v2 ==== 47+0+0 (3011845962 0 0) 0x245ff000 con 0x238b8580 -4853> 2014-11-03 11:37:52.783709 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/12612 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.781934) v2 -- ?+0 0x236c0600 con 0x238b8580 -4852> 2014-11-03 11:37:52.785094 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 168 ==== osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (52434083 0 3511520646) 0x29e1dc00 con 0x22fefdc0 -4851> 2014-11-03 11:37:52.785127 7f28f36c2700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.784930, event: header_read, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4850> 2014-11-03 11:37:52.785144 7f28f36c2700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.784935, event: throttled, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4849> 2014-11-03 11:37:52.785153 7f28f36c2700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.785081, event: all_read, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4848> 2014-11-03 11:37:52.785165 7f28f36c2700 5 -- op tracker -- seq: 3051, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4847> 2014-11-03 11:37:52.785262 7f290101d700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.785262, event: reached_pg, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4846> 2014-11-03 11:37:52.785309 7f290101d700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.785309, event: started, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4845> 2014-11-03 11:37:52.785444 7f290101d700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.785444, event: started, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4844> 2014-11-03 11:37:52.785490 7f290101d700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.785490, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4843> 2014-11-03 11:37:52.797612 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 95 ==== osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (404278356 0 3511520646) 0x2c336000 con 0x2308b840 -4842> 2014-11-03 11:37:52.797666 7f28ef985700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.797315, event: header_read, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4841> 2014-11-03 11:37:52.797694 7f28ef985700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.797319, event: throttled, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4840> 2014-11-03 11:37:52.797712 7f28ef985700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.797589, event: all_read, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4839> 2014-11-03 11:37:52.797729 7f28ef985700 5 -- op tracker -- seq: 3052, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4838> 2014-11-03 11:37:52.797813 7f28fe017700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.797813, event: reached_pg, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4837> 2014-11-03 11:37:52.797842 7f28fe017700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.797842, event: started, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4836> 2014-11-03 11:37:52.797971 7f28fe017700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.797971, event: started, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4835> 2014-11-03 11:37:52.798024 7f28fe017700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.798024, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4834> 2014-11-03 11:37:52.798744 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 96 ==== osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (133072875 0 3511520646) 0x26942000 con 0x2308b840 -4833> 2014-11-03 11:37:52.798783 7f28ef985700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.798637, event: header_read, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4832> 2014-11-03 11:37:52.798795 7f28ef985700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.798639, event: throttled, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4831> 2014-11-03 11:37:52.798803 7f28ef985700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.798726, event: all_read, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4830> 2014-11-03 11:37:52.798818 7f28ef985700 5 -- op tracker -- seq: 3053, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4829> 2014-11-03 11:37:52.798936 7f290081c700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.798934, event: reached_pg, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4828> 2014-11-03 11:37:52.798966 7f290081c700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.798966, event: started, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4827> 2014-11-03 11:37:52.799060 7f290081c700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.799060, event: started, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4826> 2014-11-03 11:37:52.799096 7f290081c700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.799096, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4825> 2014-11-03 11:37:52.814557 7f2914d0c700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.814557, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4824> 2014-11-03 11:37:52.814614 7f2914d0c700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.814614, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4823> 2014-11-03 11:37:52.814648 7f2914d0c700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.814648, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4822> 2014-11-03 11:37:52.818040 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 112 ==== osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2231272538 0 3511520646) 0x2cff3000 con 0x22fb76e0 -4821> 2014-11-03 11:37:52.818077 7f28edd69700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.817936, event: header_read, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4820> 2014-11-03 11:37:52.818092 7f28edd69700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.817938, event: throttled, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4819> 2014-11-03 11:37:52.818140 7f28edd69700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.818026, event: all_read, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4818> 2014-11-03 11:37:52.818162 7f28edd69700 5 -- op tracker -- seq: 3054, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4817> 2014-11-03 11:37:52.818238 7f28fe818700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.818238, event: reached_pg, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4816> 2014-11-03 11:37:52.818318 7f28fe818700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.818318, event: started, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4815> 2014-11-03 11:37:52.818437 7f28fe818700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.818437, event: started, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4814> 2014-11-03 11:37:52.818477 7f28fe818700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.818477, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4813> 2014-11-03 11:37:52.824119 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2f94c580 sd=66 :6802 s=0 pgs=0 cs=0 l=0 c=0x250a0dc0).accept sd=66 192.168.5.201:58676/0 -4812> 2014-11-03 11:37:52.824960 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450865 192.168.5.201:0/1029335 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "24.7d7"}) v1 ==== 80+0+0 (3756934197 0 0) 0x1adc4600 con 0x250a0dc0 -4811> 2014-11-03 11:37:52.825583 7f28fb812700 0 osd.21 105950 do_command r=0 -4810> 2014-11-03 11:37:52.825611 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029335 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x78351c0 con 0x250a0dc0 -4809> 2014-11-03 11:37:52.832205 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450865 192.168.5.201:0/1029335 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "24.7d7"}) v1 ==== 74+0+0 (774687807 0 0) 0x13defa00 con 0x250a0dc0 -4808> 2014-11-03 11:37:52.832410 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x23925580 sd=148 :6802 s=0 pgs=0 cs=0 l=0 c=0x2ce10840).accept sd=148 192.168.5.2:42918/0 -4807> 2014-11-03 11:37:52.832959 7f28fb812700 0 osd.21 105950 do_command r=0 -4806> 2014-11-03 11:37:52.832975 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029335 -- command_reply(tid 2: 0 ) v1 -- ?+13413 0x4e2a1c0 con 0x250a0dc0 -4805> 2014-11-03 11:37:52.833529 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450755 192.168.5.2:0/1000359 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x30fd400 con 0x2ce10840 -4804> 2014-11-03 11:37:52.833915 7f28fb812700 0 osd.21 105950 do_command r=0 -4803> 2014-11-03 11:37:52.833932 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000359 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x78351c0 con 0x2ce10840 -4802> 2014-11-03 11:37:52.839155 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029335 pipe(0x2f94c580 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x250a0dc0).reader couldn't read tag, (0) Success -4801> 2014-11-03 11:37:52.839195 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029335 pipe(0x2f94c580 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x250a0dc0).fault (0) Success -4800> 2014-11-03 11:37:52.839266 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x250a0dc0 session 0x235df000 -4799> 2014-11-03 11:37:52.843956 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450755 192.168.5.2:0/1000359 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2b950600 con 0x2ce10840 -4798> 2014-11-03 11:37:52.845168 7f28fb812700 0 osd.21 105950 do_command r=0 -4797> 2014-11-03 11:37:52.845199 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000359 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0xbfa8e00 con 0x2ce10840 -4796> 2014-11-03 11:37:52.847497 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 97 ==== osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2034944189 0 3511520646) 0x2cff4800 con 0x2308b840 -4795> 2014-11-03 11:37:52.847529 7f28ef985700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.847395, event: header_read, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4794> 2014-11-03 11:37:52.847546 7f28ef985700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.847397, event: throttled, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4793> 2014-11-03 11:37:52.847560 7f28ef985700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.847479, event: all_read, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4792> 2014-11-03 11:37:52.847580 7f28ef985700 5 -- op tracker -- seq: 3055, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4791> 2014-11-03 11:37:52.847643 7f290201f700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.847643, event: reached_pg, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4790> 2014-11-03 11:37:52.847671 7f290201f700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.847671, event: started, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4789> 2014-11-03 11:37:52.847789 7f290201f700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.847789, event: started, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4788> 2014-11-03 11:37:52.847833 7f290201f700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.847833, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4787> 2014-11-03 11:37:52.848226 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 98 ==== osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2583972302 0 3511520646) 0x2cff3c00 con 0x2308b840 -4786> 2014-11-03 11:37:52.848252 7f28ef985700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.848137, event: header_read, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4785> 2014-11-03 11:37:52.848267 7f28ef985700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.848139, event: throttled, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4784> 2014-11-03 11:37:52.848279 7f28ef985700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.848216, event: all_read, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4783> 2014-11-03 11:37:52.848291 7f28ef985700 5 -- op tracker -- seq: 3056, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4782> 2014-11-03 11:37:52.848344 7f28ff81a700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.848344, event: reached_pg, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4781> 2014-11-03 11:37:52.848372 7f28ff81a700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.848372, event: started, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4780> 2014-11-03 11:37:52.848466 7f28ff81a700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.848466, event: started, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4779> 2014-11-03 11:37:52.848495 7f28ff81a700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.848495, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4778> 2014-11-03 11:37:52.852594 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000359 pipe(0x23925580 sd=148 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10840).reader couldn't read tag, (0) Success -4777> 2014-11-03 11:37:52.852673 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000359 pipe(0x23925580 sd=148 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10840).fault (0) Success -4776> 2014-11-03 11:37:52.852790 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2ce10840 session 0x32fb280 -4775> 2014-11-03 11:37:52.866250 7f291450b700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.866250, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4774> 2014-11-03 11:37:52.866300 7f291450b700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.866299, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4773> 2014-11-03 11:37:52.866324 7f2911d06700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.866324, event: commit_sent, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4772> 2014-11-03 11:37:52.866337 7f291450b700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.866336, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4771> 2014-11-03 11:37:52.866375 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x258be600 con 0x22fefdc0 -4770> 2014-11-03 11:37:52.866417 7f2911d06700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.866417, event: commit_sent, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4769> 2014-11-03 11:37:52.866439 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bd2c0 con 0x2308b840 -4768> 2014-11-03 11:37:52.866432 7f2914d0c700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.866431, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4767> 2014-11-03 11:37:52.866502 7f2911d06700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.866502, event: commit_sent, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4766> 2014-11-03 11:37:52.866529 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258bd840 con 0x2308b840 -4765> 2014-11-03 11:37:52.866526 7f2914d0c700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.866526, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4764> 2014-11-03 11:37:52.866596 7f2914d0c700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.866595, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4763> 2014-11-03 11:37:52.868087 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.105 192.168.5.8:0/6777 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.867447) v2 ==== 47+0+0 (724117231 0 0) 0x17ae8600 con 0x31a92c0 -4762> 2014-11-03 11:37:52.868125 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/6777 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.867447) v2 -- ?+0 0x1ae44e00 con 0x31a92c0 -4761> 2014-11-03 11:37:52.868125 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.105 192.168.5.8:0/6777 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.867447) v2 ==== 47+0+0 (724117231 0 0) 0x23307400 con 0x2389f2c0 -4760> 2014-11-03 11:37:52.868188 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/6777 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.867447) v2 -- ?+0 0x245ff000 con 0x2389f2c0 -4759> 2014-11-03 11:37:52.869926 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 169 ==== osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (979362745 0 3511520646) 0x2398c000 con 0x22fefdc0 -4758> 2014-11-03 11:37:52.869996 7f28f36c2700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.869791, event: header_read, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4757> 2014-11-03 11:37:52.870016 7f28f36c2700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.869793, event: throttled, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4756> 2014-11-03 11:37:52.870044 7f28f36c2700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.869913, event: all_read, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4755> 2014-11-03 11:37:52.870053 7f28f36c2700 5 -- op tracker -- seq: 3057, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4754> 2014-11-03 11:37:52.870145 7f290101d700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.870144, event: reached_pg, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4753> 2014-11-03 11:37:52.870177 7f290101d700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.870177, event: started, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4752> 2014-11-03 11:37:52.870259 7f290101d700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.870259, event: started, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4751> 2014-11-03 11:37:52.870286 7f290101d700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.870286, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4750> 2014-11-03 11:37:52.871211 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 42 ==== osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4925 (1728095453 0 2191458794) 0x24662800 con 0x231ab580 -4749> 2014-11-03 11:37:52.871270 7f28ea430700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.871034, event: header_read, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4748> 2014-11-03 11:37:52.871302 7f28ea430700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.871038, event: throttled, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4747> 2014-11-03 11:37:52.871323 7f28ea430700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.871190, event: all_read, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4746> 2014-11-03 11:37:52.871342 7f28ea430700 5 -- op tracker -- seq: 3058, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4745> 2014-11-03 11:37:52.871435 7f28fe017700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.871435, event: reached_pg, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4744> 2014-11-03 11:37:52.871483 7f28fe017700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.871483, event: started, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4743> 2014-11-03 11:37:52.881319 7f28fe017700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.881318, event: started, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4742> 2014-11-03 11:37:52.881392 7f28fe017700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.881392, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4741> 2014-11-03 11:37:52.883475 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 70 ==== osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3285172098 0 3511520646) 0x26c42000 con 0x2308b2c0 -4740> 2014-11-03 11:37:52.883512 7f28eed79700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.883364, event: header_read, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4739> 2014-11-03 11:37:52.883530 7f28eed79700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.883367, event: throttled, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4738> 2014-11-03 11:37:52.883539 7f28eed79700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.883460, event: all_read, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4737> 2014-11-03 11:37:52.883554 7f28eed79700 5 -- op tracker -- seq: 3059, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4736> 2014-11-03 11:37:52.883622 7f290201f700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.883622, event: reached_pg, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4735> 2014-11-03 11:37:52.883662 7f290201f700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.883662, event: started, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4734> 2014-11-03 11:37:52.883778 7f290201f700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.883778, event: started, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4733> 2014-11-03 11:37:52.883806 7f290201f700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.883806, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4732> 2014-11-03 11:37:52.897866 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.92 192.168.5.7:0/5438 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.896153) v2 ==== 47+0+0 (1815761604 0 0) 0x23fdfc00 con 0x239f8840 -4731> 2014-11-03 11:37:52.897928 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/5438 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.896153) v2 -- ?+0 0x17ae8600 con 0x239f8840 -4730> 2014-11-03 11:37:52.897943 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.92 192.168.5.7:0/5438 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.896153) v2 ==== 47+0+0 (1815761604 0 0) 0x2782f400 con 0x238b89a0 -4729> 2014-11-03 11:37:52.897992 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/5438 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.896153) v2 -- ?+0 0x23307400 con 0x238b89a0 -4728> 2014-11-03 11:37:52.910587 7f291450b700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.910587, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4727> 2014-11-03 11:37:52.910629 7f291450b700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.910629, event: journaled_completion_queued, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4726> 2014-11-03 11:37:52.910657 7f291450b700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.910657, event: journaled_completion_queued, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4725> 2014-11-03 11:37:52.910660 7f2911d06700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.910660, event: commit_sent, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4724> 2014-11-03 11:37:52.910704 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0x2b41e840 con 0x22fb76e0 -4723> 2014-11-03 11:37:52.910739 7f2911d06700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.910739, event: commit_sent, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4722> 2014-11-03 11:37:52.910747 7f2914d0c700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.910746, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4721> 2014-11-03 11:37:52.910761 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b41e2c0 con 0x2308b840 -4720> 2014-11-03 11:37:52.910797 7f2911d06700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.910796, event: commit_sent, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4719> 2014-11-03 11:37:52.910823 7f2914d0c700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.910823, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4718> 2014-11-03 11:37:52.910856 7f2914d0c700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.910856, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4717> 2014-11-03 11:37:52.910908 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2ea7adc0 con 0x2308b840 -4716> 2014-11-03 11:37:52.916234 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.118 192.168.5.9:0/23392 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.914886) v2 ==== 47+0+0 (1826730617 0 0) 0x23307600 con 0x23c15000 -4715> 2014-11-03 11:37:52.916283 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/23392 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.914886) v2 -- ?+0 0x23fdfc00 con 0x23c15000 -4714> 2014-11-03 11:37:52.916313 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.118 192.168.5.9:0/23392 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.914886) v2 ==== 47+0+0 (1826730617 0 0) 0x17ae8400 con 0x2469ec60 -4713> 2014-11-03 11:37:52.916356 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/23392 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.914886) v2 -- ?+0 0x2782f400 con 0x2469ec60 -4712> 2014-11-03 11:37:52.919545 7f291450b700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.919544, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4711> 2014-11-03 11:37:52.919543 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MOSDPGPushReply(6.91c 105950 [PushReplyOp(a6a0e91c/rbd_data.3eaf582ae8944a.00000000000001df/head//6),PushReplyOp(e6e6a91c/rbd_data.885435b2bbeeb.000000000000d776/head//6)]) v2 -- ?+0 0x2369f000 con 0x23000b00 -4710> 2014-11-03 11:37:52.919598 7f291450b700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.919598, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4709> 2014-11-03 11:37:52.919604 7f2911d06700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.919603, event: commit_sent, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4708> 2014-11-03 11:37:52.919631 7f291450b700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.919631, event: journaled_completion_queued, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4707> 2014-11-03 11:37:52.919634 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2ea7a000 con 0x22fefdc0 -4706> 2014-11-03 11:37:52.919700 7f2911d06700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.919699, event: commit_sent, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4705> 2014-11-03 11:37:52.919729 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2ea7bb80 con 0x231ab580 -4704> 2014-11-03 11:37:52.919759 7f2911d06700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.919759, event: commit_sent, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4703> 2014-11-03 11:37:52.919780 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2ea7a2c0 con 0x2308b2c0 -4702> 2014-11-03 11:37:52.924553 7f2912507700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.924552, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4701> 2014-11-03 11:37:52.924629 7f2912507700 5 -- op tracker -- seq: 3051, time: 2014-11-03 11:37:52.924629, event: done, op: osd_sub_op(client.8703713.0:32254420 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953149 snapset=0=[]:[] snapc=0=[]) -4700> 2014-11-03 11:37:52.924621 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MOSDPGPull(6.58e 105950 [PullOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x23a92600 con 0x22fbc420 -4699> 2014-11-03 11:37:52.924802 7f2912507700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.924802, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4698> 2014-11-03 11:37:52.924850 7f2912507700 5 -- op tracker -- seq: 3052, time: 2014-11-03 11:37:52.924850, event: done, op: osd_sub_op(client.7867942.0:137197770 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000603 snapset=0=[]:[] snapc=0=[]) -4697> 2014-11-03 11:37:52.924929 7f2912507700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.924929, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4696> 2014-11-03 11:37:52.924972 7f2912507700 5 -- op tracker -- seq: 3053, time: 2014-11-03 11:37:52.924972, event: done, op: osd_sub_op(client.7867942.0:137197771 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000604 snapset=0=[]:[] snapc=0=[]) -4695> 2014-11-03 11:37:52.925127 7f2912507700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.925127, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4694> 2014-11-03 11:37:52.925173 7f2912507700 5 -- op tracker -- seq: 3054, time: 2014-11-03 11:37:52.925173, event: done, op: osd_sub_op(client.12404444.0:39592 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374238 snapset=0=[]:[] snapc=0=[]) -4693> 2014-11-03 11:37:52.925296 7f2912507700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.925296, event: sub_op_applied, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4692> 2014-11-03 11:37:52.925348 7f2912507700 5 -- op tracker -- seq: 3055, time: 2014-11-03 11:37:52.925348, event: done, op: osd_sub_op(client.7869041.0:12286834 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900013 snapset=0=[]:[] snapc=0=[]) -4691> 2014-11-03 11:37:52.925464 7f2912507700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.925464, event: sub_op_applied, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4690> 2014-11-03 11:37:52.925518 7f2912507700 5 -- op tracker -- seq: 3056, time: 2014-11-03 11:37:52.925517, event: done, op: osd_sub_op(client.7869041.0:12286835 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900014 snapset=0=[]:[] snapc=0=[]) -4689> 2014-11-03 11:37:52.925602 7f2912507700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.925601, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4688> 2014-11-03 11:37:52.925656 7f2912507700 5 -- op tracker -- seq: 3057, time: 2014-11-03 11:37:52.925656, event: done, op: osd_sub_op(client.8703713.0:32254425 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953150 snapset=0=[]:[] snapc=0=[]) -4687> 2014-11-03 11:37:52.929246 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.6 192.168.5.1:0/1181 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.928630) v2 ==== 47+0+0 (902559343 0 0) 0x23821600 con 0x24925c60 -4686> 2014-11-03 11:37:52.929296 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/1181 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.928630) v2 -- ?+0 0x23307600 con 0x24925c60 -4685> 2014-11-03 11:37:52.929908 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.6 192.168.5.1:0/1181 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.928630) v2 ==== 47+0+0 (902559343 0 0) 0x2355be00 con 0x242299a0 -4684> 2014-11-03 11:37:52.929954 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/1181 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.928630) v2 -- ?+0 0x17ae8400 con 0x242299a0 -4683> 2014-11-03 11:37:52.931865 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.41 192.168.5.3:0/28523 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.930948) v2 ==== 47+0+0 (2684285580 0 0) 0x2c390800 con 0x23b1c840 -4682> 2014-11-03 11:37:52.931903 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/28523 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.930948) v2 -- ?+0 0x23821600 con 0x23b1c840 -4681> 2014-11-03 11:37:52.931933 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.41 192.168.5.3:0/28523 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.930948) v2 ==== 47+0+0 (2684285580 0 0) 0x23307200 con 0x242c09a0 -4680> 2014-11-03 11:37:52.931974 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/28523 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.930948) v2 -- ?+0 0x2355be00 con 0x242c09a0 -4679> 2014-11-03 11:37:52.932393 7f2912507700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.932393, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4678> 2014-11-03 11:37:52.932483 7f2912507700 5 -- op tracker -- seq: 3058, time: 2014-11-03 11:37:52.932483, event: done, op: osd_sub_op(client.8703713.0:32254428 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97977 snapset=0=[]:[] snapc=0=[]) -4677> 2014-11-03 11:37:52.932727 7f2912507700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.932726, event: sub_op_applied, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4676> 2014-11-03 11:37:52.932758 7f2912507700 5 -- op tracker -- seq: 3059, time: 2014-11-03 11:37:52.932758, event: done, op: osd_sub_op(client.7868477.0:253120 6.456 d07f4456/rbd_data.3eaf7c44fcc836.00000000000000f9/head//6 [] v 105950'31870434 snapset=0=[]:[] snapc=0=[]) -4675> 2014-11-03 11:37:52.948826 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 97 ==== MOSDPGPush(6.58e 105950 [PushOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, version: 105946'11622386, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (3444284896 0 0) 0x23a92600 con 0x22fbc420 -4674> 2014-11-03 11:37:52.948894 7f28f72fe700 5 -- op tracker -- seq: 3060, time: 2014-11-03 11:37:52.929919, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, version: 105946'11622386, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4673> 2014-11-03 11:37:52.948928 7f28f72fe700 5 -- op tracker -- seq: 3060, time: 2014-11-03 11:37:52.929921, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, version: 105946'11622386, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4672> 2014-11-03 11:37:52.948964 7f28f72fe700 5 -- op tracker -- seq: 3060, time: 2014-11-03 11:37:52.948810, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, version: 105946'11622386, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4671> 2014-11-03 11:37:52.948992 7f28f72fe700 5 -- op tracker -- seq: 3060, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, version: 105946'11622386, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4670> 2014-11-03 11:37:52.948988 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 170 ==== osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3088483816 0 3511520646) 0x191b7400 con 0x22fefdc0 -4669> 2014-11-03 11:37:52.949034 7f28f36c2700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.948732, event: header_read, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4668> 2014-11-03 11:37:52.949072 7f28f36c2700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.948735, event: throttled, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4667> 2014-11-03 11:37:52.949086 7f28f36c2700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.948965, event: all_read, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4666> 2014-11-03 11:37:52.949086 7f28fe818700 5 -- op tracker -- seq: 3060, time: 2014-11-03 11:37:52.949085, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, version: 105946'11622386, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4665> 2014-11-03 11:37:52.949155 7f28f36c2700 5 -- op tracker -- seq: 3061, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4664> 2014-11-03 11:37:52.949249 7f290101d700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.949248, event: reached_pg, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4663> 2014-11-03 11:37:52.949281 7f290101d700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.949281, event: started, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4662> 2014-11-03 11:37:52.949358 7f28fe818700 5 -- op tracker -- seq: 3060, time: 2014-11-03 11:37:52.949357, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6, version: 105946'11622386, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e67a58e/rbd_data.3b14122ae8944a.00000000000008ae/head//6@105946'11622386, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4661> 2014-11-03 11:37:52.949378 7f290101d700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.949377, event: started, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4660> 2014-11-03 11:37:52.949401 7f290101d700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.949401, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4659> 2014-11-03 11:37:52.950415 7f2914d0c700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:52.950415, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4658> 2014-11-03 11:37:52.979433 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x239252c0 sd=118 :6802 s=0 pgs=0 cs=0 l=0 c=0x2ce106e0).accept sd=118 192.168.5.2:42921/0 -4657> 2014-11-03 11:37:52.980465 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450871 192.168.5.2:0/1000467 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x24139a00 con 0x2ce106e0 -4656> 2014-11-03 11:37:52.981050 7f28fb812700 0 osd.21 105950 do_command r=0 -4655> 2014-11-03 11:37:52.981079 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000467 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xa5f6c40 con 0x2ce106e0 -4654> 2014-11-03 11:37:52.984897 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.71 192.168.5.6:0/14378 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.983670) v2 ==== 47+0+0 (3707475180 0 0) 0x1063fa00 con 0x234ed2c0 -4653> 2014-11-03 11:37:52.984908 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.71 192.168.5.6:0/14378 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:52.983670) v2 ==== 47+0+0 (3707475180 0 0) 0x245ff000 con 0x233352c0 -4652> 2014-11-03 11:37:52.984974 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.983670) v2 -- ?+0 0x23307200 con 0x234ed2c0 -4651> 2014-11-03 11:37:52.985058 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:52.983670) v2 -- ?+0 0x2c390800 con 0x233352c0 -4650> 2014-11-03 11:37:52.990062 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450871 192.168.5.2:0/1000467 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2372bc00 con 0x2ce106e0 -4649> 2014-11-03 11:37:52.990792 7f28fb812700 0 osd.21 105950 do_command r=0 -4648> 2014-11-03 11:37:52.990807 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000467 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x2b23e000 con 0x2ce106e0 -4647> 2014-11-03 11:37:52.999173 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000467 pipe(0x239252c0 sd=118 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce106e0).reader couldn't read tag, (0) Success -4646> 2014-11-03 11:37:52.999221 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000467 pipe(0x239252c0 sd=118 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce106e0).fault (0) Success -4645> 2014-11-03 11:37:52.999324 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2ce106e0 session 0x235df000 -4644> 2014-11-03 11:37:53.046148 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MOSDPGPushReply(6.91c 105950 [PushReplyOp(ff5591c/rbd_data.7b544f6e3a6632.000000000000d400/head//6)]) v2 -- ?+0 0x31a6200 con 0x23000b00 -4643> 2014-11-03 11:37:53.047244 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 48 ==== MRecoveryReserve RELEASE pgid: 6.91c, query_epoch: 105950 v2 ==== 26+0+0 (896902188 0 0) 0x31a6200 con 0x23000b00 -4642> 2014-11-03 11:37:53.047280 7f290b031700 5 -- op tracker -- seq: 3062, time: 2014-11-03 11:37:53.047092, event: header_read, op: MRecoveryReserve RELEASE pgid: 6.91c, query_epoch: 105950 -4641> 2014-11-03 11:37:53.047294 7f290b031700 5 -- op tracker -- seq: 3062, time: 2014-11-03 11:37:53.047094, event: throttled, op: MRecoveryReserve RELEASE pgid: 6.91c, query_epoch: 105950 -4640> 2014-11-03 11:37:53.047313 7f290b031700 5 -- op tracker -- seq: 3062, time: 2014-11-03 11:37:53.047174, event: all_read, op: MRecoveryReserve RELEASE pgid: 6.91c, query_epoch: 105950 -4639> 2014-11-03 11:37:53.047326 7f290b031700 5 -- op tracker -- seq: 3062, time: 2014-11-03 11:37:53.047274, event: dispatched, op: MRecoveryReserve RELEASE pgid: 6.91c, query_epoch: 105950 -4638> 2014-11-03 11:37:53.047339 7f290b031700 5 -- op tracker -- seq: 3062, time: 2014-11-03 11:37:53.047339, event: waiting_for_osdmap, op: MRecoveryReserve RELEASE pgid: 6.91c, query_epoch: 105950 -4637> 2014-11-03 11:37:53.047374 7f290b031700 5 -- op tracker -- seq: 3062, time: 2014-11-03 11:37:53.047374, event: done, op: MRecoveryReserve RELEASE pgid: 6.91c, query_epoch: 105950 -4636> 2014-11-03 11:37:53.047443 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.91c( v 105950'10711003 (105913'10707148,105950'10711003] local-les=105948 n=4406 ec=164 les/c 105948/105943 105947/105947/105254) [97,21,72] r=1 lpr=105947 pi=93257-105946/345 luod=0'0 crt=105946'10710996 active] exit Started/ReplicaActive/RepRecovering 1.813147 1 0.000121 -4635> 2014-11-03 11:37:53.047517 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.91c( v 105950'10711003 (105913'10707148,105950'10711003] local-les=105948 n=4406 ec=164 les/c 105948/105943 105947/105947/105254) [97,21,72] r=1 lpr=105947 pi=93257-105946/345 luod=0'0 crt=105946'10710996 active] enter Started/ReplicaActive/RepNotRecovering -4634> 2014-11-03 11:37:53.047577 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MRecoveryReserve GRANT pgid: 6.6ac, query_epoch: 105950 v2 -- ?+0 0x2b950200 con 0x2301f580 -4633> 2014-11-03 11:37:53.047614 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.6ac( v 105950'9806086 lc 105938'9805811 (105913'9802746,105950'9806086] local-les=105948 n=9072 ec=164 les/c 105948/105943 105947/105947/105366) [118,106,21] r=2 lpr=105947 pi=79477-105946/417 luod=0'0 crt=105946'9806084 active m=29] exit Started/ReplicaActive/RepWaitRecoveryReserved 26.402559 5 0.000076 -4632> 2014-11-03 11:37:53.047631 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 49 ==== pg_info(1 pgs e105950:6.91c) v4 ==== 1743+0+0 (2924278253 0 0) 0x2369f000 con 0x23000b00 -4631> 2014-11-03 11:37:53.047646 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.6ac( v 105950'9806086 lc 105938'9805811 (105913'9802746,105950'9806086] local-les=105948 n=9072 ec=164 les/c 105948/105943 105947/105947/105366) [118,106,21] r=2 lpr=105947 pi=79477-105946/417 luod=0'0 crt=105946'9806084 active m=29] enter Started/ReplicaActive/RepRecovering -4630> 2014-11-03 11:37:53.047677 7f290b031700 5 -- op tracker -- seq: 3063, time: 2014-11-03 11:37:53.047477, event: header_read, op: pg_info(1 pgs e105950:6.91c) -4629> 2014-11-03 11:37:53.047690 7f290b031700 5 -- op tracker -- seq: 3063, time: 2014-11-03 11:37:53.047479, event: throttled, op: pg_info(1 pgs e105950:6.91c) -4628> 2014-11-03 11:37:53.047696 7f290b031700 5 -- op tracker -- seq: 3063, time: 2014-11-03 11:37:53.047592, event: all_read, op: pg_info(1 pgs e105950:6.91c) -4627> 2014-11-03 11:37:53.047709 7f290b031700 5 -- op tracker -- seq: 3063, time: 2014-11-03 11:37:53.047673, event: dispatched, op: pg_info(1 pgs e105950:6.91c) -4626> 2014-11-03 11:37:53.047715 7f290b031700 5 -- op tracker -- seq: 3063, time: 2014-11-03 11:37:53.047715, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:6.91c) -4625> 2014-11-03 11:37:53.047736 7f290b031700 5 -- op tracker -- seq: 3063, time: 2014-11-03 11:37:53.047736, event: started, op: pg_info(1 pgs e105950:6.91c) -4624> 2014-11-03 11:37:53.047807 7f290b031700 5 -- op tracker -- seq: 3063, time: 2014-11-03 11:37:53.047807, event: done, op: pg_info(1 pgs e105950:6.91c) -4623> 2014-11-03 11:37:53.074001 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 71 ==== osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3782067461 0 3511520646) 0x2605a800 con 0x2308b2c0 -4622> 2014-11-03 11:37:53.074068 7f28eed79700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.073843, event: header_read, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4621> 2014-11-03 11:37:53.074086 7f28eed79700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.073845, event: throttled, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4620> 2014-11-03 11:37:53.074150 7f28eed79700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.073978, event: all_read, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4619> 2014-11-03 11:37:53.074240 7f28eed79700 5 -- op tracker -- seq: 3064, time: 0.000000, event: dispatched, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4618> 2014-11-03 11:37:53.074344 7f28ff81a700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.074343, event: reached_pg, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4617> 2014-11-03 11:37:53.074382 7f28ff81a700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.074382, event: started, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4616> 2014-11-03 11:37:53.074506 7f28ff81a700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.074506, event: started, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4615> 2014-11-03 11:37:53.074542 7f28ff81a700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.074541, event: commit_queued_for_journal_write, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4614> 2014-11-03 11:37:53.077541 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 72 ==== osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2282309700 0 3511520646) 0x295a2000 con 0x22fbc840 -4613> 2014-11-03 11:37:53.077579 7f28f68f4700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.077447, event: header_read, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4612> 2014-11-03 11:37:53.077599 7f28f68f4700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.077450, event: throttled, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4611> 2014-11-03 11:37:53.077621 7f28f68f4700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.077529, event: all_read, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4610> 2014-11-03 11:37:53.077629 7f28f68f4700 5 -- op tracker -- seq: 3065, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4609> 2014-11-03 11:37:53.077732 7f28fd816700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.077731, event: reached_pg, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4608> 2014-11-03 11:37:53.077788 7f28fd816700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.077787, event: started, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4607> 2014-11-03 11:37:53.077959 7f28fd816700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.077958, event: started, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4606> 2014-11-03 11:37:53.078001 7f28fd816700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.078001, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4605> 2014-11-03 11:37:53.105526 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.3 192.168.5.1:0/28138 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.104651) v2 ==== 47+0+0 (1134696932 0 0) 0x23354000 con 0x23ea46e0 -4604> 2014-11-03 11:37:53.105578 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/28138 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.104651) v2 -- ?+0 0x1063fa00 con 0x23ea46e0 -4603> 2014-11-03 11:37:53.105581 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.3 192.168.5.1:0/28138 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.104651) v2 ==== 47+0+0 (1134696932 0 0) 0x8abf400 con 0x2445c580 -4602> 2014-11-03 11:37:53.105635 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/28138 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.104651) v2 -- ?+0 0x245ff000 con 0x2445c580 -4601> 2014-11-03 11:37:53.109047 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.35 192.168.5.3:0/6455 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.108197) v2 ==== 47+0+0 (2588071664 0 0) 0x229e7e00 con 0x2484a6e0 -4600> 2014-11-03 11:37:53.109077 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.35 192.168.5.3:0/6455 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.108197) v2 ==== 47+0+0 (2588071664 0 0) 0x2be31400 con 0x24321160 -4599> 2014-11-03 11:37:53.109095 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.108197) v2 -- ?+0 0x8abf400 con 0x2484a6e0 -4598> 2014-11-03 11:37:53.109177 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.108197) v2 -- ?+0 0x23354000 con 0x24321160 -4597> 2014-11-03 11:37:53.114628 7f291450b700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:53.114628, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4596> 2014-11-03 11:37:53.114716 7f2911d06700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:53.114716, event: commit_sent, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4595> 2014-11-03 11:37:53.114754 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2d424b80 con 0x22fefdc0 -4594> 2014-11-03 11:37:53.114752 7f2914d0c700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.114751, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4593> 2014-11-03 11:37:53.114804 7f2914d0c700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.114804, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4592> 2014-11-03 11:37:53.115443 7f2912507700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:53.115443, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4591> 2014-11-03 11:37:53.115462 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6871/4005 -- osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 -- ?+10000 0x27072400 con 0x22fef2c0 -4590> 2014-11-03 11:37:53.115504 7f2912507700 5 -- op tracker -- seq: 3061, time: 2014-11-03 11:37:53.115504, event: done, op: osd_sub_op(client.8703713.0:32254435 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953151 snapset=0=[]:[] snapc=0=[]) -4589> 2014-11-03 11:37:53.116244 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x279fc340 sd=86 :6802 s=0 pgs=0 cs=0 l=0 c=0x2ce10dc0).accept sd=86 192.168.5.2:42925/0 -4588> 2014-11-03 11:37:53.116446 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x2be31800 con 0x22fe0c60 -4587> 2014-11-03 11:37:53.116539 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6829/25327 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x273fd600 con 0x22fef840 -4586> 2014-11-03 11:37:53.116973 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x273fdc00 con 0x22fe0c60 -4585> 2014-11-03 11:37:53.117079 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6829/25327 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x23510c00 con 0x22fef840 -4584> 2014-11-03 11:37:53.117259 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450880 192.168.5.2:0/1000600 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23510200 con 0x2ce10dc0 -4583> 2014-11-03 11:37:53.117459 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x2350a200 con 0x22fe0c60 -4582> 2014-11-03 11:37:53.117507 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6829/25327 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x1063fa00 con 0x22fef840 -4581> 2014-11-03 11:37:53.117776 7f28fb812700 0 osd.21 105950 do_command r=0 -4580> 2014-11-03 11:37:53.117792 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000600 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x4e2a1c0 con 0x2ce10dc0 -4579> 2014-11-03 11:37:53.117817 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x2796c600 con 0x22fe0c60 -4578> 2014-11-03 11:37:53.117882 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6829/25327 -- pg_info(1 pgs e105950:24.596) v4 -- ?+0 0x245ff000 con 0x22fef840 -4577> 2014-11-03 11:37:53.118374 7f28f3eca700 1 -- 192.168.5.2:6803/7606 <== osd.83 192.168.5.6:6871/4005 38 ==== osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1202+0+0 (3994805466 0 0) 0x27072400 con 0x22fef2c0 -4576> 2014-11-03 11:37:53.118417 7f28f3eca700 5 -- op tracker -- seq: 3066, time: 2014-11-03 11:37:53.118197, event: header_read, op: osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -4575> 2014-11-03 11:37:53.118449 7f28f3eca700 5 -- op tracker -- seq: 3066, time: 2014-11-03 11:37:53.118199, event: throttled, op: osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -4574> 2014-11-03 11:37:53.118464 7f28f3eca700 5 -- op tracker -- seq: 3066, time: 2014-11-03 11:37:53.118355, event: all_read, op: osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -4573> 2014-11-03 11:37:53.118473 7f28f3eca700 5 -- op tracker -- seq: 3066, time: 0.000000, event: dispatched, op: osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -4572> 2014-11-03 11:37:53.118540 7f290081c700 5 -- op tracker -- seq: 3066, time: 2014-11-03 11:37:53.118539, event: reached_pg, op: osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -4571> 2014-11-03 11:37:53.118602 7f290081c700 5 -- op tracker -- seq: 3066, time: 2014-11-03 11:37:53.118602, event: started, op: osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -4570> 2014-11-03 11:37:53.118627 7f290081c700 5 -- op tracker -- seq: 3066, time: 2014-11-03 11:37:53.118627, event: done, op: osd_sub_op(unknown.0.0:0 26.94 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -4569> 2014-11-03 11:37:53.118655 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.83 192.168.5.6:6871/4005 39 ==== pg_info(1 pgs e105950:26.94) v4 ==== 767+0+0 (2085365371 0 0) 0x2a7c4c00 con 0x22fef2c0 -4568> 2014-11-03 11:37:53.118708 7f290b031700 5 -- op tracker -- seq: 3067, time: 2014-11-03 11:37:53.118516, event: header_read, op: pg_info(1 pgs e105950:26.94) -4567> 2014-11-03 11:37:53.118745 7f290b031700 5 -- op tracker -- seq: 3067, time: 2014-11-03 11:37:53.118517, event: throttled, op: pg_info(1 pgs e105950:26.94) -4566> 2014-11-03 11:37:53.118767 7f290b031700 5 -- op tracker -- seq: 3067, time: 2014-11-03 11:37:53.118604, event: all_read, op: pg_info(1 pgs e105950:26.94) -4565> 2014-11-03 11:37:53.118789 7f290b031700 5 -- op tracker -- seq: 3067, time: 2014-11-03 11:37:53.118700, event: dispatched, op: pg_info(1 pgs e105950:26.94) -4564> 2014-11-03 11:37:53.118814 7f290b031700 5 -- op tracker -- seq: 3067, time: 2014-11-03 11:37:53.118814, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:26.94) -4563> 2014-11-03 11:37:53.118843 7f290b031700 5 -- op tracker -- seq: 3067, time: 2014-11-03 11:37:53.118843, event: started, op: pg_info(1 pgs e105950:26.94) -4562> 2014-11-03 11:37:53.118946 7f290b031700 5 -- op tracker -- seq: 3067, time: 2014-11-03 11:37:53.118946, event: done, op: pg_info(1 pgs e105950:26.94) -4561> 2014-11-03 11:37:53.122660 7f291450b700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.122660, event: journaled_completion_queued, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4560> 2014-11-03 11:37:53.122725 7f291450b700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.122725, event: journaled_completion_queued, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4559> 2014-11-03 11:37:53.122760 7f2911d06700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.122760, event: commit_sent, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4558> 2014-11-03 11:37:53.122799 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d423840 con 0x2308b2c0 -4557> 2014-11-03 11:37:53.122837 7f2911d06700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.122837, event: commit_sent, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4556> 2014-11-03 11:37:53.122905 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d424340 con 0x22fbc840 -4555> 2014-11-03 11:37:53.122978 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x23a92800 con 0x231abb00 -4554> 2014-11-03 11:37:53.123329 7f2912507700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.123328, event: sub_op_applied, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4553> 2014-11-03 11:37:53.123382 7f2912507700 5 -- op tracker -- seq: 3065, time: 2014-11-03 11:37:53.123381, event: done, op: osd_sub_op(client.7914905.0:3651997 6.57c d295957c/rbd_data.76480a6dd10759.000000000000032a/head//6 [] v 105950'15047358 snapset=0=[]:[] snapc=0=[]) -4552> 2014-11-03 11:37:53.123441 7f2912507700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.123441, event: sub_op_applied, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4551> 2014-11-03 11:37:53.123481 7f2912507700 5 -- op tracker -- seq: 3064, time: 2014-11-03 11:37:53.123481, event: done, op: osd_sub_op(client.9052442.0:1100490 6.456 216a2c56/rbd_data.7a2aec2ae8944a.00000000000002fe/head//6 [] v 105950'31870435 snapset=0=[]:[] snapc=0=[]) -4550> 2014-11-03 11:37:53.128762 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450880 192.168.5.2:0/1000600 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2350a800 con 0x2ce10dc0 -4549> 2014-11-03 11:37:53.129512 7f28fb812700 0 osd.21 105950 do_command r=0 -4548> 2014-11-03 11:37:53.129529 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000600 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x234168c0 con 0x2ce10dc0 -4547> 2014-11-03 11:37:53.137337 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000600 pipe(0x279fc340 sd=86 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10dc0).reader couldn't read tag, (0) Success -4546> 2014-11-03 11:37:53.137424 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000600 pipe(0x279fc340 sd=86 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10dc0).fault (0) Success -4545> 2014-11-03 11:37:53.137622 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2ce10dc0 session 0x235df000 -4544> 2014-11-03 11:37:53.139863 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.28 192.168.5.3:0/13976 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.138947) v2 ==== 47+0+0 (3444223749 0 0) 0x23510800 con 0x23b1cb00 -4543> 2014-11-03 11:37:53.139907 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.28 192.168.5.3:0/13976 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.138947) v2 ==== 47+0+0 (3444223749 0 0) 0x2579fc00 con 0x242c0c60 -4542> 2014-11-03 11:37:53.139931 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/13976 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.138947) v2 -- ?+0 0x229e7e00 con 0x23b1cb00 -4541> 2014-11-03 11:37:53.140032 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/13976 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.138947) v2 -- ?+0 0x2be31400 con 0x242c0c60 -4540> 2014-11-03 11:37:53.172219 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 32 ==== osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3253715850 0 3511520646) 0x25eb7000 con 0x22fe0840 -4539> 2014-11-03 11:37:53.172267 7f28f4cd8700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172104, event: header_read, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4538> 2014-11-03 11:37:53.172290 7f28f4cd8700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172107, event: throttled, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4537> 2014-11-03 11:37:53.172308 7f28f4cd8700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172205, event: all_read, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4536> 2014-11-03 11:37:53.172324 7f28f4cd8700 5 -- op tracker -- seq: 3068, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4535> 2014-11-03 11:37:53.172382 7f28fe017700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172382, event: reached_pg, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4534> 2014-11-03 11:37:53.172413 7f28fe017700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172413, event: started, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4533> 2014-11-03 11:37:53.172743 7f28fe017700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172742, event: started, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4532> 2014-11-03 11:37:53.172844 7f28fe017700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172844, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4531> 2014-11-03 11:37:53.172929 7f2914d0c700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.172929, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4530> 2014-11-03 11:37:53.179205 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 171 ==== osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2650571817 0 3511520646) 0x2bd4f800 con 0x22fefdc0 -4529> 2014-11-03 11:37:53.179240 7f28f36c2700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.179097, event: header_read, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4528> 2014-11-03 11:37:53.179253 7f28f36c2700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.179098, event: throttled, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4527> 2014-11-03 11:37:53.179261 7f28f36c2700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.179190, event: all_read, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4526> 2014-11-03 11:37:53.179269 7f28f36c2700 5 -- op tracker -- seq: 3069, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4525> 2014-11-03 11:37:53.179357 7f28fe818700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.179357, event: reached_pg, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4524> 2014-11-03 11:37:53.179391 7f28fe818700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.179391, event: started, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4523> 2014-11-03 11:37:53.179485 7f28fe818700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.179485, event: started, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4522> 2014-11-03 11:37:53.179528 7f28fe818700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.179528, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4521> 2014-11-03 11:37:53.217846 7f291450b700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.217846, event: journaled_completion_queued, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4520> 2014-11-03 11:37:53.217939 7f2911d06700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.217938, event: commit_sent, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4519> 2014-11-03 11:37:53.217957 7f2914d0c700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.217957, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4518> 2014-11-03 11:37:53.217983 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d424600 con 0x22fe0840 -4517> 2014-11-03 11:37:53.218369 7f2912507700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.218369, event: sub_op_applied, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4516> 2014-11-03 11:37:53.218420 7f2912507700 5 -- op tracker -- seq: 3068, time: 2014-11-03 11:37:53.218419, event: done, op: osd_sub_op(client.7868837.0:1672790 6.35a ded16b5a/rbd_data.3e5d6a2ae8944a.0000000000000e00/head//6 [] v 105950'14960082 snapset=0=[]:[] snapc=0=[]) -4515> 2014-11-03 11:37:53.219031 7f291450b700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.219030, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4514> 2014-11-03 11:37:53.219069 7f2911d06700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.219069, event: commit_sent, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4513> 2014-11-03 11:37:53.219104 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2d423580 con 0x22fefdc0 -4512> 2014-11-03 11:37:53.219405 7f2912507700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.219405, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4511> 2014-11-03 11:37:53.219445 7f2912507700 5 -- op tracker -- seq: 3069, time: 2014-11-03 11:37:53.219445, event: done, op: osd_sub_op(client.8703713.0:32254444 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953152 snapset=0=[]:[] snapc=0=[]) -4510> 2014-11-03 11:37:53.221096 7f2918d14700 5 osd.21 105950 tick -4509> 2014-11-03 11:37:53.228200 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.70 192.168.5.6:0/3622 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.226263) v2 ==== 47+0+0 (3968501870 0 0) 0x9459600 con 0x238b8c60 -4508> 2014-11-03 11:37:53.228210 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.70 192.168.5.6:0/3622 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.226263) v2 ==== 47+0+0 (3968501870 0 0) 0x27b41600 con 0x239f8b00 -4507> 2014-11-03 11:37:53.228270 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/3622 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.226263) v2 -- ?+0 0x2579fc00 con 0x238b8c60 -4506> 2014-11-03 11:37:53.228325 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/3622 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.226263) v2 -- ?+0 0x23510800 con 0x239f8b00 -4505> 2014-11-03 11:37:53.234020 7f28f57e3700 1 -- 192.168.5.2:6803/7606 <== osd.40 192.168.5.3:6868/25675 41 ==== osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1282095964 0 3511520646) 0x29055800 con 0x22fea580 -4504> 2014-11-03 11:37:53.234071 7f28f57e3700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.233878, event: header_read, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4503> 2014-11-03 11:37:53.234090 7f28f57e3700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.233881, event: throttled, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4502> 2014-11-03 11:37:53.234152 7f28f57e3700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.234002, event: all_read, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4501> 2014-11-03 11:37:53.234176 7f28f57e3700 5 -- op tracker -- seq: 3070, time: 0.000000, event: dispatched, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4500> 2014-11-03 11:37:53.234245 7f290201f700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.234245, event: reached_pg, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4499> 2014-11-03 11:37:53.234278 7f290201f700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.234278, event: started, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4498> 2014-11-03 11:37:53.234371 7f290201f700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.234371, event: started, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4497> 2014-11-03 11:37:53.234400 7f290201f700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.234400, event: commit_queued_for_journal_write, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4496> 2014-11-03 11:37:53.234484 7f2914d0c700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.234484, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4495> 2014-11-03 11:37:53.242571 7f291450b700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.242571, event: journaled_completion_queued, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4494> 2014-11-03 11:37:53.242626 7f2911d06700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.242626, event: commit_sent, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4493> 2014-11-03 11:37:53.242669 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6868/25675 -- osd_sub_op_reply(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d423dc0 con 0x22fea580 -4492> 2014-11-03 11:37:53.242926 7f2912507700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.242926, event: sub_op_applied, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4491> 2014-11-03 11:37:53.242987 7f2912507700 5 -- op tracker -- seq: 3070, time: 2014-11-03 11:37:53.242987, event: done, op: osd_sub_op(client.8689523.0:158113 6.5e1 b20b65e1/rbd_data.84951e2ae8944a.0000000000000084/head//6 [] v 105950'12182998 snapset=0=[]:[] snapc=0=[]) -4490> 2014-11-03 11:37:53.255886 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 33 ==== MOSDPGPush(6.58e 105950 [PushOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, version: 105946'11622388, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1255022100 0 0) 0x23a92800 con 0x231abb00 -4489> 2014-11-03 11:37:53.255939 7f28eb945700 5 -- op tracker -- seq: 3071, time: 2014-11-03 11:37:53.239853, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, version: 105946'11622388, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4488> 2014-11-03 11:37:53.255965 7f28eb945700 5 -- op tracker -- seq: 3071, time: 2014-11-03 11:37:53.239854, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, version: 105946'11622388, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4487> 2014-11-03 11:37:53.255987 7f28eb945700 5 -- op tracker -- seq: 3071, time: 2014-11-03 11:37:53.255870, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, version: 105946'11622388, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4486> 2014-11-03 11:37:53.256004 7f28eb945700 5 -- op tracker -- seq: 3071, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, version: 105946'11622388, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4485> 2014-11-03 11:37:53.256075 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 90 ==== osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2618352728 0 3511520646) 0x2de0dc00 con 0x22ff3b00 -4484> 2014-11-03 11:37:53.256105 7f28f0d99700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.255972, event: header_read, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4483> 2014-11-03 11:37:53.256095 7f290101d700 5 -- op tracker -- seq: 3071, time: 2014-11-03 11:37:53.256094, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, version: 105946'11622388, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4482> 2014-11-03 11:37:53.256123 7f28f0d99700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.255974, event: throttled, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4481> 2014-11-03 11:37:53.256161 7f28f0d99700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.256060, event: all_read, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4480> 2014-11-03 11:37:53.256182 7f28f0d99700 5 -- op tracker -- seq: 3072, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4479> 2014-11-03 11:37:53.256265 7f290001b700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.256265, event: reached_pg, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4478> 2014-11-03 11:37:53.256295 7f290001b700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.256295, event: started, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4477> 2014-11-03 11:37:53.256339 7f290101d700 5 -- op tracker -- seq: 3071, time: 2014-11-03 11:37:53.256339, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6, version: 105946'11622388, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fa3ad8e/rbd_data.52eb902ae8944a.0000000000002434/head//6@105946'11622388, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4476> 2014-11-03 11:37:53.256390 7f290001b700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.256390, event: started, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4475> 2014-11-03 11:37:53.256415 7f290001b700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.256415, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4474> 2014-11-03 11:37:53.257604 7f2914d0c700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.257604, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4473> 2014-11-03 11:37:53.269207 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x274172c0 sd=86 :6802 s=0 pgs=0 cs=0 l=0 c=0x2ce10c60).accept sd=86 192.168.5.2:42929/0 -4472> 2014-11-03 11:37:53.269793 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419934 192.168.5.2:0/1000794 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2a8d7800 con 0x2ce10c60 -4471> 2014-11-03 11:37:53.270307 7f28fb812700 0 osd.21 105950 do_command r=0 -4470> 2014-11-03 11:37:53.270329 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000794 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x23243380 con 0x2ce10c60 -4469> 2014-11-03 11:37:53.280045 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419934 192.168.5.2:0/1000794 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2a8d7000 con 0x2ce10c60 -4468> 2014-11-03 11:37:53.280948 7f28fb812700 0 osd.21 105950 do_command r=0 -4467> 2014-11-03 11:37:53.280967 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1000794 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x2b23e540 con 0x2ce10c60 -4466> 2014-11-03 11:37:53.281251 7f28f018d700 1 -- 192.168.5.2:6803/7606 <== osd.34 192.168.5.3:6833/2034 44 ==== osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2956147231 0 3511520646) 0x246fdc00 con 0x230a9c60 -4465> 2014-11-03 11:37:53.281287 7f28f018d700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.281044, event: header_read, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4464> 2014-11-03 11:37:53.281304 7f28f018d700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.281047, event: throttled, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4463> 2014-11-03 11:37:53.281313 7f28f018d700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.281237, event: all_read, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4462> 2014-11-03 11:37:53.281325 7f28f018d700 5 -- op tracker -- seq: 3073, time: 0.000000, event: dispatched, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4461> 2014-11-03 11:37:53.281444 7f28ff81a700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.281444, event: reached_pg, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4460> 2014-11-03 11:37:53.281516 7f28ff81a700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.281516, event: started, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4459> 2014-11-03 11:37:53.281613 7f28ff81a700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.281613, event: started, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4458> 2014-11-03 11:37:53.281646 7f28ff81a700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.281646, event: commit_queued_for_journal_write, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4457> 2014-11-03 11:37:53.286945 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000794 pipe(0x274172c0 sd=86 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10c60).reader couldn't read tag, (0) Success -4456> 2014-11-03 11:37:53.286998 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1000794 pipe(0x274172c0 sd=86 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10c60).fault (0) Success -4455> 2014-11-03 11:37:53.287120 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2ce10c60 session 0x235df000 -4454> 2014-11-03 11:37:53.296501 7f291450b700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.296500, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4453> 2014-11-03 11:37:53.296570 7f2911d06700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.296570, event: commit_sent, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4452> 2014-11-03 11:37:53.296620 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23de7000 con 0x22ff3b00 -4451> 2014-11-03 11:37:53.296614 7f2914d0c700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.296613, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4450> 2014-11-03 11:37:53.296995 7f291450b700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.296994, event: journaled_completion_queued, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4449> 2014-11-03 11:37:53.297031 7f2911d06700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.297031, event: commit_sent, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4448> 2014-11-03 11:37:53.297072 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6833/2034 -- osd_sub_op_reply(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23de8b80 con 0x230a9c60 -4447> 2014-11-03 11:37:53.297185 7f2912507700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.297185, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4446> 2014-11-03 11:37:53.297237 7f2912507700 5 -- op tracker -- seq: 3072, time: 2014-11-03 11:37:53.297237, event: done, op: osd_sub_op(client.12146205.0:439312 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222069 snapset=0=[]:[] snapc=0=[]) -4445> 2014-11-03 11:37:53.297341 7f2912507700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.297341, event: sub_op_applied, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4444> 2014-11-03 11:37:53.297385 7f2912507700 5 -- op tracker -- seq: 3073, time: 2014-11-03 11:37:53.297385, event: done, op: osd_sub_op(client.7871111.0:5388933 6.6fe 8c6bcefe/rbd_data.3a33ad2ae8944a.0000000000000142/head//6 [] v 105950'9124373 snapset=0=[]:[] snapc=0=[]) -4443> 2014-11-03 11:37:53.297728 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.8371799 192.168.1.98:0/1032082 7 ==== ping magic: 0 v1 ==== 0+0+0 (0 0 0) 0x2dcb08c0 con 0x23158c60 -4442> 2014-11-03 11:37:53.301788 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x259f3400 con 0x231abb00 -4441> 2014-11-03 11:37:53.322101 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 34 ==== MOSDPGPush(6.58e 105950 [PushOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, version: 105946'11622394, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195207+0+0 (2823366835 0 0) 0x259f3400 con 0x231abb00 -4440> 2014-11-03 11:37:53.322179 7f28eb945700 5 -- op tracker -- seq: 3074, time: 2014-11-03 11:37:53.306668, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, version: 105946'11622394, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4439> 2014-11-03 11:37:53.322211 7f28eb945700 5 -- op tracker -- seq: 3074, time: 2014-11-03 11:37:53.306670, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, version: 105946'11622394, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4438> 2014-11-03 11:37:53.322226 7f28eb945700 5 -- op tracker -- seq: 3074, time: 2014-11-03 11:37:53.322084, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, version: 105946'11622394, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4437> 2014-11-03 11:37:53.322244 7f28eb945700 5 -- op tracker -- seq: 3074, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, version: 105946'11622394, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4436> 2014-11-03 11:37:53.322323 7f28fe818700 5 -- op tracker -- seq: 3074, time: 2014-11-03 11:37:53.322323, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, version: 105946'11622394, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4435> 2014-11-03 11:37:53.322536 7f28fe818700 5 -- op tracker -- seq: 3074, time: 2014-11-03 11:37:53.322536, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6, version: 105946'11622394, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9cca9d8e/rbd_data.90c1e578fbd27.00000000000044fc/head//6@105946'11622394, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4434> 2014-11-03 11:37:53.343255 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 35 ==== osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2894544201 0 3511520646) 0x24867800 con 0x22ff3420 -4433> 2014-11-03 11:37:53.343323 7f28f0b97700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.343102, event: header_read, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4432> 2014-11-03 11:37:53.343388 7f28f0b97700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.343104, event: throttled, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4431> 2014-11-03 11:37:53.343413 7f28f0b97700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.343238, event: all_read, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4430> 2014-11-03 11:37:53.343443 7f28f0b97700 5 -- op tracker -- seq: 3075, time: 0.000000, event: dispatched, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4429> 2014-11-03 11:37:53.343520 7f290201f700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.343520, event: reached_pg, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4428> 2014-11-03 11:37:53.343552 7f290201f700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.343552, event: started, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4427> 2014-11-03 11:37:53.343682 7f290201f700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.343682, event: started, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4426> 2014-11-03 11:37:53.343731 7f290201f700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.343731, event: commit_queued_for_journal_write, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4425> 2014-11-03 11:37:53.354560 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 113 ==== osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3412597389 0 3511520646) 0x96eb800 con 0x22fb76e0 -4424> 2014-11-03 11:37:53.354619 7f28edd69700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.354351, event: header_read, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4423> 2014-11-03 11:37:53.354645 7f28edd69700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.354354, event: throttled, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4422> 2014-11-03 11:37:53.354671 7f28edd69700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.354542, event: all_read, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4421> 2014-11-03 11:37:53.354691 7f28edd69700 5 -- op tracker -- seq: 3076, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4420> 2014-11-03 11:37:53.354757 7f290101d700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.354757, event: reached_pg, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4419> 2014-11-03 11:37:53.354798 7f290101d700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.354798, event: started, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4418> 2014-11-03 11:37:53.354894 7f290101d700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.354894, event: started, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4417> 2014-11-03 11:37:53.354940 7f290101d700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.354940, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4416> 2014-11-03 11:37:53.356213 7f2914d0c700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.356212, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4415> 2014-11-03 11:37:53.356251 7f2914d0c700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.356251, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4414> 2014-11-03 11:37:53.356897 7f291450b700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.356896, event: journaled_completion_queued, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4413> 2014-11-03 11:37:53.356937 7f291450b700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.356937, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4412> 2014-11-03 11:37:53.356925 7f2911d06700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.356925, event: commit_sent, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4411> 2014-11-03 11:37:53.356960 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op_reply(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23de72c0 con 0x22ff3420 -4410> 2014-11-03 11:37:53.357005 7f2911d06700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.357004, event: commit_sent, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4409> 2014-11-03 11:37:53.357034 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0x23de7580 con 0x22fb76e0 -4408> 2014-11-03 11:37:53.357204 7f2912507700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.357204, event: sub_op_applied, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4407> 2014-11-03 11:37:53.357253 7f2912507700 5 -- op tracker -- seq: 3075, time: 2014-11-03 11:37:53.357253, event: done, op: osd_sub_op(client.7871147.0:1472932 6.550 7e22ad50/rbd_data.450cad2ae8944a.0000000000000fe2/head//6 [] v 105950'29502027 snapset=0=[]:[] snapc=0=[]) -4406> 2014-11-03 11:37:53.357506 7f2912507700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.357506, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4405> 2014-11-03 11:37:53.357546 7f2912507700 5 -- op tracker -- seq: 3076, time: 2014-11-03 11:37:53.357546, event: done, op: osd_sub_op(client.12404444.0:39602 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374239 snapset=0=[]:[] snapc=0=[]) -4404> 2014-11-03 11:37:53.359696 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 114 ==== osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1106308064 0 3511520646) 0x25ed5800 con 0x22fb76e0 -4403> 2014-11-03 11:37:53.359729 7f28edd69700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.359596, event: header_read, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4402> 2014-11-03 11:37:53.359743 7f28edd69700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.359598, event: throttled, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4401> 2014-11-03 11:37:53.359755 7f28edd69700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.359682, event: all_read, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4400> 2014-11-03 11:37:53.359763 7f28edd69700 5 -- op tracker -- seq: 3077, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4399> 2014-11-03 11:37:53.359836 7f28fe818700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.359836, event: reached_pg, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4398> 2014-11-03 11:37:53.359862 7f28fe818700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.359862, event: started, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4397> 2014-11-03 11:37:53.360035 7f28fe818700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.360035, event: started, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4396> 2014-11-03 11:37:53.360067 7f28fe818700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.360067, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4395> 2014-11-03 11:37:53.360158 7f2914d0c700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.360157, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4394> 2014-11-03 11:37:53.360536 7f291450b700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.360536, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4393> 2014-11-03 11:37:53.360579 7f2911d06700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.360579, event: commit_sent, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4392> 2014-11-03 11:37:53.360633 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0x2650eb80 con 0x22fb76e0 -4391> 2014-11-03 11:37:53.361016 7f2912507700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.361016, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4390> 2014-11-03 11:37:53.361054 7f2912507700 5 -- op tracker -- seq: 3077, time: 2014-11-03 11:37:53.361054, event: done, op: osd_sub_op(client.12404444.0:39603 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374240 snapset=0=[]:[] snapc=0=[]) -4389> 2014-11-03 11:37:53.361456 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MOSDPGPull(6.58e 105950 [PullOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x23a92400 con 0x231abb00 -4388> 2014-11-03 11:37:53.369390 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 33 ==== MOSDPGPush(6.6ac 105950 [PushOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6, version: 105938'9805812, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6@105938'9805812, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6, version: 105938'9805817, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6@105938'9805817, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (2653221126 0 0) 0x2b950200 con 0x2301f580 -4387> 2014-11-03 11:37:53.369506 7f28f2db9700 5 -- op tracker -- seq: 3078, time: 2014-11-03 11:37:53.346720, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6, version: 105938'9805812, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6@105938'9805812, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6, version: 105938'9805817, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6@105938'9805817, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4386> 2014-11-03 11:37:53.369577 7f28f2db9700 5 -- op tracker -- seq: 3078, time: 2014-11-03 11:37:53.346723, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6, version: 105938'9805812, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6@105938'9805812, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6, version: 105938'9805817, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6@105938'9805817, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4385> 2014-11-03 11:37:53.369610 7f28f2db9700 5 -- op tracker -- seq: 3078, time: 2014-11-03 11:37:53.369361, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6, version: 105938'9805812, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6@105938'9805812, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6, version: 105938'9805817, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6@105938'9805817, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4384> 2014-11-03 11:37:53.369687 7f28f2db9700 5 -- op tracker -- seq: 3078, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6, version: 105938'9805812, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6@105938'9805812, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6, version: 105938'9805817, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6@105938'9805817, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4383> 2014-11-03 11:37:53.369781 7f290081c700 5 -- op tracker -- seq: 3078, time: 2014-11-03 11:37:53.369780, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6, version: 105938'9805812, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6@105938'9805812, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6, version: 105938'9805817, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6@105938'9805817, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4382> 2014-11-03 11:37:53.370028 7f290081c700 5 -- op tracker -- seq: 3078, time: 2014-11-03 11:37:53.370027, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6, version: 105938'9805812, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6@105938'9805812, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6, version: 105938'9805817, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6@105938'9805817, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4381> 2014-11-03 11:37:53.372038 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.105 192.168.5.8:0/6777 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.371372) v2 ==== 47+0+0 (2779843957 0 0) 0x273fde00 con 0x31a92c0 -4380> 2014-11-03 11:37:53.372083 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.105 192.168.5.8:0/6777 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.371372) v2 ==== 47+0+0 (2779843957 0 0) 0x2a2e0800 con 0x2389f2c0 -4379> 2014-11-03 11:37:53.372094 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/6777 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.371372) v2 -- ?+0 0x27b41600 con 0x31a92c0 -4378> 2014-11-03 11:37:53.372201 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/6777 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.371372) v2 -- ?+0 0x9459600 con 0x2389f2c0 -4377> 2014-11-03 11:37:53.385203 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 34 ==== MOSDPGPush(6.6ac 105950 [PushOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6, version: 105938'9805821, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6@105938'9805821, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6, version: 105938'9805839, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6@105938'9805839, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (3729975762 0 0) 0x2a77de00 con 0x2301f580 -4376> 2014-11-03 11:37:53.385278 7f28f2db9700 5 -- op tracker -- seq: 3079, time: 2014-11-03 11:37:53.369780, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6, version: 105938'9805821, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6@105938'9805821, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6, version: 105938'9805839, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6@105938'9805839, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4375> 2014-11-03 11:37:53.385329 7f28f2db9700 5 -- op tracker -- seq: 3079, time: 2014-11-03 11:37:53.369781, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6, version: 105938'9805821, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6@105938'9805821, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6, version: 105938'9805839, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6@105938'9805839, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4374> 2014-11-03 11:37:53.385360 7f28f2db9700 5 -- op tracker -- seq: 3079, time: 2014-11-03 11:37:53.385166, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6, version: 105938'9805821, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6@105938'9805821, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6, version: 105938'9805839, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6@105938'9805839, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4373> 2014-11-03 11:37:53.385390 7f28f2db9700 5 -- op tracker -- seq: 3079, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6, version: 105938'9805821, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6@105938'9805821, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6, version: 105938'9805839, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6@105938'9805839, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4372> 2014-11-03 11:37:53.385538 7f28fe017700 5 -- op tracker -- seq: 3079, time: 2014-11-03 11:37:53.385537, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6, version: 105938'9805821, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6@105938'9805821, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6, version: 105938'9805839, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6@105938'9805839, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4371> 2014-11-03 11:37:53.385932 7f28fe017700 5 -- op tracker -- seq: 3079, time: 2014-11-03 11:37:53.385932, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6, version: 105938'9805821, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6@105938'9805821, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6, version: 105938'9805839, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6@105938'9805839, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4370> 2014-11-03 11:37:53.391463 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 35 ==== MOSDPGPush(6.6ac 105950 [PushOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6, version: 105938'9805843, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6@105938'9805843, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1656352080 0 0) 0x2340c800 con 0x2301f580 -4369> 2014-11-03 11:37:53.391508 7f28f2db9700 5 -- op tracker -- seq: 3080, time: 2014-11-03 11:37:53.385494, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6, version: 105938'9805843, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6@105938'9805843, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4368> 2014-11-03 11:37:53.391530 7f28f2db9700 5 -- op tracker -- seq: 3080, time: 2014-11-03 11:37:53.385496, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6, version: 105938'9805843, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6@105938'9805843, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4367> 2014-11-03 11:37:53.391547 7f28f2db9700 5 -- op tracker -- seq: 3080, time: 2014-11-03 11:37:53.391445, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6, version: 105938'9805843, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6@105938'9805843, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4366> 2014-11-03 11:37:53.391563 7f28f2db9700 5 -- op tracker -- seq: 3080, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6, version: 105938'9805843, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6@105938'9805843, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4365> 2014-11-03 11:37:53.391684 7f290081c700 5 -- op tracker -- seq: 3080, time: 2014-11-03 11:37:53.391683, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6, version: 105938'9805843, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6@105938'9805843, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4364> 2014-11-03 11:37:53.391874 7f290081c700 5 -- op tracker -- seq: 3080, time: 2014-11-03 11:37:53.391874, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6, version: 105938'9805843, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6@105938'9805843, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4363> 2014-11-03 11:37:53.413638 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 115 ==== osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (4243566263 0 3511520646) 0x29e1e800 con 0x22fb76e0 -4362> 2014-11-03 11:37:53.413675 7f28edd69700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.413525, event: header_read, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4361> 2014-11-03 11:37:53.413692 7f28edd69700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.413527, event: throttled, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4360> 2014-11-03 11:37:53.413704 7f28edd69700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.413621, event: all_read, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4359> 2014-11-03 11:37:53.413716 7f28edd69700 5 -- op tracker -- seq: 3081, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4358> 2014-11-03 11:37:53.413810 7f290101d700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.413810, event: reached_pg, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4357> 2014-11-03 11:37:53.413845 7f290101d700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.413845, event: started, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4356> 2014-11-03 11:37:53.413937 7f290101d700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.413937, event: started, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4355> 2014-11-03 11:37:53.413968 7f290101d700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.413968, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4354> 2014-11-03 11:37:53.431129 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x315242c0 sd=537 :6802 s=0 pgs=0 cs=0 l=0 c=0x2ce10b00).accept sd=537 192.168.5.2:42934/0 -4353> 2014-11-03 11:37:53.432112 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450889 192.168.5.2:0/1001052 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0xa6b5000 con 0x2ce10b00 -4352> 2014-11-03 11:37:53.432689 7f28fb812700 0 osd.21 105950 do_command r=0 -4351> 2014-11-03 11:37:53.432721 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1001052 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x4e2a000 con 0x2ce10b00 -4350> 2014-11-03 11:37:53.437507 7f28eb945700 1 -- 192.168.5.2:6803/7606 <== osd.43 192.168.5.4:6812/19461 35 ==== MOSDPGPush(6.58e 105950 [PushOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, version: 105946'11622395, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (923198078 0 0) 0x23a92400 con 0x231abb00 -4349> 2014-11-03 11:37:53.437552 7f28eb945700 5 -- op tracker -- seq: 3082, time: 2014-11-03 11:37:53.422631, event: header_read, op: MOSDPGPush(6.58e 105950 [PushOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, version: 105946'11622395, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4348> 2014-11-03 11:37:53.437572 7f28eb945700 5 -- op tracker -- seq: 3082, time: 2014-11-03 11:37:53.422632, event: throttled, op: MOSDPGPush(6.58e 105950 [PushOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, version: 105946'11622395, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4347> 2014-11-03 11:37:53.437585 7f28eb945700 5 -- op tracker -- seq: 3082, time: 2014-11-03 11:37:53.437490, event: all_read, op: MOSDPGPush(6.58e 105950 [PushOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, version: 105946'11622395, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4346> 2014-11-03 11:37:53.437611 7f28eb945700 5 -- op tracker -- seq: 3082, time: 0.000000, event: dispatched, op: MOSDPGPush(6.58e 105950 [PushOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, version: 105946'11622395, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4345> 2014-11-03 11:37:53.437707 7f28fe818700 5 -- op tracker -- seq: 3082, time: 2014-11-03 11:37:53.437706, event: reached_pg, op: MOSDPGPush(6.58e 105950 [PushOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, version: 105946'11622395, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4344> 2014-11-03 11:37:53.437896 7f28fe818700 5 -- op tracker -- seq: 3082, time: 2014-11-03 11:37:53.437895, event: done, op: MOSDPGPush(6.58e 105950 [PushOp(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6, version: 105946'11622395, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7927058e/rbd_data.7675012ae8944a.0000000000000b87/head//6@105946'11622395, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4343> 2014-11-03 11:37:53.444553 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450889 192.168.5.2:0/1001052 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2a7c4400 con 0x2ce10b00 -4342> 2014-11-03 11:37:53.445831 7f28fb812700 0 osd.21 105950 do_command r=0 -4341> 2014-11-03 11:37:53.445860 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1001052 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x6ef0380 con 0x2ce10b00 -4340> 2014-11-03 11:37:53.451156 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1001052 pipe(0x315242c0 sd=537 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10b00).reader couldn't read tag, (0) Success -4339> 2014-11-03 11:37:53.451215 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1001052 pipe(0x315242c0 sd=537 :6802 s=2 pgs=2 cs=1 l=1 c=0x2ce10b00).fault (0) Success -4338> 2014-11-03 11:37:53.451351 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2ce10b00 session 0x235df000 -4337> 2014-11-03 11:37:53.489841 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.71 192.168.5.6:0/14378 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.488796) v2 ==== 47+0+0 (584751339 0 0) 0x1063fc00 con 0x234ed2c0 -4336> 2014-11-03 11:37:53.489893 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.488796) v2 -- ?+0 0x2a2e0800 con 0x234ed2c0 -4335> 2014-11-03 11:37:53.490403 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.71 192.168.5.6:0/14378 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.488796) v2 ==== 47+0+0 (584751339 0 0) 0x2369f600 con 0x233352c0 -4334> 2014-11-03 11:37:53.490450 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.488796) v2 -- ?+0 0x273fde00 con 0x233352c0 -4333> 2014-11-03 11:37:53.490540 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 36 ==== osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2561972184 0 3511520646) 0x2fe0cc00 con 0x22ff3420 -4332> 2014-11-03 11:37:53.490577 7f28f0b97700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.490427, event: header_read, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4331> 2014-11-03 11:37:53.490597 7f28f0b97700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.490428, event: throttled, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4330> 2014-11-03 11:37:53.490612 7f28f0b97700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.490524, event: all_read, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4329> 2014-11-03 11:37:53.490621 7f28f0b97700 5 -- op tracker -- seq: 3083, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4328> 2014-11-03 11:37:53.490682 7f28ff81a700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.490682, event: reached_pg, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4327> 2014-11-03 11:37:53.490713 7f28ff81a700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.490712, event: started, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4326> 2014-11-03 11:37:53.490805 7f28ff81a700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.490805, event: started, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4325> 2014-11-03 11:37:53.490837 7f28ff81a700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.490837, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4324> 2014-11-03 11:37:53.491841 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 37 ==== osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (4083935277 0 3511520646) 0x23eb7400 con 0x22ff3420 -4323> 2014-11-03 11:37:53.491883 7f28f0b97700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.491713, event: header_read, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4322> 2014-11-03 11:37:53.491910 7f28f0b97700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.491715, event: throttled, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4321> 2014-11-03 11:37:53.491936 7f28f0b97700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.491828, event: all_read, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4320> 2014-11-03 11:37:53.491955 7f28f0b97700 5 -- op tracker -- seq: 3084, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4319> 2014-11-03 11:37:53.492010 7f290201f700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.492010, event: reached_pg, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4318> 2014-11-03 11:37:53.492039 7f290201f700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.492039, event: started, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4317> 2014-11-03 11:37:53.492124 7f290201f700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.492124, event: started, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4316> 2014-11-03 11:37:53.492158 7f290201f700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.492158, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4315> 2014-11-03 11:37:53.503785 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.90 192.168.5.7:0/27918 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.502250) v2 ==== 47+0+0 (1957353504 0 0) 0x23414c00 con 0x2445c6e0 -4314> 2014-11-03 11:37:53.503826 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/27918 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.502250) v2 -- ?+0 0x2369f600 con 0x2445c6e0 -4313> 2014-11-03 11:37:53.503947 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.90 192.168.5.7:0/27918 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.502250) v2 ==== 47+0+0 (1957353504 0 0) 0x234d5600 con 0x23ea4840 -4312> 2014-11-03 11:37:53.504001 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/27918 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.502250) v2 -- ?+0 0x1063fc00 con 0x23ea4840 -4311> 2014-11-03 11:37:53.513022 7f2914d0c700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.513022, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4310> 2014-11-03 11:37:53.513781 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.60 192.168.5.5:0/5983 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.512130) v2 ==== 47+0+0 (2044044833 0 0) 0x1106c400 con 0x23ea4b00 -4309> 2014-11-03 11:37:53.513831 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.512130) v2 -- ?+0 0x234d5600 con 0x23ea4b00 -4308> 2014-11-03 11:37:53.513849 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.60 192.168.5.5:0/5983 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.512130) v2 ==== 47+0+0 (2044044833 0 0) 0x23ba6800 con 0x2445c9a0 -4307> 2014-11-03 11:37:53.513884 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.512130) v2 -- ?+0 0x23414c00 con 0x2445c9a0 -4306> 2014-11-03 11:37:53.514407 7f2914d0c700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.514407, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4305> 2014-11-03 11:37:53.514455 7f2914d0c700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.514455, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4304> 2014-11-03 11:37:53.534918 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 99 ==== osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (794922891 0 3511520646) 0x2ce3bc00 con 0x2308b840 -4303> 2014-11-03 11:37:53.534955 7f28ef985700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.534710, event: header_read, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4302> 2014-11-03 11:37:53.534970 7f28ef985700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.534713, event: throttled, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4301> 2014-11-03 11:37:53.534979 7f28ef985700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.534902, event: all_read, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4300> 2014-11-03 11:37:53.535003 7f28ef985700 5 -- op tracker -- seq: 3085, time: 0.000000, event: dispatched, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4299> 2014-11-03 11:37:53.535093 7f28fe017700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.535093, event: reached_pg, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4298> 2014-11-03 11:37:53.535129 7f28fe017700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.535129, event: started, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4297> 2014-11-03 11:37:53.535233 7f28fe017700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.535233, event: started, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4296> 2014-11-03 11:37:53.535264 7f28fe017700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.535264, event: commit_queued_for_journal_write, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4295> 2014-11-03 11:37:53.570803 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31525340 sd=232 :6802 s=0 pgs=0 cs=0 l=0 c=0x154dd580).accept sd=232 192.168.5.2:42939/0 -4294> 2014-11-03 11:37:53.571520 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450895 192.168.5.2:0/1002265 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23ba6600 con 0x154dd580 -4293> 2014-11-03 11:37:53.572148 7f28fb812700 0 osd.21 105950 do_command r=0 -4292> 2014-11-03 11:37:53.572182 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1002265 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x4e2a700 con 0x154dd580 -4291> 2014-11-03 11:37:53.582387 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 100 ==== osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3849956030 0 3511520646) 0x27eda400 con 0x2308b840 -4290> 2014-11-03 11:37:53.582417 7f28ef985700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.582285, event: header_read, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4289> 2014-11-03 11:37:53.582431 7f28ef985700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.582287, event: throttled, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4288> 2014-11-03 11:37:53.582440 7f28ef985700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.582375, event: all_read, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4287> 2014-11-03 11:37:53.582448 7f28ef985700 5 -- op tracker -- seq: 3086, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4286> 2014-11-03 11:37:53.582555 7f28ff81a700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.582555, event: reached_pg, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4285> 2014-11-03 11:37:53.582605 7f28ff81a700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.582605, event: started, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4284> 2014-11-03 11:37:53.582762 7f28ff81a700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.582762, event: started, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4283> 2014-11-03 11:37:53.582808 7f28ff81a700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.582808, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4282> 2014-11-03 11:37:53.584056 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450895 192.168.5.2:0/1002265 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23ba6200 con 0x154dd580 -4281> 2014-11-03 11:37:53.585508 7f28fb812700 0 osd.21 105950 do_command r=0 -4280> 2014-11-03 11:37:53.585536 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1002265 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x6ef0380 con 0x154dd580 -4279> 2014-11-03 11:37:53.588658 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 172 ==== osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2591057530 0 3511520646) 0x2cff4800 con 0x22fefdc0 -4278> 2014-11-03 11:37:53.588689 7f28f36c2700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.588477, event: header_read, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4277> 2014-11-03 11:37:53.588704 7f28f36c2700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.588480, event: throttled, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4276> 2014-11-03 11:37:53.588713 7f28f36c2700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.588644, event: all_read, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4275> 2014-11-03 11:37:53.588721 7f28f36c2700 5 -- op tracker -- seq: 3087, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4274> 2014-11-03 11:37:53.588776 7f290101d700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.588776, event: reached_pg, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4273> 2014-11-03 11:37:53.588807 7f290101d700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.588807, event: started, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4272> 2014-11-03 11:37:53.588889 7f290101d700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.588889, event: started, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4271> 2014-11-03 11:37:53.588920 7f290101d700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.588920, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4270> 2014-11-03 11:37:53.589845 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 43 ==== osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4925 (1467095393 0 261295794) 0x27000000 con 0x231ab580 -4269> 2014-11-03 11:37:53.589875 7f28ea430700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.589700, event: header_read, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4268> 2014-11-03 11:37:53.589892 7f28ea430700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.589704, event: throttled, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4267> 2014-11-03 11:37:53.589901 7f28ea430700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.589831, event: all_read, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4266> 2014-11-03 11:37:53.589909 7f28ea430700 5 -- op tracker -- seq: 3088, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4265> 2014-11-03 11:37:53.589977 7f290081c700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.589976, event: reached_pg, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4264> 2014-11-03 11:37:53.589995 7f290081c700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.589995, event: started, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4263> 2014-11-03 11:37:53.590081 7f290081c700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.590081, event: started, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4262> 2014-11-03 11:37:53.590107 7f290081c700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.590107, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4261> 2014-11-03 11:37:53.591528 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1002265 pipe(0x31525340 sd=232 :6802 s=2 pgs=2 cs=1 l=1 c=0x154dd580).reader couldn't read tag, (0) Success -4260> 2014-11-03 11:37:53.591570 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1002265 pipe(0x31525340 sd=232 :6802 s=2 pgs=2 cs=1 l=1 c=0x154dd580).fault (0) Success -4259> 2014-11-03 11:37:53.591683 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x154dd580 session 0x235df000 -4258> 2014-11-03 11:37:53.598956 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.52 192.168.5.4:0/3307 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.597924) v2 ==== 47+0+0 (108171078 0 0) 0x2512a400 con 0x242c0000 -4257> 2014-11-03 11:37:53.599029 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.52 192.168.5.4:0/3307 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.597924) v2 ==== 47+0+0 (108171078 0 0) 0x2369d400 con 0x2469e9a0 -4256> 2014-11-03 11:37:53.599062 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/3307 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.597924) v2 -- ?+0 0x23ba6800 con 0x242c0000 -4255> 2014-11-03 11:37:53.599348 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/3307 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.597924) v2 -- ?+0 0x1106c400 con 0x2469e9a0 -4254> 2014-11-03 11:37:53.613550 7f28f018d700 1 -- 192.168.5.2:6803/7606 <== osd.34 192.168.5.3:6833/2034 45 ==== osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2570891866 0 3511520646) 0x27eae400 con 0x230a9c60 -4253> 2014-11-03 11:37:53.613599 7f28f018d700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.613406, event: header_read, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4252> 2014-11-03 11:37:53.613615 7f28f018d700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.613409, event: throttled, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4251> 2014-11-03 11:37:53.613624 7f28f018d700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.613537, event: all_read, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4250> 2014-11-03 11:37:53.613653 7f28f018d700 5 -- op tracker -- seq: 3089, time: 0.000000, event: dispatched, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4249> 2014-11-03 11:37:53.613772 7f290201f700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.613772, event: reached_pg, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4248> 2014-11-03 11:37:53.613836 7f290201f700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.613836, event: started, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4247> 2014-11-03 11:37:53.613983 7f290201f700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.613983, event: started, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4246> 2014-11-03 11:37:53.614033 7f290201f700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.614033, event: commit_queued_for_journal_write, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4245> 2014-11-03 11:37:53.630958 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(4bc4eac/rbd_data.7a2aec2ae8944a.0000000000000018/head//6),PushReplyOp(b26e36ac/rbd_data.7ab7cd2ae8944a.00000000000001fc/head//6)]) v2 -- ?+0 0x2310a800 con 0x2301f580 -4244> 2014-11-03 11:37:53.636949 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.94 192.168.5.7:0/10578 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.634840) v2 ==== 47+0+0 (987624991 0 0) 0x8664e00 con 0x2327a6e0 -4243> 2014-11-03 11:37:53.637007 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/10578 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.634840) v2 -- ?+0 0x2369d400 con 0x2327a6e0 -4242> 2014-11-03 11:37:53.637341 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.94 192.168.5.7:0/10578 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.634840) v2 ==== 47+0+0 (987624991 0 0) 0x9459000 con 0x2327a420 -4241> 2014-11-03 11:37:53.637386 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/10578 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.634840) v2 -- ?+0 0x2512a400 con 0x2327a420 -4240> 2014-11-03 11:37:53.654568 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 44 ==== osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2178146812 0 3511520646) 0x2d47d400 con 0x231ab580 -4239> 2014-11-03 11:37:53.654618 7f28ea430700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.654443, event: header_read, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4238> 2014-11-03 11:37:53.654644 7f28ea430700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.654446, event: throttled, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4237> 2014-11-03 11:37:53.654661 7f28ea430700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.654558, event: all_read, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4236> 2014-11-03 11:37:53.654683 7f28ea430700 5 -- op tracker -- seq: 3090, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4235> 2014-11-03 11:37:53.654760 7f28ff019700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.654759, event: reached_pg, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4234> 2014-11-03 11:37:53.654800 7f28ff019700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.654800, event: started, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4233> 2014-11-03 11:37:53.654946 7f28ff019700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.654945, event: started, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4232> 2014-11-03 11:37:53.654985 7f28ff019700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.654984, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4231> 2014-11-03 11:37:53.662862 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.47 192.168.5.4:0/12077 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.661117) v2 ==== 47+0+0 (2494833684 0 0) 0x24800e00 con 0x242926e0 -4230> 2014-11-03 11:37:53.662920 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.661117) v2 -- ?+0 0x9459000 con 0x242926e0 -4229> 2014-11-03 11:37:53.662962 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.47 192.168.5.4:0/12077 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.661117) v2 ==== 47+0+0 (2494833684 0 0) 0x2ba4dc00 con 0x243216e0 -4228> 2014-11-03 11:37:53.663019 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.661117) v2 -- ?+0 0x8664e00 con 0x243216e0 -4227> 2014-11-03 11:37:53.677194 7f291450b700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.677193, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4226> 2014-11-03 11:37:53.677241 7f2911d06700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.677241, event: commit_sent, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4225> 2014-11-03 11:37:53.677250 7f291450b700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.677250, event: journaled_completion_queued, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4224> 2014-11-03 11:37:53.677281 7f291450b700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.677281, event: journaled_completion_queued, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4223> 2014-11-03 11:37:53.677280 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0x2650d000 con 0x22fb76e0 -4222> 2014-11-03 11:37:53.677366 7f2911d06700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.677365, event: commit_sent, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4221> 2014-11-03 11:37:53.677417 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op_reply(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650e8c0 con 0x22ff3420 -4220> 2014-11-03 11:37:53.677416 7f2914d0c700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.677416, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4219> 2014-11-03 11:37:53.677444 7f2911d06700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.677444, event: commit_sent, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4218> 2014-11-03 11:37:53.677465 7f2914d0c700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.677465, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4217> 2014-11-03 11:37:53.677503 7f2914d0c700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.677503, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4216> 2014-11-03 11:37:53.677537 7f2914d0c700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.677536, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4215> 2014-11-03 11:37:53.677553 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op_reply(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269afb00 con 0x22ff3420 -4214> 2014-11-03 11:37:53.677674 7f2914d0c700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.677673, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4213> 2014-11-03 11:37:53.677704 7f2914d0c700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.677704, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4212> 2014-11-03 11:37:53.688347 7f291450b700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.688347, event: journaled_completion_queued, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4211> 2014-11-03 11:37:53.688382 7f2911d06700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.688382, event: commit_sent, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4210> 2014-11-03 11:37:53.688417 7f291450b700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.688417, event: journaled_completion_queued, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4209> 2014-11-03 11:37:53.688430 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269af2c0 con 0x2308b840 -4208> 2014-11-03 11:37:53.688442 7f291450b700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.688442, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4207> 2014-11-03 11:37:53.688490 7f291450b700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.688490, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4206> 2014-11-03 11:37:53.688491 7f2911d06700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.688491, event: commit_sent, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4205> 2014-11-03 11:37:53.688521 7f291450b700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.688521, event: journaled_completion_queued, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4204> 2014-11-03 11:37:53.688545 7f291450b700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.688544, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4203> 2014-11-03 11:37:53.688612 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7000 con 0x2308b840 -4202> 2014-11-03 11:37:53.688653 7f2911d06700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.688653, event: commit_sent, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4201> 2014-11-03 11:37:53.688687 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7840 con 0x22fefdc0 -4200> 2014-11-03 11:37:53.688728 7f2911d06700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.688728, event: commit_sent, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4199> 2014-11-03 11:37:53.688770 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2e4b88c0 con 0x231ab580 -4198> 2014-11-03 11:37:53.688809 7f2911d06700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.688809, event: commit_sent, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4197> 2014-11-03 11:37:53.688855 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6833/2034 -- osd_sub_op_reply(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263aedc0 con 0x230a9c60 -4196> 2014-11-03 11:37:53.688892 7f2911d06700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.688892, event: commit_sent, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4195> 2014-11-03 11:37:53.688932 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263af8c0 con 0x231ab580 -4194> 2014-11-03 11:37:53.778973 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(eb93eeac/rbd_data.3e5d6a2ae8944a.0000000000001225/head//6),PushReplyOp(1068e6ac/rbd_data.3ea57474ca1fcf.0000000000001044/head//6)]) v2 -- ?+0 0xa6b5400 con 0x2301f580 -4193> 2014-11-03 11:37:53.796309 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 98 ==== osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (716555010 0 3511520646) 0x2de0e800 con 0x22fbc420 -4192> 2014-11-03 11:37:53.796345 7f28f72fe700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796188, event: header_read, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4191> 2014-11-03 11:37:53.796366 7f28f72fe700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796190, event: throttled, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4190> 2014-11-03 11:37:53.796379 7f28f72fe700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796293, event: all_read, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4189> 2014-11-03 11:37:53.796391 7f28f72fe700 5 -- op tracker -- seq: 3091, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4188> 2014-11-03 11:37:53.796485 7f28ff81a700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796484, event: reached_pg, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4187> 2014-11-03 11:37:53.796521 7f28ff81a700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796521, event: started, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4186> 2014-11-03 11:37:53.796622 7f28ff81a700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796622, event: started, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4185> 2014-11-03 11:37:53.796655 7f28ff81a700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796655, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4184> 2014-11-03 11:37:53.796740 7f2914d0c700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.796740, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4183> 2014-11-03 11:37:53.797675 7f291450b700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.797675, event: journaled_completion_queued, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4182> 2014-11-03 11:37:53.797722 7f2911d06700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.797721, event: commit_sent, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4181> 2014-11-03 11:37:53.797755 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263ae840 con 0x22fbc420 -4180> 2014-11-03 11:37:53.797810 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(2079e6ac/rbd_data.74cd532a7698ee.0000000000000e31/head//6)]) v2 -- ?+0 0x2b950200 con 0x2301f580 -4179> 2014-11-03 11:37:53.798725 7f2912507700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.798725, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4178> 2014-11-03 11:37:53.798813 7f2912507700 5 -- op tracker -- seq: 3081, time: 2014-11-03 11:37:53.798813, event: done, op: osd_sub_op(client.12404444.0:39607 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374241 snapset=0=[]:[] snapc=0=[]) -4177> 2014-11-03 11:37:53.798922 7f2912507700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.798922, event: sub_op_applied, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4176> 2014-11-03 11:37:53.798965 7f2912507700 5 -- op tracker -- seq: 3083, time: 2014-11-03 11:37:53.798965, event: done, op: osd_sub_op(client.7868825.0:12014138 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502028 snapset=0=[]:[] snapc=0=[]) -4175> 2014-11-03 11:37:53.799144 7f2912507700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.799143, event: sub_op_applied, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4174> 2014-11-03 11:37:53.799222 7f2912507700 5 -- op tracker -- seq: 3084, time: 2014-11-03 11:37:53.799221, event: done, op: osd_sub_op(client.7868825.0:12014139 6.550 dcfad50/rbd_data.4d51727ffe6675.00000000000006a5/head//6 [] v 105950'29502029 snapset=0=[]:[] snapc=0=[]) -4173> 2014-11-03 11:37:53.799309 7f2912507700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.799309, event: sub_op_applied, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4172> 2014-11-03 11:37:53.799375 7f2912507700 5 -- op tracker -- seq: 3085, time: 2014-11-03 11:37:53.799375, event: done, op: osd_sub_op(client.8133525.0:451263 6.508 9f7ddd08/rbd_data.417ce42ae8944a.0000000000000153/head//6 [] v 105950'20000605 snapset=0=[]:[] snapc=0=[]) -4171> 2014-11-03 11:37:53.799486 7f2912507700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.799486, event: sub_op_applied, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4170> 2014-11-03 11:37:53.799543 7f2912507700 5 -- op tracker -- seq: 3086, time: 2014-11-03 11:37:53.799543, event: done, op: osd_sub_op(client.7869041.0:12286851 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900015 snapset=0=[]:[] snapc=0=[]) -4169> 2014-11-03 11:37:53.799689 7f2912507700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.799689, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4168> 2014-11-03 11:37:53.799741 7f2912507700 5 -- op tracker -- seq: 3087, time: 2014-11-03 11:37:53.799741, event: done, op: osd_sub_op(client.8703713.0:32254457 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953153 snapset=0=[]:[] snapc=0=[]) -4167> 2014-11-03 11:37:53.800017 7f2912507700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.800017, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4166> 2014-11-03 11:37:53.800064 7f2912507700 5 -- op tracker -- seq: 3088, time: 2014-11-03 11:37:53.800064, event: done, op: osd_sub_op(client.8703713.0:32254463 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97978 snapset=0=[]:[] snapc=0=[]) -4165> 2014-11-03 11:37:53.800204 7f2912507700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.800204, event: sub_op_applied, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4164> 2014-11-03 11:37:53.800251 7f2912507700 5 -- op tracker -- seq: 3089, time: 2014-11-03 11:37:53.800251, event: done, op: osd_sub_op(client.7930322.0:2285100 6.6fe 2a1276fe/rbd_data.7887825d9a0d8c.0000000000000e95/head//6 [] v 105950'9124374 snapset=0=[]:[] snapc=0=[]) -4163> 2014-11-03 11:37:53.800375 7f2912507700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.800375, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4162> 2014-11-03 11:37:53.800439 7f2912507700 5 -- op tracker -- seq: 3090, time: 2014-11-03 11:37:53.800439, event: done, op: osd_sub_op(client.8236383.0:16630867 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629964 snapset=0=[]:[] snapc=0=[]) -4161> 2014-11-03 11:37:53.800552 7f2912507700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.800551, event: sub_op_applied, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4160> 2014-11-03 11:37:53.800604 7f2912507700 5 -- op tracker -- seq: 3091, time: 2014-11-03 11:37:53.800603, event: done, op: osd_sub_op(client.7868407.0:1196452 6.4c4 a5064c4/rbd_data.596d152ecbea4f.0000000000000344/head//6 [] v 105950'10944776 snapset=0=[]:[] snapc=0=[]) -4159> 2014-11-03 11:37:53.802941 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- MRecoveryReserve RELEASE pgid: 6.58e, query_epoch: 105950 v2 -- ?+0 0x26fb1e00 con 0x22fbc420 -4158> 2014-11-03 11:37:53.802995 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- MRecoveryReserve RELEASE pgid: 6.58e, query_epoch: 105950 v2 -- ?+0 0x26fb1800 con 0x231abb00 -4157> 2014-11-03 11:37:53.803042 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.58e( v 105950'11622402 (105891'11618891,105950'11622402] local-les=105948 n=9172 ec=164 les/c 105948/105944 105947/105947/105947) [21,43,7] r=0 lpr=105947 pi=105933-105946/4 crt=105946'11622397 mlcod 105950'11622401 active+degraded] exit Started/Primary/Active/Recovering 9.685215 5 0.000118 -4156> 2014-11-03 11:37:53.803157 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.58e( v 105950'11622402 (105891'11618891,105950'11622402] local-les=105948 n=9172 ec=164 les/c 105948/105944 105947/105947/105947) [21,43,7] r=0 lpr=105947 pi=105933-105946/4 crt=105946'11622397 mlcod 105950'11622401 active+degraded] enter Started/Primary/Active/Recovered -4155> 2014-11-03 11:37:53.803230 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.58e( v 105950'11622402 (105891'11618891,105950'11622402] local-les=105948 n=9172 ec=164 les/c 105948/105944 105947/105947/105947) [21,43,7] r=0 lpr=105947 pi=105933-105946/4 crt=105946'11622397 mlcod 105950'11622401 active] exit Started/Primary/Active/Recovered 0.000073 0 0.000000 -4154> 2014-11-03 11:37:53.803262 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.58e( v 105950'11622402 (105891'11618891,105950'11622402] local-les=105948 n=9172 ec=164 les/c 105948/105944 105947/105947/105947) [21,43,7] r=0 lpr=105947 pi=105933-105946/4 crt=105946'11622397 mlcod 105950'11622401 active] enter Started/Primary/Active/Clean -4153> 2014-11-03 11:37:53.803329 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 lc 105946'65846 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[2e8~1,2fe~2,302~4,307~1]] exit Started/Primary/Active/WaitLocalRecoveryReserved 25.849680 2 0.000059 -4152> 2014-11-03 11:37:53.803394 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 lc 105946'65846 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[2e8~1,2fe~2,302~4,307~1]] enter Started/Primary/Active/WaitRemoteRecoveryReserved -4151> 2014-11-03 11:37:53.803406 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- pg_info(1 pgs e105950:6.58e) v4 -- ?+0 0x25e25800 con 0x22fbc420 -4150> 2014-11-03 11:37:53.803436 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- MRecoveryReserve REQUEST pgid: 24.a30, query_epoch: 105950 v2 -- ?+0 0x259f3a00 con 0x23184000 -4149> 2014-11-03 11:37:53.803617 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6812/19461 -- pg_info(1 pgs e105950:6.58e) v4 -- ?+0 0x252d7800 con 0x231abb00 -4148> 2014-11-03 11:37:53.804988 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.66 192.168.5.5:6852/919 7 ==== MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 v2 ==== 26+0+0 (1167531170 0 0) 0x259f3a00 con 0x23184000 -4147> 2014-11-03 11:37:53.805020 7f290b031700 5 -- op tracker -- seq: 3092, time: 2014-11-03 11:37:53.804894, event: header_read, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4146> 2014-11-03 11:37:53.805032 7f290b031700 5 -- op tracker -- seq: 3092, time: 2014-11-03 11:37:53.804896, event: throttled, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4145> 2014-11-03 11:37:53.805038 7f290b031700 5 -- op tracker -- seq: 3092, time: 2014-11-03 11:37:53.804949, event: all_read, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4144> 2014-11-03 11:37:53.805057 7f290b031700 5 -- op tracker -- seq: 3092, time: 2014-11-03 11:37:53.805015, event: dispatched, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4143> 2014-11-03 11:37:53.805068 7f290b031700 5 -- op tracker -- seq: 3092, time: 2014-11-03 11:37:53.805067, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4142> 2014-11-03 11:37:53.805112 7f290b031700 5 -- op tracker -- seq: 3092, time: 2014-11-03 11:37:53.805112, event: done, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4141> 2014-11-03 11:37:53.805137 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MRecoveryReserve REQUEST pgid: 24.a30, query_epoch: 105950 v2 -- ?+0 0x23299600 con 0x23000b00 -4140> 2014-11-03 11:37:53.806825 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 50 ==== MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 v2 ==== 26+0+0 (1167531170 0 0) 0x23299600 con 0x23000b00 -4139> 2014-11-03 11:37:53.806878 7f290b031700 5 -- op tracker -- seq: 3093, time: 2014-11-03 11:37:53.806671, event: header_read, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4138> 2014-11-03 11:37:53.806891 7f290b031700 5 -- op tracker -- seq: 3093, time: 2014-11-03 11:37:53.806674, event: throttled, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4137> 2014-11-03 11:37:53.806897 7f290b031700 5 -- op tracker -- seq: 3093, time: 2014-11-03 11:37:53.806750, event: all_read, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4136> 2014-11-03 11:37:53.806901 7f290b031700 5 -- op tracker -- seq: 3093, time: 2014-11-03 11:37:53.806873, event: dispatched, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4135> 2014-11-03 11:37:53.806923 7f290b031700 5 -- op tracker -- seq: 3093, time: 2014-11-03 11:37:53.806923, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4134> 2014-11-03 11:37:53.806986 7f290b031700 5 -- op tracker -- seq: 3093, time: 2014-11-03 11:37:53.806985, event: done, op: MRecoveryReserve GRANT pgid: 24.a30, query_epoch: 105950 -4133> 2014-11-03 11:37:53.807054 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 lc 105946'65846 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[2e8~1,2fe~2,302~4,307~1]] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.003660 2 0.000204 -4132> 2014-11-03 11:37:53.807120 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 lc 105946'65846 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[2e8~1,2fe~2,302~4,307~1]] enter Started/Primary/Active/Recovering -4131> 2014-11-03 11:37:53.807261 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MOSDPGPull(24.a30 105950 [PullOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x252d7a00 con 0x23000b00 -4130> 2014-11-03 11:37:53.812958 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.73 192.168.5.6:0/1260 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.811606) v2 ==== 47+0+0 (2287653449 0 0) 0x29fcd200 con 0x23b1c6e0 -4129> 2014-11-03 11:37:53.812989 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.73 192.168.5.6:0/1260 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.811606) v2 ==== 47+0+0 (2287653449 0 0) 0x29884c00 con 0x242c0dc0 -4128> 2014-11-03 11:37:53.813038 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/1260 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.811606) v2 -- ?+0 0x24800e00 con 0x23b1c6e0 -4127> 2014-11-03 11:37:53.813099 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/1260 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.811606) v2 -- ?+0 0x2ba4dc00 con 0x242c0dc0 -4126> 2014-11-03 11:37:53.839722 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 47 ==== osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (3792238035 0 3511520646) 0x25ed4c00 con 0x22fbc000 -4125> 2014-11-03 11:37:53.839766 7f28f71fd700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.839581, event: header_read, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4124> 2014-11-03 11:37:53.839800 7f28f71fd700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.839584, event: throttled, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4123> 2014-11-03 11:37:53.839823 7f28f71fd700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.839704, event: all_read, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4122> 2014-11-03 11:37:53.839839 7f28f71fd700 5 -- op tracker -- seq: 3094, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4121> 2014-11-03 11:37:53.839901 7f28fd816700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.839901, event: reached_pg, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4120> 2014-11-03 11:37:53.839949 7f28fd816700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.839949, event: started, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4119> 2014-11-03 11:37:53.840056 7f28fd816700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.840056, event: started, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4118> 2014-11-03 11:37:53.840099 7f28fd816700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.840099, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4117> 2014-11-03 11:37:53.871157 7f2914d0c700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:53.871157, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -4116> 2014-11-03 11:37:53.888029 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 116 ==== osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3318769932 0 3511520646) 0x29e1f400 con 0x22fb76e0 -4115> 2014-11-03 11:37:53.888070 7f28edd69700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:53.887908, event: header_read, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4114> 2014-11-03 11:37:53.888090 7f28edd69700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:53.887910, event: throttled, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4113> 2014-11-03 11:37:53.888105 7f28edd69700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:53.888018, event: all_read, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4112> 2014-11-03 11:37:53.888174 7f28edd69700 5 -- op tracker -- seq: 3095, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4111> 2014-11-03 11:37:53.888307 7f290001b700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:53.888307, event: reached_pg, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4110> 2014-11-03 11:37:53.888341 7f290001b700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:53.888340, event: started, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4109> 2014-11-03 11:37:53.888435 7f290001b700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:53.888435, event: started, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4108> 2014-11-03 11:37:53.888474 7f290001b700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:53.888474, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -4107> 2014-11-03 11:37:53.908611 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 51 ==== MOSDPGPush(24.a30 105950 [PushOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, version: 105946'65847, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195286+0+0 (2305328489 0 0) 0x252d7a00 con 0x23000b00 -4106> 2014-11-03 11:37:53.908675 7f28f21ad700 5 -- op tracker -- seq: 3096, time: 2014-11-03 11:37:53.884773, event: header_read, op: MOSDPGPush(24.a30 105950 [PushOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, version: 105946'65847, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4105> 2014-11-03 11:37:53.908718 7f28f21ad700 5 -- op tracker -- seq: 3096, time: 2014-11-03 11:37:53.884776, event: throttled, op: MOSDPGPush(24.a30 105950 [PushOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, version: 105946'65847, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4104> 2014-11-03 11:37:53.908736 7f28f21ad700 5 -- op tracker -- seq: 3096, time: 2014-11-03 11:37:53.908590, event: all_read, op: MOSDPGPush(24.a30 105950 [PushOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, version: 105946'65847, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4103> 2014-11-03 11:37:53.908749 7f28f21ad700 5 -- op tracker -- seq: 3096, time: 0.000000, event: dispatched, op: MOSDPGPush(24.a30 105950 [PushOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, version: 105946'65847, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4102> 2014-11-03 11:37:53.908900 7f28fe017700 5 -- op tracker -- seq: 3096, time: 2014-11-03 11:37:53.908900, event: reached_pg, op: MOSDPGPush(24.a30 105950 [PushOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, version: 105946'65847, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4101> 2014-11-03 11:37:53.909200 7f28fe017700 5 -- op tracker -- seq: 3096, time: 2014-11-03 11:37:53.909199, event: done, op: MOSDPGPush(24.a30 105950 [PushOp(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24, version: 105946'65847, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(50b93a30/rbd_data.47bd962ae8944a.000000000000082e/head//24@105946'65847, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -4100> 2014-11-03 11:37:53.910191 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.85 192.168.5.7:0/27520 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.908570) v2 ==== 47+0+0 (44880476 0 0) 0x23a9e800 con 0x233cb2c0 -4099> 2014-11-03 11:37:53.910204 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.85 192.168.5.7:0/27520 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.908570) v2 ==== 47+0+0 (44880476 0 0) 0x22a08800 con 0x233cb6e0 -4098> 2014-11-03 11:37:53.910269 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/27520 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.908570) v2 -- ?+0 0x29fcd200 con 0x233cb2c0 -4097> 2014-11-03 11:37:53.910341 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/27520 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.908570) v2 -- ?+0 0x29884c00 con 0x233cb6e0 -4096> 2014-11-03 11:37:53.914716 7f28e9326700 1 -- 192.168.5.2:6802/7606 <== client.9598152 192.168.9.2:0/1027858 11 ==== osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) v4 ==== 229+0+8192 (248615254 0 3219069030) 0x7c35280 con 0x2311c9a0 -4095> 2014-11-03 11:37:53.914751 7f28e9326700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.914622, event: header_read, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4094> 2014-11-03 11:37:53.914773 7f28e9326700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.914626, event: throttled, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4093> 2014-11-03 11:37:53.914791 7f28e9326700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.914703, event: all_read, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4092> 2014-11-03 11:37:53.914807 7f28e9326700 5 -- op tracker -- seq: 3097, time: 0.000000, event: dispatched, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4091> 2014-11-03 11:37:53.914932 7f28fe818700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.914932, event: reached_pg, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4090> 2014-11-03 11:37:53.915116 7f28fe818700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.915116, event: started, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4089> 2014-11-03 11:37:53.915144 7f28fe818700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.915143, event: started, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4088> 2014-11-03 11:37:53.915201 7f28fe818700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.915201, event: waiting for subops from 45,107, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4087> 2014-11-03 11:37:53.915240 7f28fe818700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131095 snapset=0=[]:[] snapc=0=[]) v11 -- ?+9021 0x2cff3000 con 0x22ff3420 -4086> 2014-11-03 11:37:53.915339 7f28fe818700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6833/17474 -- osd_sub_op(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131095 snapset=0=[]:[] snapc=0=[]) v11 -- ?+9021 0x26dbc400 con 0x22fe0dc0 -4085> 2014-11-03 11:37:53.915455 7f28fe818700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.915454, event: commit_queued_for_journal_write, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4084> 2014-11-03 11:37:53.917239 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 38 ==== osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (204182821 0 0) 0xa23e000 con 0x22ff3420 -4083> 2014-11-03 11:37:53.917294 7f28f0b97700 5 -- op tracker -- seq: 3098, time: 2014-11-03 11:37:53.917137, event: header_read, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4082> 2014-11-03 11:37:53.917326 7f28f0b97700 5 -- op tracker -- seq: 3098, time: 2014-11-03 11:37:53.917139, event: throttled, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4081> 2014-11-03 11:37:53.917339 7f28f0b97700 5 -- op tracker -- seq: 3098, time: 2014-11-03 11:37:53.917223, event: all_read, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4080> 2014-11-03 11:37:53.917351 7f28f0b97700 5 -- op tracker -- seq: 3098, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4079> 2014-11-03 11:37:53.917410 7f290101d700 5 -- op tracker -- seq: 3098, time: 2014-11-03 11:37:53.917410, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4078> 2014-11-03 11:37:53.917432 7f290101d700 5 -- op tracker -- seq: 3098, time: 2014-11-03 11:37:53.917432, event: started, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4077> 2014-11-03 11:37:53.917445 7f290101d700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.917445, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4076> 2014-11-03 11:37:53.917469 7f290101d700 5 -- op tracker -- seq: 3098, time: 2014-11-03 11:37:53.917469, event: done, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4075> 2014-11-03 11:37:53.917455 7f28f52de700 1 -- 192.168.5.2:6803/7606 <== osd.107 192.168.5.8:6833/17474 34 ==== osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (1800821114 0 0) 0xa23f8c0 con 0x22fe0dc0 -4074> 2014-11-03 11:37:53.917514 7f28f52de700 5 -- op tracker -- seq: 3099, time: 2014-11-03 11:37:53.917392, event: header_read, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4073> 2014-11-03 11:37:53.917531 7f28f52de700 5 -- op tracker -- seq: 3099, time: 2014-11-03 11:37:53.917393, event: throttled, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4072> 2014-11-03 11:37:53.917541 7f28f52de700 5 -- op tracker -- seq: 3099, time: 2014-11-03 11:37:53.917444, event: all_read, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4071> 2014-11-03 11:37:53.917563 7f28f52de700 5 -- op tracker -- seq: 3099, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4070> 2014-11-03 11:37:53.917622 7f28fe818700 5 -- op tracker -- seq: 3099, time: 2014-11-03 11:37:53.917622, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4069> 2014-11-03 11:37:53.917653 7f28fe818700 5 -- op tracker -- seq: 3099, time: 2014-11-03 11:37:53.917653, event: started, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4068> 2014-11-03 11:37:53.917666 7f28fe818700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:53.917666, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -4067> 2014-11-03 11:37:53.917692 7f28fe818700 5 -- op tracker -- seq: 3099, time: 2014-11-03 11:37:53.917692, event: done, op: osd_sub_op_reply(client.9598152.0:459475 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -4066> 2014-11-03 11:37:53.934002 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 117 ==== osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2786355269 0 3511520646) 0x29e1dc00 con 0x22fb76e0 -4065> 2014-11-03 11:37:53.934036 7f28edd69700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:53.933897, event: header_read, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4064> 2014-11-03 11:37:53.934050 7f28edd69700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:53.933899, event: throttled, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4063> 2014-11-03 11:37:53.934067 7f28edd69700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:53.933986, event: all_read, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4062> 2014-11-03 11:37:53.934083 7f28edd69700 5 -- op tracker -- seq: 3100, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4061> 2014-11-03 11:37:53.934183 7f290101d700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:53.934183, event: reached_pg, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4060> 2014-11-03 11:37:53.934216 7f290101d700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:53.934216, event: started, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4059> 2014-11-03 11:37:53.934302 7f290101d700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:53.934301, event: started, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4058> 2014-11-03 11:37:53.934329 7f290101d700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:53.934329, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -4057> 2014-11-03 11:37:53.959290 7f28e60f4700 1 -- 192.168.5.2:6802/7606 <== client.9732651 192.168.1.69:0/1023508 4 ==== osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) v4 ==== 266+0+131072 (1690661497 0 3944876957) 0x2d281280 con 0x23158b00 -4056> 2014-11-03 11:37:53.959332 7f28e60f4700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.958648, event: header_read, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4055> 2014-11-03 11:37:53.959347 7f28e60f4700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.958651, event: throttled, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4054> 2014-11-03 11:37:53.959357 7f28e60f4700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.959274, event: all_read, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4053> 2014-11-03 11:37:53.959367 7f28e60f4700 5 -- op tracker -- seq: 3101, time: 0.000000, event: dispatched, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4052> 2014-11-03 11:37:53.959466 7f290201f700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.959466, event: reached_pg, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4051> 2014-11-03 11:37:53.959583 7f290201f700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.959583, event: started, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4050> 2014-11-03 11:37:53.959610 7f290201f700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.959610, event: started, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4049> 2014-11-03 11:37:53.959749 7f290201f700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.959749, event: waiting for subops from 49,93, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4048> 2014-11-03 11:37:53.959819 7f290201f700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- osd_sub_op(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] v 105950'7950028 snapset=0=[]:[] snapc=0=[]) v11 -- ?+131892 0x2cff4800 con 0x22fb7160 -4047> 2014-11-03 11:37:53.959899 7f290201f700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- osd_sub_op(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] v 105950'7950028 snapset=0=[]:[] snapc=0=[]) v11 -- ?+131892 0x25ed5800 con 0x22fe0420 -4046> 2014-11-03 11:37:53.960039 7f290201f700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.960039, event: commit_queued_for_journal_write, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4045> 2014-11-03 11:37:53.962872 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 39 ==== osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) v2 ==== 177+0+0 (3491292581 0 0) 0x27417000 con 0x22fe0420 -4044> 2014-11-03 11:37:53.962917 7f28f5be7700 5 -- op tracker -- seq: 3102, time: 2014-11-03 11:37:53.962770, event: header_read, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4043> 2014-11-03 11:37:53.962936 7f28f5be7700 5 -- op tracker -- seq: 3102, time: 2014-11-03 11:37:53.962772, event: throttled, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4042> 2014-11-03 11:37:53.962959 7f28f5be7700 5 -- op tracker -- seq: 3102, time: 2014-11-03 11:37:53.962857, event: all_read, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4041> 2014-11-03 11:37:53.962984 7f28f5be7700 5 -- op tracker -- seq: 3102, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4040> 2014-11-03 11:37:53.963050 7f28ff81a700 5 -- op tracker -- seq: 3102, time: 2014-11-03 11:37:53.963050, event: reached_pg, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4039> 2014-11-03 11:37:53.963083 7f28ff81a700 5 -- op tracker -- seq: 3102, time: 2014-11-03 11:37:53.963083, event: started, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4038> 2014-11-03 11:37:53.963106 7f28ff81a700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.963106, event: sub_op_commit_rec, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4037> 2014-11-03 11:37:53.963143 7f28ff81a700 5 -- op tracker -- seq: 3102, time: 2014-11-03 11:37:53.963143, event: done, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4036> 2014-11-03 11:37:53.963470 7f28ed05c700 1 -- 192.168.5.2:6803/7606 <== osd.49 192.168.5.4:6841/21769 51 ==== osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) v2 ==== 177+0+0 (2300282795 0 0) 0x240a9b80 con 0x22fb7160 -4035> 2014-11-03 11:37:53.963500 7f28ed05c700 5 -- op tracker -- seq: 3103, time: 2014-11-03 11:37:53.963350, event: header_read, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4034> 2014-11-03 11:37:53.963528 7f28ed05c700 5 -- op tracker -- seq: 3103, time: 2014-11-03 11:37:53.963351, event: throttled, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4033> 2014-11-03 11:37:53.963541 7f28ed05c700 5 -- op tracker -- seq: 3103, time: 2014-11-03 11:37:53.963457, event: all_read, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4032> 2014-11-03 11:37:53.963551 7f28ed05c700 5 -- op tracker -- seq: 3103, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4031> 2014-11-03 11:37:53.963600 7f290201f700 5 -- op tracker -- seq: 3103, time: 2014-11-03 11:37:53.963600, event: reached_pg, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4030> 2014-11-03 11:37:53.963625 7f290201f700 5 -- op tracker -- seq: 3103, time: 2014-11-03 11:37:53.963625, event: started, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4029> 2014-11-03 11:37:53.963643 7f290201f700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:53.963643, event: sub_op_commit_rec, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -4028> 2014-11-03 11:37:53.963662 7f290201f700 5 -- op tracker -- seq: 3103, time: 2014-11-03 11:37:53.963662, event: done, op: osd_sub_op_reply(client.9732651.0:220595 6.9d8 f6f4e9d8/rbd_data.95169218ab737.0000000000000acd/head//6 [] ondisk, result = 0) -4027> 2014-11-03 11:37:53.967492 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.107 192.168.5.8:0/17474 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.965987) v2 ==== 47+0+0 (3657961239 0 0) 0x2b950a00 con 0x239f8580 -4026> 2014-11-03 11:37:53.967537 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.965987) v2 -- ?+0 0x23a9e800 con 0x239f8580 -4025> 2014-11-03 11:37:53.967568 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.107 192.168.5.8:0/17474 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:53.965987) v2 ==== 47+0+0 (3657961239 0 0) 0x23305200 con 0x238b86e0 -4024> 2014-11-03 11:37:53.967628 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:53.965987) v2 -- ?+0 0x22a08800 con 0x238b86e0 -4023> 2014-11-03 11:37:53.981689 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x315258c0 sd=171 :6802 s=0 pgs=0 cs=0 l=0 c=0x26906c60).accept sd=171 192.168.5.2:42945/0 -4022> 2014-11-03 11:37:53.983058 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419940 192.168.5.2:0/1002819 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23a9ee00 con 0x26906c60 -4021> 2014-11-03 11:37:53.983765 7f28fb812700 0 osd.21 105950 do_command r=0 -4020> 2014-11-03 11:37:53.983803 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1002819 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968fa80 con 0x26906c60 -4019> 2014-11-03 11:37:53.998024 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419940 192.168.5.2:0/1002819 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23a9e000 con 0x26906c60 -4018> 2014-11-03 11:37:53.998823 7f28fb812700 0 osd.21 105950 do_command r=0 -4017> 2014-11-03 11:37:53.998839 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1002819 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x6ef0000 con 0x26906c60 -4016> 2014-11-03 11:37:54.005237 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1002819 pipe(0x315258c0 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x26906c60).reader couldn't read tag, (0) Success -4015> 2014-11-03 11:37:54.005275 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1002819 pipe(0x315258c0 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x26906c60).fault (0) Success -4014> 2014-11-03 11:37:54.005371 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x26906c60 session 0x235df000 -4013> 2014-11-03 11:37:54.036058 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.6 192.168.5.1:0/1181 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.035723) v2 ==== 47+0+0 (86506824 0 0) 0x23305e00 con 0x242299a0 -4012> 2014-11-03 11:37:54.036067 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.6 192.168.5.1:0/1181 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.035723) v2 ==== 47+0+0 (86506824 0 0) 0x23a41000 con 0x24925c60 -4011> 2014-11-03 11:37:54.036132 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/1181 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.035723) v2 -- ?+0 0x23305200 con 0x242299a0 -4010> 2014-11-03 11:37:54.036200 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/1181 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.035723) v2 -- ?+0 0x2b950a00 con 0x24925c60 -4009> 2014-11-03 11:37:54.036971 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.64 192.168.5.5:0/24829 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.035755) v2 ==== 47+0+0 (3047144559 0 0) 0xa5d2800 con 0x23b1c2c0 -4008> 2014-11-03 11:37:54.037007 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/24829 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.035755) v2 -- ?+0 0x23a41000 con 0x23b1c2c0 -4007> 2014-11-03 11:37:54.037136 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.64 192.168.5.5:0/24829 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.035755) v2 ==== 47+0+0 (3047144559 0 0) 0x31a6800 con 0x161ffdc0 -4006> 2014-11-03 11:37:54.037175 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/24829 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.035755) v2 -- ?+0 0x23305e00 con 0x161ffdc0 -4005> 2014-11-03 11:37:54.068432 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.48 192.168.5.4:0/17727 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.067376) v2 ==== 47+0+0 (1494855642 0 0) 0x2a7c4a00 con 0x23335420 -4004> 2014-11-03 11:37:54.068477 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/17727 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.067376) v2 -- ?+0 0xa5d2800 con 0x23335420 -4003> 2014-11-03 11:37:54.068615 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.48 192.168.5.4:0/17727 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.067376) v2 ==== 47+0+0 (1494855642 0 0) 0x23576e00 con 0x234ed420 -4002> 2014-11-03 11:37:54.068681 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/17727 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.067376) v2 -- ?+0 0x31a6800 con 0x234ed420 -4001> 2014-11-03 11:37:54.087817 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 36 ==== MOSDPGPush(6.6ac 105950 [PushOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6, version: 105938'9805863, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6@105938'9805863, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6, version: 105938'9805864, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6@105938'9805864, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (542714152 0 0) 0x2b950200 con 0x2301f580 -4000> 2014-11-03 11:37:54.087873 7f28f2db9700 5 -- op tracker -- seq: 3104, time: 2014-11-03 11:37:54.063171, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6, version: 105938'9805863, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6@105938'9805863, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6, version: 105938'9805864, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6@105938'9805864, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3999> 2014-11-03 11:37:54.087914 7f28f2db9700 5 -- op tracker -- seq: 3104, time: 2014-11-03 11:37:54.063174, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6, version: 105938'9805863, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6@105938'9805863, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6, version: 105938'9805864, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6@105938'9805864, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3998> 2014-11-03 11:37:54.087945 7f28f2db9700 5 -- op tracker -- seq: 3104, time: 2014-11-03 11:37:54.087800, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6, version: 105938'9805863, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6@105938'9805863, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6, version: 105938'9805864, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6@105938'9805864, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3997> 2014-11-03 11:37:54.087970 7f28f2db9700 5 -- op tracker -- seq: 3104, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6, version: 105938'9805863, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6@105938'9805863, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6, version: 105938'9805864, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6@105938'9805864, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3996> 2014-11-03 11:37:54.088118 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.89 192.168.5.7:0/24271 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.086027) v2 ==== 47+0+0 (2114022102 0 0) 0x2310a400 con 0x24925580 -3995> 2014-11-03 11:37:54.088121 7f290081c700 5 -- op tracker -- seq: 3104, time: 2014-11-03 11:37:54.088120, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6, version: 105938'9805863, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6@105938'9805863, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6, version: 105938'9805864, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6@105938'9805864, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3994> 2014-11-03 11:37:54.088168 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/24271 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.086027) v2 -- ?+0 0x2a7c4a00 con 0x24925580 -3993> 2014-11-03 11:37:54.088172 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.89 192.168.5.7:0/24271 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.086027) v2 ==== 47+0+0 (2114022102 0 0) 0x2782fe00 con 0x242292c0 -3992> 2014-11-03 11:37:54.088222 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/24271 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.086027) v2 -- ?+0 0x23576e00 con 0x242292c0 -3991> 2014-11-03 11:37:54.088352 7f290081c700 5 -- op tracker -- seq: 3104, time: 2014-11-03 11:37:54.088352, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6, version: 105938'9805863, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6@105938'9805863, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6, version: 105938'9805864, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6@105938'9805864, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3990> 2014-11-03 11:37:54.123102 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 37 ==== MOSDPGPush(6.6ac 105950 [PushOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6, version: 105938'9805868, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6@105938'9805868, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6, version: 105945'9805884, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6@105945'9805884, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (3656528309 0 0) 0xa6b5400 con 0x2301f580 -3989> 2014-11-03 11:37:54.123169 7f28f2db9700 5 -- op tracker -- seq: 3105, time: 2014-11-03 11:37:54.088035, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6, version: 105938'9805868, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6@105938'9805868, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6, version: 105945'9805884, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6@105945'9805884, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3988> 2014-11-03 11:37:54.123249 7f28f2db9700 5 -- op tracker -- seq: 3105, time: 2014-11-03 11:37:54.088036, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6, version: 105938'9805868, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6@105938'9805868, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6, version: 105945'9805884, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6@105945'9805884, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3987> 2014-11-03 11:37:54.123307 7f28f2db9700 5 -- op tracker -- seq: 3105, time: 2014-11-03 11:37:54.123084, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6, version: 105938'9805868, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6@105938'9805868, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6, version: 105945'9805884, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6@105945'9805884, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3986> 2014-11-03 11:37:54.123352 7f28f2db9700 5 -- op tracker -- seq: 3105, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6, version: 105938'9805868, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6@105938'9805868, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6, version: 105945'9805884, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6@105945'9805884, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3985> 2014-11-03 11:37:54.123518 7f28fe017700 5 -- op tracker -- seq: 3105, time: 2014-11-03 11:37:54.123517, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6, version: 105938'9805868, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6@105938'9805868, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6, version: 105945'9805884, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6@105945'9805884, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3984> 2014-11-03 11:37:54.123907 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31525080 sd=171 :6802 s=0 pgs=0 cs=0 l=0 c=0x2408bdc0).accept sd=171 192.168.5.2:42948/0 -3983> 2014-11-03 11:37:54.124024 7f28fe017700 5 -- op tracker -- seq: 3105, time: 2014-11-03 11:37:54.124023, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6, version: 105938'9805868, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6@105938'9805868, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6, version: 105945'9805884, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6@105945'9805884, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3982> 2014-11-03 11:37:54.124639 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450904 192.168.5.2:0/1003073 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2782fc00 con 0x2408bdc0 -3981> 2014-11-03 11:37:54.125124 7f28fb812700 0 osd.21 105950 do_command r=0 -3980> 2014-11-03 11:37:54.125147 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003073 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f8c0 con 0x2408bdc0 -3979> 2014-11-03 11:37:54.129184 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 33 ==== osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2968115652 0 3511520646) 0x25eb9400 con 0x22fe0840 -3978> 2014-11-03 11:37:54.129221 7f28f4cd8700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.128979, event: header_read, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3977> 2014-11-03 11:37:54.129241 7f28f4cd8700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.128981, event: throttled, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3976> 2014-11-03 11:37:54.129260 7f28f4cd8700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.129166, event: all_read, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3975> 2014-11-03 11:37:54.129295 7f28f4cd8700 5 -- op tracker -- seq: 3106, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3974> 2014-11-03 11:37:54.129388 7f290081c700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.129388, event: reached_pg, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3973> 2014-11-03 11:37:54.129423 7f290081c700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.129423, event: started, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3972> 2014-11-03 11:37:54.129520 7f290081c700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.129520, event: started, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3971> 2014-11-03 11:37:54.129552 7f290081c700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.129552, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3970> 2014-11-03 11:37:54.130079 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 34 ==== osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3677126937 0 3511520646) 0x2fdf4000 con 0x22fe0840 -3969> 2014-11-03 11:37:54.130151 7f28f4cd8700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.129964, event: header_read, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3968> 2014-11-03 11:37:54.130192 7f28f4cd8700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.129965, event: throttled, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3967> 2014-11-03 11:37:54.130213 7f28f4cd8700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.130069, event: all_read, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3966> 2014-11-03 11:37:54.130230 7f28f4cd8700 5 -- op tracker -- seq: 3107, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3965> 2014-11-03 11:37:54.130274 7f28fe017700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.130274, event: reached_pg, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3964> 2014-11-03 11:37:54.130302 7f28fe017700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.130302, event: started, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3963> 2014-11-03 11:37:54.130381 7f28fe017700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.130381, event: started, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3962> 2014-11-03 11:37:54.130407 7f28fe017700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.130407, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3961> 2014-11-03 11:37:54.134643 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450904 192.168.5.2:0/1003073 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x259f3200 con 0x2408bdc0 -3960> 2014-11-03 11:37:54.135455 7f28fb812700 0 osd.21 105950 do_command r=0 -3959> 2014-11-03 11:37:54.135472 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003073 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x6ef0000 con 0x2408bdc0 -3958> 2014-11-03 11:37:54.140897 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 38 ==== MOSDPGPush(6.6ac 105950 [PushOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6, version: 105945'9805887, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6@105945'9805887, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (2231658042 0 0) 0x2310a800 con 0x2301f580 -3957> 2014-11-03 11:37:54.140966 7f28f2db9700 5 -- op tracker -- seq: 3108, time: 2014-11-03 11:37:54.123456, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6, version: 105945'9805887, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6@105945'9805887, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3956> 2014-11-03 11:37:54.141007 7f28f2db9700 5 -- op tracker -- seq: 3108, time: 2014-11-03 11:37:54.123465, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6, version: 105945'9805887, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6@105945'9805887, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3955> 2014-11-03 11:37:54.141027 7f28f2db9700 5 -- op tracker -- seq: 3108, time: 2014-11-03 11:37:54.140874, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6, version: 105945'9805887, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6@105945'9805887, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3954> 2014-11-03 11:37:54.141047 7f28f2db9700 5 -- op tracker -- seq: 3108, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6, version: 105945'9805887, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6@105945'9805887, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3953> 2014-11-03 11:37:54.141186 7f290081c700 5 -- op tracker -- seq: 3108, time: 2014-11-03 11:37:54.141185, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6, version: 105945'9805887, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6@105945'9805887, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3952> 2014-11-03 11:37:54.141390 7f290081c700 5 -- op tracker -- seq: 3108, time: 2014-11-03 11:37:54.141390, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6, version: 105945'9805887, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6@105945'9805887, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3951> 2014-11-03 11:37:54.142767 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003073 pipe(0x31525080 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x2408bdc0).reader couldn't read tag, (0) Success -3950> 2014-11-03 11:37:54.142811 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003073 pipe(0x31525080 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x2408bdc0).fault (0) Success -3949> 2014-11-03 11:37:54.142937 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2408bdc0 session 0x235df000 -3948> 2014-11-03 11:37:54.156929 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 109 ==== osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3910498460 0 3511520646) 0x2c338400 con 0x22fe0c60 -3947> 2014-11-03 11:37:54.157016 7f28f4eda700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.156759, event: header_read, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3946> 2014-11-03 11:37:54.157037 7f28f4eda700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.156763, event: throttled, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3945> 2014-11-03 11:37:54.157055 7f28f4eda700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.156909, event: all_read, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3944> 2014-11-03 11:37:54.157069 7f28f4eda700 5 -- op tracker -- seq: 3109, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3943> 2014-11-03 11:37:54.157161 7f28fe017700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.157160, event: reached_pg, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3942> 2014-11-03 11:37:54.157202 7f28fe017700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.157201, event: started, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3941> 2014-11-03 11:37:54.157300 7f28fe017700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.157300, event: started, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3940> 2014-11-03 11:37:54.157334 7f28fe017700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.157334, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3939> 2014-11-03 11:37:54.158273 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 110 ==== osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3498017202 0 3511520646) 0x2e169400 con 0x22fe0c60 -3938> 2014-11-03 11:37:54.158326 7f28f4eda700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.158165, event: header_read, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3937> 2014-11-03 11:37:54.158340 7f28f4eda700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.158166, event: throttled, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3936> 2014-11-03 11:37:54.158349 7f28f4eda700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.158260, event: all_read, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3935> 2014-11-03 11:37:54.158378 7f28f4eda700 5 -- op tracker -- seq: 3110, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3934> 2014-11-03 11:37:54.158503 7f290081c700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.158503, event: reached_pg, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3933> 2014-11-03 11:37:54.158559 7f290081c700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.158559, event: started, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3932> 2014-11-03 11:37:54.158696 7f290081c700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.158695, event: started, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3931> 2014-11-03 11:37:54.158740 7f290081c700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.158739, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3930> 2014-11-03 11:37:54.159679 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 111 ==== osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1435603563 0 3511520646) 0x2d47bc00 con 0x22fe0c60 -3929> 2014-11-03 11:37:54.159719 7f28f4eda700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.159596, event: header_read, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3928> 2014-11-03 11:37:54.159742 7f28f4eda700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.159598, event: throttled, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3927> 2014-11-03 11:37:54.159755 7f28f4eda700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.159668, event: all_read, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3926> 2014-11-03 11:37:54.159771 7f28f4eda700 5 -- op tracker -- seq: 3111, time: 0.000000, event: dispatched, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3925> 2014-11-03 11:37:54.159834 7f28fe017700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.159833, event: reached_pg, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3924> 2014-11-03 11:37:54.159861 7f28fe017700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.159861, event: started, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3923> 2014-11-03 11:37:54.159954 7f28fe017700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.159954, event: started, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3922> 2014-11-03 11:37:54.159982 7f28fe017700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.159982, event: commit_queued_for_journal_write, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3921> 2014-11-03 11:37:54.168298 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 173 ==== osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2187185595 0 3511520646) 0x26942000 con 0x22fefdc0 -3920> 2014-11-03 11:37:54.168335 7f28f36c2700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.168194, event: header_read, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3919> 2014-11-03 11:37:54.168353 7f28f36c2700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.168196, event: throttled, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3918> 2014-11-03 11:37:54.168366 7f28f36c2700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.168283, event: all_read, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3917> 2014-11-03 11:37:54.168377 7f28f36c2700 5 -- op tracker -- seq: 3112, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3916> 2014-11-03 11:37:54.168449 7f28fe818700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.168449, event: reached_pg, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3915> 2014-11-03 11:37:54.168480 7f28fe818700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.168480, event: started, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3914> 2014-11-03 11:37:54.168589 7f28fe818700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.168588, event: started, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3913> 2014-11-03 11:37:54.168621 7f28fe818700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.168621, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3912> 2014-11-03 11:37:54.169348 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.47 192.168.5.4:0/12077 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.167616) v2 ==== 47+0+0 (4132708453 0 0) 0x3353000 con 0x243216e0 -3911> 2014-11-03 11:37:54.169411 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.167616) v2 -- ?+0 0x2782fe00 con 0x243216e0 -3910> 2014-11-03 11:37:54.169408 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.47 192.168.5.4:0/12077 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.167616) v2 ==== 47+0+0 (4132708453 0 0) 0x2a8efe00 con 0x242926e0 -3909> 2014-11-03 11:37:54.169490 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.167616) v2 -- ?+0 0x2310a400 con 0x242926e0 -3908> 2014-11-03 11:37:54.170263 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31525600 sd=171 :6802 s=0 pgs=0 cs=0 l=0 c=0x7553000).accept sd=171 192.168.5.201:58688/0 -3907> 2014-11-03 11:37:54.171052 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450907 192.168.5.201:0/1029670 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "24.7d7"}) v1 ==== 80+0+0 (3756934197 0 0) 0x2a3f8000 con 0x7553000 -3906> 2014-11-03 11:37:54.171496 7f28fb812700 0 osd.21 105950 do_command r=0 -3905> 2014-11-03 11:37:54.171517 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029670 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f700 con 0x7553000 -3904> 2014-11-03 11:37:54.173711 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.11 192.168.5.1:0/18824 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.173374) v2 ==== 47+0+0 (4055958768 0 0) 0x23640600 con 0x244ce000 -3903> 2014-11-03 11:37:54.173753 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.11 192.168.5.1:0/18824 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.173374) v2 ==== 47+0+0 (4055958768 0 0) 0x24139a00 con 0x238b8dc0 -3902> 2014-11-03 11:37:54.173764 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/18824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.173374) v2 -- ?+0 0x2a8efe00 con 0x244ce000 -3901> 2014-11-03 11:37:54.173895 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/18824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.173374) v2 -- ?+0 0x3353000 con 0x238b8dc0 -3900> 2014-11-03 11:37:54.179293 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450907 192.168.5.201:0/1029670 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "24.7d7"}) v1 ==== 74+0+0 (774687807 0 0) 0x23510200 con 0x7553000 -3899> 2014-11-03 11:37:54.180003 7f28fb812700 0 osd.21 105950 do_command r=0 -3898> 2014-11-03 11:37:54.180018 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029670 -- command_reply(tid 2: 0 ) v1 -- ?+13413 0x6ef01c0 con 0x7553000 -3897> 2014-11-03 11:37:54.185043 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029670 pipe(0x31525600 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x7553000).reader couldn't read tag, (0) Success -3896> 2014-11-03 11:37:54.185075 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029670 pipe(0x31525600 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x7553000).fault (0) Success -3895> 2014-11-03 11:37:54.185196 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x7553000 session 0x235df000 -3894> 2014-11-03 11:37:54.208901 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 91 ==== osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2117130538 0 3511520646) 0x2de0d000 con 0x22ff3b00 -3893> 2014-11-03 11:37:54.208973 7f28f0d99700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.208658, event: header_read, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3892> 2014-11-03 11:37:54.209000 7f28f0d99700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.208663, event: throttled, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3891> 2014-11-03 11:37:54.209020 7f28f0d99700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.208875, event: all_read, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3890> 2014-11-03 11:37:54.209038 7f28f0d99700 5 -- op tracker -- seq: 3113, time: 0.000000, event: dispatched, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3889> 2014-11-03 11:37:54.209103 7f28fd816700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.209103, event: reached_pg, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3888> 2014-11-03 11:37:54.209138 7f28fd816700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.209138, event: started, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3887> 2014-11-03 11:37:54.209303 7f28fd816700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.209302, event: started, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3886> 2014-11-03 11:37:54.209351 7f28fd816700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.209351, event: commit_queued_for_journal_write, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3885> 2014-11-03 11:37:54.211145 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.76 192.168.5.6:0/22493 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.210636) v2 ==== 47+0+0 (3537015536 0 0) 0x8025c00 con 0x24229dc0 -3884> 2014-11-03 11:37:54.211197 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.210636) v2 -- ?+0 0x24139a00 con 0x24229dc0 -3883> 2014-11-03 11:37:54.211584 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.76 192.168.5.6:0/22493 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.210636) v2 ==== 47+0+0 (3537015536 0 0) 0x2d361000 con 0x23b1c160 -3882> 2014-11-03 11:37:54.211659 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.210636) v2 -- ?+0 0x23640600 con 0x23b1c160 -3881> 2014-11-03 11:37:54.221212 7f2918d14700 5 osd.21 105950 tick -3880> 2014-11-03 11:37:54.221273 7f2918d14700 10 monclient: _send_mon_message to mon.a at 192.168.5.201:6789/0 -3879> 2014-11-03 11:37:54.221291 7f2918d14700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:6789/0 -- pg_stats(4 pgs tid 6 v 0) v1 -- ?+0 0x3149f2c0 con 0x22c29160 -3878> 2014-11-03 11:37:54.262629 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.74 192.168.5.6:0/9268 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.261660) v2 ==== 47+0+0 (2615971903 0 0) 0x23247000 con 0x24925420 -3877> 2014-11-03 11:37:54.262689 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/9268 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.261660) v2 -- ?+0 0x2d361000 con 0x24925420 -3876> 2014-11-03 11:37:54.262693 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.74 192.168.5.6:0/9268 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.261660) v2 ==== 47+0+0 (2615971903 0 0) 0x2340c000 con 0x24229160 -3875> 2014-11-03 11:37:54.262757 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/9268 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.261660) v2 -- ?+0 0x8025c00 con 0x24229160 -3874> 2014-11-03 11:37:54.273789 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31524b00 sd=171 :6802 s=0 pgs=0 cs=0 l=0 c=0x75532c0).accept sd=171 192.168.5.2:42952/0 -3873> 2014-11-03 11:37:54.274741 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450782 192.168.5.2:0/1003226 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2372bc00 con 0x75532c0 -3872> 2014-11-03 11:37:54.275437 7f28fb812700 0 osd.21 105950 do_command r=0 -3871> 2014-11-03 11:37:54.275477 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003226 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f000 con 0x75532c0 -3870> 2014-11-03 11:37:54.281718 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.101 192.168.5.8:0/13676 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.279389) v2 ==== 47+0+0 (2698394984 0 0) 0x2a2e0e00 con 0x239f86e0 -3869> 2014-11-03 11:37:54.281764 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/13676 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.279389) v2 -- ?+0 0x23247000 con 0x239f86e0 -3868> 2014-11-03 11:37:54.281805 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.101 192.168.5.8:0/13676 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.279389) v2 ==== 47+0+0 (2698394984 0 0) 0x24dce800 con 0x238b8840 -3867> 2014-11-03 11:37:54.281851 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/13676 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.279389) v2 -- ?+0 0x2340c000 con 0x238b8840 -3866> 2014-11-03 11:37:54.285530 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450782 192.168.5.2:0/1003226 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23414000 con 0x75532c0 -3865> 2014-11-03 11:37:54.286341 7f28fb812700 0 osd.21 105950 do_command r=0 -3864> 2014-11-03 11:37:54.286355 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003226 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x6ef0000 con 0x75532c0 -3863> 2014-11-03 11:37:54.293505 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003226 pipe(0x31524b00 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x75532c0).reader couldn't read tag, (0) Success -3862> 2014-11-03 11:37:54.293549 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003226 pipe(0x31524b00 sd=171 :6802 s=2 pgs=2 cs=1 l=1 c=0x75532c0).fault (0) Success -3861> 2014-11-03 11:37:54.293629 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x75532c0 session 0x7c35a00 -3860> 2014-11-03 11:37:54.301560 7f291450b700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:54.301560, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -3859> 2014-11-03 11:37:54.301630 7f2911d06700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:54.301629, event: commit_sent, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -3858> 2014-11-03 11:37:54.301657 7f2914d0c700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:54.301657, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -3857> 2014-11-03 11:37:54.301684 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263ae000 con 0x22fbc000 -3856> 2014-11-03 11:37:54.302097 7f2912507700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:54.302096, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -3855> 2014-11-03 11:37:54.302187 7f2912507700 5 -- op tracker -- seq: 3094, time: 2014-11-03 11:37:54.302187, event: done, op: osd_sub_op(client.7867942.0:137197985 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100464 snapset=0=[]:[] snapc=0=[]) -3854> 2014-11-03 11:37:54.302532 7f2914d0c700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:54.302532, event: write_thread_in_journal_buffer, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -3853> 2014-11-03 11:37:54.302589 7f2914d0c700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:54.302589, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -3852> 2014-11-03 11:37:54.302639 7f2914d0c700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:54.302639, event: write_thread_in_journal_buffer, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -3851> 2014-11-03 11:37:54.305976 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 97 ==== osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (3747616655 0 3511520646) 0x2d47b000 con 0x22fbc580 -3850> 2014-11-03 11:37:54.306037 7f28f70fc700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.305762, event: header_read, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3849> 2014-11-03 11:37:54.306053 7f28f70fc700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.305765, event: throttled, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3848> 2014-11-03 11:37:54.306062 7f28f70fc700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.305950, event: all_read, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3847> 2014-11-03 11:37:54.306084 7f28f70fc700 5 -- op tracker -- seq: 3114, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3846> 2014-11-03 11:37:54.306187 7f290081c700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.306187, event: reached_pg, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3845> 2014-11-03 11:37:54.306243 7f290081c700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.306243, event: started, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3844> 2014-11-03 11:37:54.306343 7f290081c700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.306343, event: started, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3843> 2014-11-03 11:37:54.306326 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 52 ==== osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (3315517530 0 3511520646) 0x25ed6400 con 0x22ff3580 -3842> 2014-11-03 11:37:54.306371 7f28f17a3700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.306165, event: header_read, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3841> 2014-11-03 11:37:54.306390 7f290081c700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.306390, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3840> 2014-11-03 11:37:54.306402 7f28f17a3700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.306167, event: throttled, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3839> 2014-11-03 11:37:54.306431 7f28f17a3700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.306299, event: all_read, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3838> 2014-11-03 11:37:54.306457 7f28f17a3700 5 -- op tracker -- seq: 3115, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3837> 2014-11-03 11:37:54.306534 7f290001b700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.306534, event: reached_pg, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3836> 2014-11-03 11:37:54.306570 7f290001b700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.306570, event: started, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3835> 2014-11-03 11:37:54.306633 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 48 ==== osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2811504652 0 3511520646) 0x26152800 con 0x22fbc000 -3834> 2014-11-03 11:37:54.306679 7f290001b700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.306678, event: started, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3833> 2014-11-03 11:37:54.306681 7f28f71fd700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.306504, event: header_read, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3832> 2014-11-03 11:37:54.306711 7f290001b700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.306711, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3831> 2014-11-03 11:37:54.306722 7f28f71fd700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.306506, event: throttled, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3830> 2014-11-03 11:37:54.306745 7f28f71fd700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.306617, event: all_read, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3829> 2014-11-03 11:37:54.306779 7f28f71fd700 5 -- op tracker -- seq: 3116, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3828> 2014-11-03 11:37:54.306869 7f28fd816700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.306869, event: reached_pg, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3827> 2014-11-03 11:37:54.306919 7f28fd816700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.306919, event: started, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3826> 2014-11-03 11:37:54.307063 7f28fd816700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.307063, event: started, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3825> 2014-11-03 11:37:54.307109 7f28fd816700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.307108, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3824> 2014-11-03 11:37:54.307362 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 98 ==== osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (3938921146 0 3511520646) 0x2cff5400 con 0x22fbc580 -3823> 2014-11-03 11:37:54.307392 7f28f70fc700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.307260, event: header_read, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3822> 2014-11-03 11:37:54.307411 7f28f70fc700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.307262, event: throttled, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3821> 2014-11-03 11:37:54.307420 7f28f70fc700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.307348, event: all_read, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3820> 2014-11-03 11:37:54.307428 7f28f70fc700 5 -- op tracker -- seq: 3117, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3819> 2014-11-03 11:37:54.307489 7f28fe017700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.307489, event: reached_pg, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3818> 2014-11-03 11:37:54.307529 7f28fe017700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.307529, event: started, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3817> 2014-11-03 11:37:54.307644 7f28fe017700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.307644, event: started, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3816> 2014-11-03 11:37:54.307692 7f28fe017700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.307691, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3815> 2014-11-03 11:37:54.307861 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 49 ==== osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2255766630 0 3511520646) 0x2de0f400 con 0x22fbc000 -3814> 2014-11-03 11:37:54.307898 7f28f71fd700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.307714, event: header_read, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3813> 2014-11-03 11:37:54.307932 7f28f71fd700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.307716, event: throttled, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3812> 2014-11-03 11:37:54.307961 7f28f71fd700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.307844, event: all_read, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3811> 2014-11-03 11:37:54.307986 7f28f71fd700 5 -- op tracker -- seq: 3118, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3810> 2014-11-03 11:37:54.308141 7f290001b700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.308141, event: reached_pg, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3809> 2014-11-03 11:37:54.308156 7f28f17a3700 1 -- 192.168.5.2:6803/7606 <== osd.64 192.168.5.5:6848/24829 53 ==== osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (517696812 0 3511520646) 0x25ed4000 con 0x22ff3580 -3808> 2014-11-03 11:37:54.308175 7f290001b700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.308175, event: started, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3807> 2014-11-03 11:37:54.308205 7f28f17a3700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.308064, event: header_read, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3806> 2014-11-03 11:37:54.308229 7f28f17a3700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.308066, event: throttled, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3805> 2014-11-03 11:37:54.308239 7f28f17a3700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.308137, event: all_read, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3804> 2014-11-03 11:37:54.308251 7f28f17a3700 5 -- op tracker -- seq: 3119, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3803> 2014-11-03 11:37:54.308298 7f28fd816700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.308298, event: reached_pg, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3802> 2014-11-03 11:37:54.308303 7f290001b700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.308302, event: started, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3801> 2014-11-03 11:37:54.308322 7f28fd816700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.308322, event: started, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3800> 2014-11-03 11:37:54.308336 7f290001b700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.308336, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3799> 2014-11-03 11:37:54.308405 7f28fd816700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.308404, event: started, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3798> 2014-11-03 11:37:54.308434 7f28fd816700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.308434, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3797> 2014-11-03 11:37:54.308679 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 99 ==== osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (1848965220 0 3511520646) 0x26c42c00 con 0x22fbc580 -3796> 2014-11-03 11:37:54.308706 7f28f70fc700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.308579, event: header_read, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3795> 2014-11-03 11:37:54.308753 7f28f70fc700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.308581, event: throttled, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3794> 2014-11-03 11:37:54.308769 7f28f70fc700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.308666, event: all_read, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3793> 2014-11-03 11:37:54.308781 7f28f70fc700 5 -- op tracker -- seq: 3120, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3792> 2014-11-03 11:37:54.308833 7f290081c700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.308832, event: reached_pg, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3791> 2014-11-03 11:37:54.308858 7f290081c700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.308858, event: started, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3790> 2014-11-03 11:37:54.308960 7f290081c700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.308959, event: started, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3789> 2014-11-03 11:37:54.309004 7f290081c700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.309004, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3788> 2014-11-03 11:37:54.309343 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 50 ==== osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (226132783 0 3511520646) 0x2de0dc00 con 0x22fbc000 -3787> 2014-11-03 11:37:54.309384 7f28f71fd700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.309218, event: header_read, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3786> 2014-11-03 11:37:54.309403 7f28f71fd700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.309220, event: throttled, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3785> 2014-11-03 11:37:54.309423 7f28f71fd700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.309328, event: all_read, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3784> 2014-11-03 11:37:54.309442 7f28f71fd700 5 -- op tracker -- seq: 3121, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3783> 2014-11-03 11:37:54.309492 7f290001b700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.309491, event: reached_pg, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3782> 2014-11-03 11:37:54.309515 7f290001b700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.309515, event: started, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3781> 2014-11-03 11:37:54.309595 7f290001b700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.309595, event: started, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3780> 2014-11-03 11:37:54.309619 7f290001b700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.309619, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3779> 2014-11-03 11:37:54.311623 7f28f018d700 1 -- 192.168.5.2:6803/7606 <== osd.34 192.168.5.3:6833/2034 46 ==== osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (547101164 0 3511520646) 0x29e11c00 con 0x230a9c60 -3778> 2014-11-03 11:37:54.311660 7f28f018d700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.311477, event: header_read, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3777> 2014-11-03 11:37:54.311678 7f28f018d700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.311479, event: throttled, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3776> 2014-11-03 11:37:54.311695 7f28f018d700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.311608, event: all_read, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3775> 2014-11-03 11:37:54.311705 7f28f018d700 5 -- op tracker -- seq: 3122, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3774> 2014-11-03 11:37:54.311764 7f28ff81a700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.311763, event: reached_pg, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3773> 2014-11-03 11:37:54.311795 7f28ff81a700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.311795, event: started, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3772> 2014-11-03 11:37:54.311881 7f28ff81a700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.311881, event: started, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3771> 2014-11-03 11:37:54.311911 7f28ff81a700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.311911, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3770> 2014-11-03 11:37:54.314339 7f28f018d700 1 -- 192.168.5.2:6803/7606 <== osd.34 192.168.5.3:6833/2034 47 ==== osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (3457263005 0 3511520646) 0xd4a5c00 con 0x230a9c60 -3769> 2014-11-03 11:37:54.314371 7f28f018d700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.314210, event: header_read, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3768> 2014-11-03 11:37:54.314389 7f28f018d700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.314212, event: throttled, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3767> 2014-11-03 11:37:54.314401 7f28f018d700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.314325, event: all_read, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3766> 2014-11-03 11:37:54.314417 7f28f018d700 5 -- op tracker -- seq: 3123, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3765> 2014-11-03 11:37:54.314484 7f290201f700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.314484, event: reached_pg, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3764> 2014-11-03 11:37:54.314512 7f290201f700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.314512, event: started, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3763> 2014-11-03 11:37:54.314607 7f290201f700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.314607, event: started, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3762> 2014-11-03 11:37:54.314636 7f290201f700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.314636, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3761> 2014-11-03 11:37:54.315228 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 39 ==== MOSDPGPush(6.6ac 105950 [PushOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6, version: 105945'9805905, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6@105945'9805905, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6, version: 105945'9805907, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6@105945'9805907, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390497+0+0 (1228953816 0 0) 0x244ab000 con 0x2301f580 -3760> 2014-11-03 11:37:54.315277 7f28f2db9700 5 -- op tracker -- seq: 3124, time: 2014-11-03 11:37:54.282574, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6, version: 105945'9805905, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6@105945'9805905, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6, version: 105945'9805907, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6@105945'9805907, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3759> 2014-11-03 11:37:54.315303 7f28f2db9700 5 -- op tracker -- seq: 3124, time: 2014-11-03 11:37:54.282576, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6, version: 105945'9805905, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6@105945'9805905, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6, version: 105945'9805907, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6@105945'9805907, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3758> 2014-11-03 11:37:54.315325 7f28f2db9700 5 -- op tracker -- seq: 3124, time: 2014-11-03 11:37:54.315212, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6, version: 105945'9805905, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6@105945'9805905, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6, version: 105945'9805907, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6@105945'9805907, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3757> 2014-11-03 11:37:54.315346 7f28f2db9700 5 -- op tracker -- seq: 3124, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6, version: 105945'9805905, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6@105945'9805905, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6, version: 105945'9805907, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6@105945'9805907, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3756> 2014-11-03 11:37:54.315411 7f28fe017700 5 -- op tracker -- seq: 3124, time: 2014-11-03 11:37:54.315411, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6, version: 105945'9805905, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6@105945'9805905, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6, version: 105945'9805907, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6@105945'9805907, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3755> 2014-11-03 11:37:54.315653 7f28fe017700 5 -- op tracker -- seq: 3124, time: 2014-11-03 11:37:54.315653, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6, version: 105945'9805905, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6@105945'9805905, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6, version: 105945'9805907, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6@105945'9805907, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3754> 2014-11-03 11:37:54.316263 7f28f018d700 1 -- 192.168.5.2:6803/7606 <== osd.34 192.168.5.3:6833/2034 48 ==== osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2911941020 0 3511520646) 0x26dbac00 con 0x230a9c60 -3753> 2014-11-03 11:37:54.316292 7f28f018d700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.316184, event: header_read, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3752> 2014-11-03 11:37:54.316308 7f28f018d700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.316186, event: throttled, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3751> 2014-11-03 11:37:54.316328 7f28f018d700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.316251, event: all_read, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3750> 2014-11-03 11:37:54.316336 7f28f018d700 5 -- op tracker -- seq: 3125, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3749> 2014-11-03 11:37:54.316384 7f28ff81a700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.316384, event: reached_pg, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3748> 2014-11-03 11:37:54.316413 7f28ff81a700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.316413, event: started, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3747> 2014-11-03 11:37:54.316529 7f28ff81a700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.316529, event: started, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3746> 2014-11-03 11:37:54.316571 7f28ff81a700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.316571, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3745> 2014-11-03 11:37:54.320434 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.73 192.168.5.6:0/1260 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.319166) v2 ==== 47+0+0 (3988153949 0 0) 0x23fdfe00 con 0x23b1c6e0 -3744> 2014-11-03 11:37:54.320462 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.73 192.168.5.6:0/1260 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.319166) v2 ==== 47+0+0 (3988153949 0 0) 0x29884800 con 0x242c0dc0 -3743> 2014-11-03 11:37:54.320491 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/1260 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.319166) v2 -- ?+0 0x2a2e0e00 con 0x23b1c6e0 -3742> 2014-11-03 11:37:54.320547 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/1260 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.319166) v2 -- ?+0 0x24dce800 con 0x242c0dc0 -3741> 2014-11-03 11:37:54.344826 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 40 ==== MOSDPGPush(6.6ac 105950 [PushOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6, version: 105945'9805908, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6@105945'9805908, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6, version: 105945'9805913, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6@105945'9805913, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390497+0+0 (1126792956 0 0) 0x2350a800 con 0x2301f580 -3740> 2014-11-03 11:37:54.344885 7f28f2db9700 5 -- op tracker -- seq: 3126, time: 2014-11-03 11:37:54.315410, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6, version: 105945'9805908, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6@105945'9805908, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6, version: 105945'9805913, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6@105945'9805913, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3739> 2014-11-03 11:37:54.344914 7f28f2db9700 5 -- op tracker -- seq: 3126, time: 2014-11-03 11:37:54.315411, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6, version: 105945'9805908, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6@105945'9805908, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6, version: 105945'9805913, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6@105945'9805913, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3738> 2014-11-03 11:37:54.344934 7f28f2db9700 5 -- op tracker -- seq: 3126, time: 2014-11-03 11:37:54.344804, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6, version: 105945'9805908, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6@105945'9805908, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6, version: 105945'9805913, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6@105945'9805913, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3737> 2014-11-03 11:37:54.344973 7f28f2db9700 5 -- op tracker -- seq: 3126, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6, version: 105945'9805908, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6@105945'9805908, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6, version: 105945'9805913, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6@105945'9805913, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3736> 2014-11-03 11:37:54.345084 7f290081c700 5 -- op tracker -- seq: 3126, time: 2014-11-03 11:37:54.345083, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6, version: 105945'9805908, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6@105945'9805908, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6, version: 105945'9805913, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6@105945'9805913, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3735> 2014-11-03 11:37:54.345280 7f290081c700 5 -- op tracker -- seq: 3126, time: 2014-11-03 11:37:54.345280, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6, version: 105945'9805908, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6@105945'9805908, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6, version: 105945'9805913, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6@105945'9805913, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3734> 2014-11-03 11:37:54.359178 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 41 ==== MOSDPGPush(6.6ac 105950 [PushOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6, version: 105945'9805915, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6@105945'9805915, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1949954029 0 0) 0x23247000 con 0x2301f580 -3733> 2014-11-03 11:37:54.359229 7f28f2db9700 5 -- op tracker -- seq: 3127, time: 2014-11-03 11:37:54.345051, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6, version: 105945'9805915, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6@105945'9805915, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3732> 2014-11-03 11:37:54.359251 7f28f2db9700 5 -- op tracker -- seq: 3127, time: 2014-11-03 11:37:54.345053, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6, version: 105945'9805915, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6@105945'9805915, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3731> 2014-11-03 11:37:54.359264 7f28f2db9700 5 -- op tracker -- seq: 3127, time: 2014-11-03 11:37:54.359162, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6, version: 105945'9805915, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6@105945'9805915, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3730> 2014-11-03 11:37:54.359283 7f28f2db9700 5 -- op tracker -- seq: 3127, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6, version: 105945'9805915, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6@105945'9805915, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3729> 2014-11-03 11:37:54.359371 7f28fe017700 5 -- op tracker -- seq: 3127, time: 2014-11-03 11:37:54.359371, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6, version: 105945'9805915, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6@105945'9805915, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3728> 2014-11-03 11:37:54.359597 7f28fe017700 5 -- op tracker -- seq: 3127, time: 2014-11-03 11:37:54.359597, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6, version: 105945'9805915, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6@105945'9805915, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3727> 2014-11-03 11:37:54.369809 7f28ea531700 1 -- 192.168.5.2:6803/7606 <== osd.92 192.168.5.7:6810/5438 21 ==== osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1667146303 0 3511520646) 0x2d3a8800 con 0x231ab6e0 -3726> 2014-11-03 11:37:54.369843 7f28ea531700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.369696, event: header_read, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3725> 2014-11-03 11:37:54.369857 7f28ea531700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.369699, event: throttled, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3724> 2014-11-03 11:37:54.369867 7f28ea531700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.369794, event: all_read, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3723> 2014-11-03 11:37:54.369875 7f28ea531700 5 -- op tracker -- seq: 3128, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3722> 2014-11-03 11:37:54.369930 7f28fd816700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.369929, event: reached_pg, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3721> 2014-11-03 11:37:54.369964 7f28fd816700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.369964, event: started, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3720> 2014-11-03 11:37:54.370066 7f28fd816700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.370066, event: started, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3719> 2014-11-03 11:37:54.370098 7f28fd816700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.370098, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3718> 2014-11-03 11:37:54.372012 7f28ea531700 1 -- 192.168.5.2:6803/7606 <== osd.92 192.168.5.7:6810/5438 22 ==== osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (922572510 0 3511520646) 0x2d3a9400 con 0x231ab6e0 -3717> 2014-11-03 11:37:54.372045 7f28ea531700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.371917, event: header_read, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3716> 2014-11-03 11:37:54.372062 7f28ea531700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.371919, event: throttled, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3715> 2014-11-03 11:37:54.372074 7f28ea531700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.371998, event: all_read, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3714> 2014-11-03 11:37:54.372086 7f28ea531700 5 -- op tracker -- seq: 3129, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3713> 2014-11-03 11:37:54.372147 7f290001b700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.372147, event: reached_pg, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3712> 2014-11-03 11:37:54.372213 7f290001b700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.372213, event: started, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3711> 2014-11-03 11:37:54.372304 7f290001b700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.372304, event: started, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3710> 2014-11-03 11:37:54.372334 7f290001b700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.372334, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3709> 2014-11-03 11:37:54.377833 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.124 192.168.5.9:0/28288 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.376953) v2 ==== 47+0+0 (2627607069 0 0) 0x6023a00 con 0x31a9160 -3708> 2014-11-03 11:37:54.377881 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/28288 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.376953) v2 -- ?+0 0x23fdfe00 con 0x31a9160 -3707> 2014-11-03 11:37:54.377900 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.124 192.168.5.9:0/28288 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.376953) v2 ==== 47+0+0 (2627607069 0 0) 0x1a327000 con 0x2389f160 -3706> 2014-11-03 11:37:54.377984 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/28288 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.376953) v2 -- ?+0 0x29884800 con 0x2389f160 -3705> 2014-11-03 11:37:54.395778 7f28ea935700 1 -- 192.168.5.2:6803/7606 <== osd.101 192.168.5.8:6809/13676 24 ==== osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (4183523962 0 3511520646) 0x23fae000 con 0x23157000 -3704> 2014-11-03 11:37:54.395820 7f28ea935700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.395614, event: header_read, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3703> 2014-11-03 11:37:54.395845 7f28ea935700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.395617, event: throttled, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3702> 2014-11-03 11:37:54.395858 7f28ea935700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.395763, event: all_read, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3701> 2014-11-03 11:37:54.395871 7f28ea935700 5 -- op tracker -- seq: 3130, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3700> 2014-11-03 11:37:54.395937 7f28fd816700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.395936, event: reached_pg, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3699> 2014-11-03 11:37:54.395976 7f28fd816700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.395976, event: started, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3698> 2014-11-03 11:37:54.396078 7f28fd816700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.396078, event: started, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3697> 2014-11-03 11:37:54.396109 7f28fd816700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.396109, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3696> 2014-11-03 11:37:54.398293 7f28ea935700 1 -- 192.168.5.2:6803/7606 <== osd.101 192.168.5.8:6809/13676 25 ==== osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3248414133 0 3511520646) 0x23fb0400 con 0x23157000 -3695> 2014-11-03 11:37:54.398329 7f28ea935700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.398174, event: header_read, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3694> 2014-11-03 11:37:54.398348 7f28ea935700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.398176, event: throttled, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3693> 2014-11-03 11:37:54.398356 7f28ea935700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.398279, event: all_read, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3692> 2014-11-03 11:37:54.398364 7f28ea935700 5 -- op tracker -- seq: 3131, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3691> 2014-11-03 11:37:54.398417 7f290001b700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.398416, event: reached_pg, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3690> 2014-11-03 11:37:54.398450 7f290001b700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.398450, event: started, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3689> 2014-11-03 11:37:54.398547 7f290001b700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.398546, event: started, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3688> 2014-11-03 11:37:54.398575 7f290001b700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.398575, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3687> 2014-11-03 11:37:54.401222 7f291450b700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:54.401221, event: journaled_completion_queued, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -3686> 2014-11-03 11:37:54.401256 7f291450b700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:54.401256, event: journaled_completion_queued, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -3685> 2014-11-03 11:37:54.401287 7f291450b700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:54.401287, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -3684> 2014-11-03 11:37:54.401297 7f2911d06700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:54.401297, event: commit_sent, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -3683> 2014-11-03 11:37:54.401332 7f291450b700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:54.401332, event: journaled_completion_queued, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -3682> 2014-11-03 11:37:54.401353 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] ondisk, result = 0) v2 -- ?+0 0x263af080 con 0x22fb76e0 -3681> 2014-11-03 11:37:54.401409 7f2911d06700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:54.401409, event: op_commit, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -3680> 2014-11-03 11:37:54.401501 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.2:0/1027858 -- osd_op_reply(459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] v105950'131095 uv131095 ondisk = 0) v6 -- ?+0 0x23c1c840 con 0x2311c9a0 -3679> 2014-11-03 11:37:54.401550 7f2911d06700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:54.401550, event: commit_sent, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -3678> 2014-11-03 11:37:54.401585 7f2911d06700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:54.401585, event: commit_sent, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -3677> 2014-11-03 11:37:54.401609 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0x263aeb00 con 0x22fb76e0 -3676> 2014-11-03 11:37:54.401634 7f2911d06700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:54.401634, event: op_commit, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -3675> 2014-11-03 11:37:54.401681 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.69:0/1023508 -- osd_op_reply(220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] v105950'7950028 uv7950028 ondisk = 0) v6 -- ?+0 0x9fff580 con 0x23158b00 -3674> 2014-11-03 11:37:54.401721 7f2911d06700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:54.401720, event: commit_sent, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -3673> 2014-11-03 11:37:54.401881 7f2912507700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:54.401880, event: sub_op_applied, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -3672> 2014-11-03 11:37:54.401936 7f2912507700 5 -- op tracker -- seq: 3095, time: 2014-11-03 11:37:54.401936, event: done, op: osd_sub_op(client.7912219.0:1154963 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187824 snapset=0=[]:[] snapc=0=[]) -3671> 2014-11-03 11:37:54.402300 7f2912507700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:54.402300, event: op_applied, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -3670> 2014-11-03 11:37:54.402386 7f2912507700 5 -- op tracker -- seq: 3097, time: 2014-11-03 11:37:54.402386, event: done, op: osd_op(client.9598152.0:459475 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3088384~8192] 24.8f258d3e ack+ondisk+write e105950) -3669> 2014-11-03 11:37:54.402477 7f2912507700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:54.402477, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -3668> 2014-11-03 11:37:54.402508 7f2912507700 5 -- op tracker -- seq: 3100, time: 2014-11-03 11:37:54.402508, event: done, op: osd_sub_op(client.12404444.0:39608 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374242 snapset=0=[]:[] snapc=0=[]) -3667> 2014-11-03 11:37:54.402842 7f28e9326700 1 -- 192.168.5.2:6802/7606 <== client.9598152 192.168.9.2:0/1027858 12 ==== osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) v4 ==== 229+0+4096 (4257598571 0 2284057730) 0x32be000 con 0x2311c9a0 -3666> 2014-11-03 11:37:54.402919 7f28e9326700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.402734, event: header_read, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3665> 2014-11-03 11:37:54.402956 7f28e9326700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.402738, event: throttled, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3664> 2014-11-03 11:37:54.402973 7f28e9326700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.402827, event: all_read, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3663> 2014-11-03 11:37:54.402978 7f2914d0c700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.402978, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3662> 2014-11-03 11:37:54.402993 7f28e9326700 5 -- op tracker -- seq: 3132, time: 0.000000, event: dispatched, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3661> 2014-11-03 11:37:54.403035 7f2912507700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:54.403035, event: op_applied, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -3660> 2014-11-03 11:37:54.403064 7f2914d0c700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.403064, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3659> 2014-11-03 11:37:54.403131 7f2912507700 5 -- op tracker -- seq: 3101, time: 2014-11-03 11:37:54.403131, event: done, op: osd_op(client.9732651.0:220595 rbd_data.95169218ab737.0000000000000acd [stat,set-alloc-hint object_size 4194304 write_size 4194304,write 741376~131072] 6.f6f4e9d8 ack+ondisk+write e105950) -3658> 2014-11-03 11:37:54.403243 7f290101d700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.403242, event: reached_pg, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3657> 2014-11-03 11:37:54.403360 7f290101d700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.403360, event: started, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3656> 2014-11-03 11:37:54.403386 7f290101d700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.403386, event: started, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3655> 2014-11-03 11:37:54.403459 7f290101d700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.403459, event: waiting for subops from 45,107, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3654> 2014-11-03 11:37:54.403509 7f290101d700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- osd_sub_op(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131096 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4925 0x2493e400 con 0x22ff3420 -3653> 2014-11-03 11:37:54.403600 7f290101d700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6833/17474 -- osd_sub_op(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] v 105950'131096 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4925 0xd55f400 con 0x22fe0dc0 -3652> 2014-11-03 11:37:54.403764 7f290101d700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.403764, event: commit_queued_for_journal_write, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3651> 2014-11-03 11:37:54.405676 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 39 ==== osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (877424885 0 0) 0xa23f340 con 0x22ff3420 -3650> 2014-11-03 11:37:54.405714 7f28f0b97700 5 -- op tracker -- seq: 3133, time: 2014-11-03 11:37:54.405570, event: header_read, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3649> 2014-11-03 11:37:54.405731 7f28f0b97700 5 -- op tracker -- seq: 3133, time: 2014-11-03 11:37:54.405571, event: throttled, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3648> 2014-11-03 11:37:54.405741 7f28f0b97700 5 -- op tracker -- seq: 3133, time: 2014-11-03 11:37:54.405667, event: all_read, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3647> 2014-11-03 11:37:54.405752 7f28f0b97700 5 -- op tracker -- seq: 3133, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3646> 2014-11-03 11:37:54.405800 7f28fe818700 5 -- op tracker -- seq: 3133, time: 2014-11-03 11:37:54.405800, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3645> 2014-11-03 11:37:54.405797 7f28f52de700 1 -- 192.168.5.2:6803/7606 <== osd.107 192.168.5.8:6833/17474 35 ==== osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (1395768490 0 0) 0xa23f600 con 0x22fe0dc0 -3644> 2014-11-03 11:37:54.405835 7f28fe818700 5 -- op tracker -- seq: 3133, time: 2014-11-03 11:37:54.405835, event: started, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3643> 2014-11-03 11:37:54.405838 7f28f52de700 5 -- op tracker -- seq: 3134, time: 2014-11-03 11:37:54.405680, event: header_read, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3642> 2014-11-03 11:37:54.405863 7f28fe818700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.405863, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3641> 2014-11-03 11:37:54.405905 7f28f52de700 5 -- op tracker -- seq: 3134, time: 2014-11-03 11:37:54.405681, event: throttled, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3640> 2014-11-03 11:37:54.405919 7f28f52de700 5 -- op tracker -- seq: 3134, time: 2014-11-03 11:37:54.405776, event: all_read, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3639> 2014-11-03 11:37:54.405924 7f28fe818700 5 -- op tracker -- seq: 3133, time: 2014-11-03 11:37:54.405924, event: done, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3638> 2014-11-03 11:37:54.405936 7f28f52de700 5 -- op tracker -- seq: 3134, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3637> 2014-11-03 11:37:54.406048 7f290101d700 5 -- op tracker -- seq: 3134, time: 2014-11-03 11:37:54.406048, event: reached_pg, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3636> 2014-11-03 11:37:54.406087 7f290101d700 5 -- op tracker -- seq: 3134, time: 2014-11-03 11:37:54.406087, event: started, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3635> 2014-11-03 11:37:54.406101 7f290101d700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.406101, event: sub_op_commit_rec, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3634> 2014-11-03 11:37:54.406127 7f290101d700 5 -- op tracker -- seq: 3134, time: 2014-11-03 11:37:54.406127, event: done, op: osd_sub_op_reply(client.9598152.0:459476 24.53e 8f258d3e/rbd_data.92d20c2ae8944a.0000000000000084/head//24 [] ondisk, result = 0) -3633> 2014-11-03 11:37:54.427209 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 42 ==== MOSDPGPush(6.6ac 105950 [PushOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6, version: 105946'9805965, data_included: [0~3473408], data_size: 3473408, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6@105946'9805965, copy_subset: [0~3473408], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3473408, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6, version: 105946'9805983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6@105946'9805983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6, version: 105946'9805998, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6@105946'9805998, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 11864841+0+0 (2668501570 0 0) 0x2a347400 con 0x2301f580 -3632> 2014-11-03 11:37:54.427273 7f28f2db9700 5 -- op tracker -- seq: 3135, time: 2014-11-03 11:37:54.383864, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6, version: 105946'9805965, data_included: [0~3473408], data_size: 3473408, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6@105946'9805965, copy_subset: [0~3473408], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3473408, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6, version: 105946'9805983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6@105946'9805983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6, version: 105946'9805998, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6@105946'9805998, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3631> 2014-11-03 11:37:54.427375 7f28f2db9700 5 -- op tracker -- seq: 3135, time: 2014-11-03 11:37:54.383867, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6, version: 105946'9805965, data_included: [0~3473408], data_size: 3473408, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6@105946'9805965, copy_subset: [0~3473408], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3473408, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6, version: 105946'9805983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6@105946'9805983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6, version: 105946'9805998, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6@105946'9805998, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3630> 2014-11-03 11:37:54.427428 7f28f2db9700 5 -- op tracker -- seq: 3135, time: 2014-11-03 11:37:54.427190, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6, version: 105946'9805965, data_included: [0~3473408], data_size: 3473408, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6@105946'9805965, copy_subset: [0~3473408], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3473408, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6, version: 105946'9805983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6@105946'9805983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6, version: 105946'9805998, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6@105946'9805998, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3629> 2014-11-03 11:37:54.427457 7f28f2db9700 5 -- op tracker -- seq: 3135, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6, version: 105946'9805965, data_included: [0~3473408], data_size: 3473408, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6@105946'9805965, copy_subset: [0~3473408], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3473408, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6, version: 105946'9805983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6@105946'9805983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6, version: 105946'9805998, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6@105946'9805998, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3628> 2014-11-03 11:37:54.427576 7f290081c700 5 -- op tracker -- seq: 3135, time: 2014-11-03 11:37:54.427576, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6, version: 105946'9805965, data_included: [0~3473408], data_size: 3473408, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6@105946'9805965, copy_subset: [0~3473408], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3473408, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6, version: 105946'9805983, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6@105946'9805983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6, version: 105946'9805998, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6@105946'9805998, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3627> 2014-11-03 11:37:54.427958 7f290081c700 5 -- op tracker -- seq: 3135, time: 2014-11-03 11:37:54.427958, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6, version: 105946'9805965, data_included: [0~3473408], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6@105946'9805965, copy_subset: [0~3473408], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3473408, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6, version: 105946'9805983, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6@105946'9805983, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6, version: 105946'9805998, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6@105946'9805998, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3626> 2014-11-03 11:37:54.460510 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31524840 sd=78 :6802 s=0 pgs=0 cs=0 l=0 c=0xa204580).accept sd=78 192.168.5.2:42958/0 -3625> 2014-11-03 11:37:54.461293 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450794 192.168.5.2:0/1003395 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x26a9f000 con 0xa204580 -3624> 2014-11-03 11:37:54.461854 7f28fb812700 0 osd.21 105950 do_command r=0 -3623> 2014-11-03 11:37:54.461880 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003395 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x968f1c0 con 0xa204580 -3622> 2014-11-03 11:37:54.462454 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 43 ==== MOSDPGPush(6.6ac 105950 [PushOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6, version: 105946'9806001, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6@105946'9806001, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6, version: 105946'9806036, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6@105946'9806036, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (1984871055 0 0) 0x2340c000 con 0x2301f580 -3621> 2014-11-03 11:37:54.462516 7f28f2db9700 5 -- op tracker -- seq: 3136, time: 2014-11-03 11:37:54.427549, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6, version: 105946'9806001, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6@105946'9806001, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6, version: 105946'9806036, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6@105946'9806036, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3620> 2014-11-03 11:37:54.462565 7f28f2db9700 5 -- op tracker -- seq: 3136, time: 2014-11-03 11:37:54.427550, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6, version: 105946'9806001, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6@105946'9806001, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6, version: 105946'9806036, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6@105946'9806036, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3619> 2014-11-03 11:37:54.462606 7f28f2db9700 5 -- op tracker -- seq: 3136, time: 2014-11-03 11:37:54.462435, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6, version: 105946'9806001, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6@105946'9806001, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6, version: 105946'9806036, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6@105946'9806036, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3618> 2014-11-03 11:37:54.462638 7f28f2db9700 5 -- op tracker -- seq: 3136, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6, version: 105946'9806001, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6@105946'9806001, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6, version: 105946'9806036, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6@105946'9806036, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3617> 2014-11-03 11:37:54.462723 7f28fe017700 5 -- op tracker -- seq: 3136, time: 2014-11-03 11:37:54.462723, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6, version: 105946'9806001, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6@105946'9806001, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6, version: 105946'9806036, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6@105946'9806036, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3616> 2014-11-03 11:37:54.462953 7f28fe017700 5 -- op tracker -- seq: 3136, time: 2014-11-03 11:37:54.462953, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6, version: 105946'9806001, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6@105946'9806001, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6, version: 105946'9806036, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6@105946'9806036, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3615> 2014-11-03 11:37:54.463919 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- MRecoveryReserve RELEASE pgid: 24.a30, query_epoch: 105950 v2 -- ?+0 0x23576400 con 0x23184000 -3614> 2014-11-03 11:37:54.463957 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- MRecoveryReserve RELEASE pgid: 24.a30, query_epoch: 105950 v2 -- ?+0 0x2340d000 con 0x23000b00 -3613> 2014-11-03 11:37:54.463984 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 mlcod 0'0 active+degraded snaptrimq=[2e8~1,2fe~2,302~4,307~1]] exit Started/Primary/Active/Recovering 0.656865 1 0.000114 -3612> 2014-11-03 11:37:54.464022 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 mlcod 0'0 active+degraded snaptrimq=[2e8~1,2fe~2,302~4,307~1]] enter Started/Primary/Active/Recovered -3611> 2014-11-03 11:37:54.464064 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 mlcod 0'0 active snaptrimq=[2e8~1,2fe~2,302~4,307~1]] exit Started/Primary/Active/Recovered 0.000042 0 0.000000 -3610> 2014-11-03 11:37:54.464085 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.a30( v 105946'65847 (91663'61444,105946'65847] local-les=105948 n=89 ec=25000 les/c 105948/105943 105947/105947/105947) [21,97,66] r=0 lpr=105947 pi=105933-105946/4 crt=105946'65847 mlcod 0'0 active snaptrimq=[2e8~1,2fe~2,302~4,307~1]] enter Started/Primary/Active/Clean -3609> 2014-11-03 11:37:54.464132 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 lc 105913'5299 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1] exit Started/Primary/Active/WaitLocalRecoveryReserved 26.507931 2 0.000121 -3608> 2014-11-03 11:37:54.464187 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 lc 105913'5299 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1] enter Started/Primary/Active/WaitRemoteRecoveryReserved -3607> 2014-11-03 11:37:54.464206 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2340d200 con 0x23184000 -3606> 2014-11-03 11:37:54.464223 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- MRecoveryReserve REQUEST pgid: 26.4bd, query_epoch: 105950 v2 -- ?+0 0x252d7600 con 0x22fe0c60 -3605> 2014-11-03 11:37:54.464354 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2c390000 con 0x23000b00 -3604> 2014-11-03 11:37:54.464761 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2a6e8a00 con 0x23184000 -3603> 2014-11-03 11:37:54.464798 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2318a600 con 0x23000b00 -3602> 2014-11-03 11:37:54.465101 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x24dcf000 con 0x23184000 -3601> 2014-11-03 11:37:54.465199 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2782fc00 con 0x23000b00 -3600> 2014-11-03 11:37:54.465203 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 112 ==== MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 v2 ==== 26+0+0 (529053550 0 0) 0x252d7600 con 0x22fe0c60 -3599> 2014-11-03 11:37:54.465235 7f290b031700 5 -- op tracker -- seq: 3137, time: 2014-11-03 11:37:54.465118, event: header_read, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3598> 2014-11-03 11:37:54.465259 7f290b031700 5 -- op tracker -- seq: 3137, time: 2014-11-03 11:37:54.465119, event: throttled, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3597> 2014-11-03 11:37:54.465275 7f290b031700 5 -- op tracker -- seq: 3137, time: 2014-11-03 11:37:54.465164, event: all_read, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3596> 2014-11-03 11:37:54.465291 7f290b031700 5 -- op tracker -- seq: 3137, time: 2014-11-03 11:37:54.465228, event: dispatched, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3595> 2014-11-03 11:37:54.465307 7f290b031700 5 -- op tracker -- seq: 3137, time: 2014-11-03 11:37:54.465307, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3594> 2014-11-03 11:37:54.465351 7f290b031700 5 -- op tracker -- seq: 3137, time: 2014-11-03 11:37:54.465351, event: done, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3593> 2014-11-03 11:37:54.465386 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6816/27523 -- MRecoveryReserve REQUEST pgid: 26.4bd, query_epoch: 105950 v2 -- ?+0 0x252d7000 con 0x230c2160 -3592> 2014-11-03 11:37:54.465472 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x23a9e000 con 0x23184000 -3591> 2014-11-03 11:37:54.465578 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x23a9ee00 con 0x23000b00 -3590> 2014-11-03 11:37:54.465830 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x29884800 con 0x23184000 -3589> 2014-11-03 11:37:54.465885 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x3501000 con 0x23000b00 -3588> 2014-11-03 11:37:54.466331 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x23a92800 con 0x23184000 -3587> 2014-11-03 11:37:54.466428 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x24dcfc00 con 0x23000b00 -3586> 2014-11-03 11:37:54.466666 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.44 192.168.5.4:6816/27523 10 ==== MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 v2 ==== 26+0+0 (529053550 0 0) 0x252d7000 con 0x230c2160 -3585> 2014-11-03 11:37:54.466693 7f290b031700 5 -- op tracker -- seq: 3138, time: 2014-11-03 11:37:54.466583, event: header_read, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3584> 2014-11-03 11:37:54.466707 7f290b031700 5 -- op tracker -- seq: 3138, time: 2014-11-03 11:37:54.466584, event: throttled, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3583> 2014-11-03 11:37:54.466716 7f290b031700 5 -- op tracker -- seq: 3138, time: 2014-11-03 11:37:54.466636, event: all_read, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3582> 2014-11-03 11:37:54.466723 7f290b031700 5 -- op tracker -- seq: 3138, time: 2014-11-03 11:37:54.466689, event: dispatched, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3581> 2014-11-03 11:37:54.466729 7f290b031700 5 -- op tracker -- seq: 3138, time: 2014-11-03 11:37:54.466729, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3580> 2014-11-03 11:37:54.466753 7f290b031700 5 -- op tracker -- seq: 3138, time: 2014-11-03 11:37:54.466753, event: done, op: MRecoveryReserve GRANT pgid: 26.4bd, query_epoch: 105950 -3579> 2014-11-03 11:37:54.466788 7f2902820700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 lc 105913'5299 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.002600 2 0.000163 -3578> 2014-11-03 11:37:54.466812 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x24dcf200 con 0x23184000 -3577> 2014-11-03 11:37:54.466831 7f2902820700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 lc 105913'5299 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1] enter Started/Primary/Active/Recovering -3576> 2014-11-03 11:37:54.466858 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2324b200 con 0x23000b00 -3575> 2014-11-03 11:37:54.466920 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6816/27523 -- MOSDPGPull(26.4bd 105950 [PullOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x2438f000 con 0x230c2160 -3574> 2014-11-03 11:37:54.467220 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2324be00 con 0x23184000 -3573> 2014-11-03 11:37:54.467349 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2324bc00 con 0x23000b00 -3572> 2014-11-03 11:37:54.467621 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6852/919 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x2324b800 con 0x23184000 -3571> 2014-11-03 11:37:54.467681 7f28fc013700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- pg_info(1 pgs e105950:24.a30) v4 -- ?+0 0x22fd4200 con 0x23000b00 -3570> 2014-11-03 11:37:54.471053 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450794 192.168.5.2:0/1003395 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x852f800 con 0xa204580 -3569> 2014-11-03 11:37:54.471835 7f28fb812700 0 osd.21 105950 do_command r=0 -3568> 2014-11-03 11:37:54.471852 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003395 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x3684e00 con 0xa204580 -3567> 2014-11-03 11:37:54.476740 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.105 192.168.5.8:0/6777 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.475566) v2 ==== 47+0+0 (3013185595 0 0) 0x29fcd400 con 0x2389f2c0 -3566> 2014-11-03 11:37:54.476786 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/6777 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.475566) v2 -- ?+0 0x1a327000 con 0x2389f2c0 -3565> 2014-11-03 11:37:54.477768 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.105 192.168.5.8:0/6777 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.475566) v2 ==== 47+0+0 (3013185595 0 0) 0x1a327200 con 0x31a92c0 -3564> 2014-11-03 11:37:54.477804 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/6777 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.475566) v2 -- ?+0 0x6023a00 con 0x31a92c0 -3563> 2014-11-03 11:37:54.478714 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003395 pipe(0x31524840 sd=78 :6802 s=2 pgs=2 cs=1 l=1 c=0xa204580).reader couldn't read tag, (0) Success -3562> 2014-11-03 11:37:54.478756 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003395 pipe(0x31524840 sd=78 :6802 s=2 pgs=2 cs=1 l=1 c=0xa204580).fault (0) Success -3561> 2014-11-03 11:37:54.478888 7f290c834700 1 osd.21 105950 ms_handle_reset con 0xa204580 session 0x2d281280 -3560> 2014-11-03 11:37:54.499533 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 174 ==== osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3433625498 0 3511520646) 0x24662800 con 0x22fefdc0 -3559> 2014-11-03 11:37:54.499589 7f28f36c2700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.499411, event: header_read, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3558> 2014-11-03 11:37:54.499614 7f28f36c2700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.499413, event: throttled, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3557> 2014-11-03 11:37:54.499633 7f28f36c2700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.499512, event: all_read, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3556> 2014-11-03 11:37:54.499650 7f28f36c2700 5 -- op tracker -- seq: 3139, time: 0.000000, event: dispatched, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3555> 2014-11-03 11:37:54.499742 7f28fe818700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.499742, event: reached_pg, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3554> 2014-11-03 11:37:54.499773 7f28fe818700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.499773, event: started, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3553> 2014-11-03 11:37:54.499957 7f28fe818700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.499956, event: started, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3552> 2014-11-03 11:37:54.500011 7f28fe818700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.500011, event: commit_queued_for_journal_write, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3551> 2014-11-03 11:37:54.509214 7f28ebd49700 1 -- 192.168.5.2:6803/7606 <== osd.44 192.168.5.4:6816/27523 11 ==== MOSDPGPush(26.4bd 105950 [PushOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, version: 105946'5300, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (3115703390 0 0) 0x2438f000 con 0x230c2160 -3550> 2014-11-03 11:37:54.509278 7f28ebd49700 5 -- op tracker -- seq: 3140, time: 2014-11-03 11:37:54.497252, event: header_read, op: MOSDPGPush(26.4bd 105950 [PushOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, version: 105946'5300, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3549> 2014-11-03 11:37:54.509324 7f28ebd49700 5 -- op tracker -- seq: 3140, time: 2014-11-03 11:37:54.497254, event: throttled, op: MOSDPGPush(26.4bd 105950 [PushOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, version: 105946'5300, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3548> 2014-11-03 11:37:54.509338 7f28ebd49700 5 -- op tracker -- seq: 3140, time: 2014-11-03 11:37:54.509197, event: all_read, op: MOSDPGPush(26.4bd 105950 [PushOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, version: 105946'5300, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3547> 2014-11-03 11:37:54.509367 7f28ebd49700 5 -- op tracker -- seq: 3140, time: 0.000000, event: dispatched, op: MOSDPGPush(26.4bd 105950 [PushOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, version: 105946'5300, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3546> 2014-11-03 11:37:54.509430 7f290081c700 5 -- op tracker -- seq: 3140, time: 2014-11-03 11:37:54.509430, event: reached_pg, op: MOSDPGPush(26.4bd 105950 [PushOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, version: 105946'5300, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3545> 2014-11-03 11:37:54.509629 7f290081c700 5 -- op tracker -- seq: 3140, time: 2014-11-03 11:37:54.509629, event: done, op: MOSDPGPush(26.4bd 105950 [PushOp(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26, version: 105946'5300, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(33e58cbd/rbd_data.4626792ae8944a.0000000000000655/head//26@105946'5300, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3544> 2014-11-03 11:37:54.533521 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x3110e600 sd=66 :6802 s=0 pgs=0 cs=0 l=0 c=0x154dd580).accept sd=66 192.168.5.201:58692/0 -3543> 2014-11-03 11:37:54.534384 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419952 192.168.5.201:0/1029777 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2893de00 con 0x154dd580 -3542> 2014-11-03 11:37:54.535040 7f28fb812700 0 osd.21 105950 do_command r=0 -3541> 2014-11-03 11:37:54.535078 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029777 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xfc73700 con 0x154dd580 -3540> 2014-11-03 11:37:54.541638 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419952 192.168.5.201:0/1029777 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2796ca00 con 0x154dd580 -3539> 2014-11-03 11:37:54.542780 7f28fb812700 0 osd.21 105950 do_command r=0 -3538> 2014-11-03 11:37:54.542800 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029777 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x6ef0700 con 0x154dd580 -3537> 2014-11-03 11:37:54.547381 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029777 pipe(0x3110e600 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x154dd580).reader couldn't read tag, (0) Success -3536> 2014-11-03 11:37:54.547417 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029777 pipe(0x3110e600 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x154dd580).fault (0) Success -3535> 2014-11-03 11:37:54.547504 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x154dd580 session 0x7c35a00 -3534> 2014-11-03 11:37:54.570220 7f291450b700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.570220, event: journaled_completion_queued, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3533> 2014-11-03 11:37:54.570329 7f291450b700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.570329, event: journaled_completion_queued, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3532> 2014-11-03 11:37:54.570383 7f2911d06700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.570383, event: commit_sent, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3531> 2014-11-03 11:37:54.570425 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] ondisk, result = 0) v2 -- ?+0 0x29059840 con 0x22fe0840 -3530> 2014-11-03 11:37:54.570439 7f2914d0c700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.570439, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3529> 2014-11-03 11:37:54.570454 7f2911d06700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.570454, event: commit_sent, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3528> 2014-11-03 11:37:54.570504 7f2914d0c700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.570503, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3527> 2014-11-03 11:37:54.570567 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(7ff036ac/rbd_data.89cd895a911542.0000000000009669/head//6),PushReplyOp(3e0a66ac/rbd_data.9039d22ae8944a.00000000000000af/head//6)]) v2 -- ?+0 0x2340c800 con 0x2301f580 -3526> 2014-11-03 11:37:54.570574 7f2914d0c700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.570574, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3525> 2014-11-03 11:37:54.570634 7f2914d0c700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.570634, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3524> 2014-11-03 11:37:54.570669 7f2914d0c700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.570669, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3523> 2014-11-03 11:37:54.570680 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] ondisk, result = 0) v2 -- ?+0 0x29059b00 con 0x22fe0840 -3522> 2014-11-03 11:37:54.570712 7f2914d0c700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.570712, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3521> 2014-11-03 11:37:54.570876 7f2914d0c700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.570876, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3520> 2014-11-03 11:37:54.570922 7f2914d0c700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.570922, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3519> 2014-11-03 11:37:54.570947 7f2914d0c700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.570947, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3518> 2014-11-03 11:37:54.570979 7f2914d0c700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.570979, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3517> 2014-11-03 11:37:54.571009 7f2914d0c700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.571009, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3516> 2014-11-03 11:37:54.571034 7f2914d0c700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.571034, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3515> 2014-11-03 11:37:54.571065 7f2914d0c700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.571065, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3514> 2014-11-03 11:37:54.571096 7f2914d0c700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.571096, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3513> 2014-11-03 11:37:54.571117 7f2914d0c700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.571117, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3512> 2014-11-03 11:37:54.573123 7f2914d0c700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.573123, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3511> 2014-11-03 11:37:54.582504 7f28f3eca700 1 -- 192.168.5.2:6803/7606 <== osd.83 192.168.5.6:6871/4005 40 ==== osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (964668894 0 3511520646) 0xd4b6c00 con 0x22fef2c0 -3510> 2014-11-03 11:37:54.582556 7f28f3eca700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.582329, event: header_read, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3509> 2014-11-03 11:37:54.582591 7f28f3eca700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.582333, event: throttled, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3508> 2014-11-03 11:37:54.582615 7f28f3eca700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.582485, event: all_read, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3507> 2014-11-03 11:37:54.582632 7f28f3eca700 5 -- op tracker -- seq: 3141, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3506> 2014-11-03 11:37:54.582734 7f28fe017700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.582733, event: reached_pg, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3505> 2014-11-03 11:37:54.582765 7f28fe017700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.582765, event: started, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3504> 2014-11-03 11:37:54.582879 7f28fe017700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.582879, event: started, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3503> 2014-11-03 11:37:54.582929 7f28fe017700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.582929, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -3502> 2014-11-03 11:37:54.596213 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.71 192.168.5.6:0/14378 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.595262) v2 ==== 47+0+0 (3122384734 0 0) 0x2c015c00 con 0x233352c0 -3501> 2014-11-03 11:37:54.596287 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.595262) v2 -- ?+0 0x1a327200 con 0x233352c0 -3500> 2014-11-03 11:37:54.596400 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.71 192.168.5.6:0/14378 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.595262) v2 ==== 47+0+0 (3122384734 0 0) 0x27b01400 con 0x234ed2c0 -3499> 2014-11-03 11:37:54.596473 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.595262) v2 -- ?+0 0x29fcd400 con 0x234ed2c0 -3498> 2014-11-03 11:37:54.603952 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x269b0b80 sd=66 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bc160).accept sd=66 192.168.5.2:42962/0 -3497> 2014-11-03 11:37:54.603987 7f2912507700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.603987, event: sub_op_applied, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3496> 2014-11-03 11:37:54.604068 7f2912507700 5 -- op tracker -- seq: 3106, time: 2014-11-03 11:37:54.604068, event: done, op: osd_sub_op(client.7868807.0:3345548 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960083 snapset=0=[]:[] snapc=0=[]) -3495> 2014-11-03 11:37:54.604452 7f2912507700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.604451, event: sub_op_applied, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3494> 2014-11-03 11:37:54.604533 7f2912507700 5 -- op tracker -- seq: 3107, time: 2014-11-03 11:37:54.604533, event: done, op: osd_sub_op(client.7868807.0:3345549 6.35a 97e97b5a/rbd_data.663cf85ce6fabc.0000000000000001/head//6 [] v 105950'14960084 snapset=0=[]:[] snapc=0=[]) -3493> 2014-11-03 11:37:54.604884 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450797 192.168.5.2:0/1003534 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x22fd4400 con 0x1c9bc160 -3492> 2014-11-03 11:37:54.605534 7f28fb812700 0 osd.21 105950 do_command r=0 -3491> 2014-11-03 11:37:54.605563 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003534 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x6ef0e00 con 0x1c9bc160 -3490> 2014-11-03 11:37:54.616691 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450797 192.168.5.2:0/1003534 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x29fcd400 con 0x1c9bc160 -3489> 2014-11-03 11:37:54.617876 7f28fb812700 0 osd.21 105950 do_command r=0 -3488> 2014-11-03 11:37:54.617905 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003534 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x6635380 con 0x1c9bc160 -3487> 2014-11-03 11:37:54.618585 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.60 192.168.5.5:0/5983 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.616900) v2 ==== 47+0+0 (4188070284 0 0) 0x2a7c4800 con 0x2445c9a0 -3486> 2014-11-03 11:37:54.618641 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.60 192.168.5.5:0/5983 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.616900) v2 ==== 47+0+0 (4188070284 0 0) 0x26bdba00 con 0x23ea4b00 -3485> 2014-11-03 11:37:54.618663 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.616900) v2 -- ?+0 0x2c015c00 con 0x2445c9a0 -3484> 2014-11-03 11:37:54.618788 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/5983 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.616900) v2 -- ?+0 0x27b01400 con 0x23ea4b00 -3483> 2014-11-03 11:37:54.623922 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003534 pipe(0x269b0b80 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc160).reader couldn't read tag, (0) Success -3482> 2014-11-03 11:37:54.624002 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003534 pipe(0x269b0b80 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc160).fault (0) Success -3481> 2014-11-03 11:37:54.624100 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bc160 session 0x7c35a00 -3480> 2014-11-03 11:37:54.632884 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 43 ==== osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2759509923 0 3511520646) 0x27624400 con 0x230c2c60 -3479> 2014-11-03 11:37:54.632968 7f28ee16d700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.632653, event: header_read, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3478> 2014-11-03 11:37:54.632998 7f28ee16d700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.632658, event: throttled, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3477> 2014-11-03 11:37:54.633043 7f28ee16d700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.632858, event: all_read, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3476> 2014-11-03 11:37:54.633071 7f28ee16d700 5 -- op tracker -- seq: 3142, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3475> 2014-11-03 11:37:54.633196 7f28fd816700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.633195, event: reached_pg, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3474> 2014-11-03 11:37:54.633253 7f28fd816700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.633253, event: started, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3473> 2014-11-03 11:37:54.633404 7f28fd816700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.633404, event: started, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3472> 2014-11-03 11:37:54.633455 7f28fd816700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.633455, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -3471> 2014-11-03 11:37:54.647377 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x23dfa340 sd=66 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bc2c0).accept sd=66 192.168.5.201:58695/0 -3470> 2014-11-03 11:37:54.648417 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450800 192.168.5.201:0/1029816 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.58e"}) v1 ==== 79+0+0 (3567090102 0 0) 0x23307200 con 0x1c9bc2c0 -3469> 2014-11-03 11:37:54.649008 7f28fb812700 0 osd.21 105950 do_command r=0 -3468> 2014-11-03 11:37:54.649044 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029816 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x6ef0e00 con 0x1c9bc2c0 -3467> 2014-11-03 11:37:54.658671 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450800 192.168.5.201:0/1029816 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.58e"}) v1 ==== 73+0+0 (125580214 0 0) 0x2c390800 con 0x1c9bc2c0 -3466> 2014-11-03 11:37:54.659767 7f28fb812700 0 osd.21 105950 do_command r=0 -3465> 2014-11-03 11:37:54.659789 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029816 -- command_reply(tid 2: 0 ) v1 -- ?+14243 0x6635380 con 0x1c9bc2c0 -3464> 2014-11-03 11:37:54.665289 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029816 pipe(0x23dfa340 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc2c0).reader couldn't read tag, (0) Success -3463> 2014-11-03 11:37:54.665336 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029816 pipe(0x23dfa340 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc2c0).fault (0) Success -3462> 2014-11-03 11:37:54.665413 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bc2c0 session 0x7c35a00 -3461> 2014-11-03 11:37:54.680338 7f291450b700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.680338, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3460> 2014-11-03 11:37:54.680396 7f291450b700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.680396, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3459> 2014-11-03 11:37:54.680412 7f2911d06700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.680412, event: commit_sent, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3458> 2014-11-03 11:37:54.680430 7f291450b700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.680430, event: journaled_completion_queued, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3457> 2014-11-03 11:37:54.680454 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x290592c0 con 0x22fe0c60 -3456> 2014-11-03 11:37:54.680455 7f291450b700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.680455, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3455> 2014-11-03 11:37:54.680478 7f291450b700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.680477, event: journaled_completion_queued, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3454> 2014-11-03 11:37:54.680485 7f2911d06700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.680485, event: commit_sent, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3453> 2014-11-03 11:37:54.680519 7f291450b700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.680519, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3452> 2014-11-03 11:37:54.680549 7f291450b700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.680549, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3451> 2014-11-03 11:37:54.680575 7f291450b700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.680574, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3450> 2014-11-03 11:37:54.680602 7f291450b700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.680602, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3449> 2014-11-03 11:37:54.680613 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650d2c0 con 0x22fe0c60 -3448> 2014-11-03 11:37:54.680626 7f291450b700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.680626, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3447> 2014-11-03 11:37:54.680650 7f291450b700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.680649, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3446> 2014-11-03 11:37:54.680651 7f2911d06700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.680651, event: commit_sent, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3445> 2014-11-03 11:37:54.680685 7f291450b700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.680685, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3444> 2014-11-03 11:37:54.680710 7f291450b700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.680709, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3443> 2014-11-03 11:37:54.680726 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650db00 con 0x22fe0c60 -3442> 2014-11-03 11:37:54.680733 7f291450b700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.680733, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3441> 2014-11-03 11:37:54.680754 7f291450b700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.680754, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3440> 2014-11-03 11:37:54.680756 7f2911d06700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.680756, event: commit_sent, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3439> 2014-11-03 11:37:54.680787 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2650ddc0 con 0x22fefdc0 -3438> 2014-11-03 11:37:54.680783 7f291450b700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.680783, event: journaled_completion_queued, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3437> 2014-11-03 11:37:54.680825 7f2911d06700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.680825, event: commit_sent, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3436> 2014-11-03 11:37:54.680844 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2650d840 con 0x22ff3b00 -3435> 2014-11-03 11:37:54.680866 7f2911d06700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.680866, event: commit_sent, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3434> 2014-11-03 11:37:54.680884 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f340 con 0x22fbc580 -3433> 2014-11-03 11:37:54.680910 7f2911d06700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.680910, event: commit_sent, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3432> 2014-11-03 11:37:54.680936 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917fb80 con 0x22ff3580 -3431> 2014-11-03 11:37:54.680999 7f2911d06700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.680999, event: commit_sent, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3430> 2014-11-03 11:37:54.681040 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917edc0 con 0x22fbc000 -3429> 2014-11-03 11:37:54.681077 7f2911d06700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.681077, event: commit_sent, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3428> 2014-11-03 11:37:54.681099 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917e2c0 con 0x22fbc580 -3427> 2014-11-03 11:37:54.681120 7f2911d06700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.681120, event: commit_sent, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3426> 2014-11-03 11:37:54.681145 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917e840 con 0x22fbc000 -3425> 2014-11-03 11:37:54.681175 7f2911d06700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.681175, event: commit_sent, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3424> 2014-11-03 11:37:54.681203 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6848/24829 -- osd_sub_op_reply(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1917f600 con 0x22ff3580 -3423> 2014-11-03 11:37:54.681234 7f2911d06700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.681234, event: commit_sent, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3422> 2014-11-03 11:37:54.681262 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2d423580 con 0x22fbc580 -3421> 2014-11-03 11:37:54.681289 7f2911d06700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.681289, event: commit_sent, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3420> 2014-11-03 11:37:54.681312 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] ondisk, result = 0) v2 -- ?+0 0x31525340 con 0x22fbc000 -3419> 2014-11-03 11:37:54.681339 7f2911d06700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.681338, event: commit_sent, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3418> 2014-11-03 11:37:54.681357 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6833/2034 -- osd_sub_op_reply(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x31524580 con 0x230a9c60 -3417> 2014-11-03 11:37:54.681378 7f2911d06700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.681378, event: commit_sent, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3416> 2014-11-03 11:37:54.681472 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6833/2034 -- osd_sub_op_reply(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x31524000 con 0x230a9c60 -3415> 2014-11-03 11:37:54.681514 7f2911d06700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.681514, event: commit_sent, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -3414> 2014-11-03 11:37:54.681539 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6833/2034 -- osd_sub_op_reply(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1c2c0 con 0x230a9c60 -3413> 2014-11-03 11:37:54.681912 7f2914d0c700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.681912, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3412> 2014-11-03 11:37:54.681955 7f2914d0c700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.681955, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3411> 2014-11-03 11:37:54.681989 7f2914d0c700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.681989, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3410> 2014-11-03 11:37:54.682013 7f2914d0c700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.682013, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3409> 2014-11-03 11:37:54.682048 7f2914d0c700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.682048, event: write_thread_in_journal_buffer, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3408> 2014-11-03 11:37:54.685622 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 175 ==== osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3359674836 0 3511520646) 0xd4b7800 con 0x22fefdc0 -3407> 2014-11-03 11:37:54.685657 7f28f36c2700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.685450, event: header_read, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3406> 2014-11-03 11:37:54.685672 7f28f36c2700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.685452, event: throttled, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3405> 2014-11-03 11:37:54.685681 7f28f36c2700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.685605, event: all_read, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3404> 2014-11-03 11:37:54.685699 7f28f36c2700 5 -- op tracker -- seq: 3143, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3403> 2014-11-03 11:37:54.685770 7f290101d700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.685770, event: reached_pg, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3402> 2014-11-03 11:37:54.685804 7f290101d700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.685803, event: started, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3401> 2014-11-03 11:37:54.685889 7f290101d700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.685889, event: started, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3400> 2014-11-03 11:37:54.685922 7f290101d700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.685922, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -3399> 2014-11-03 11:37:54.692754 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.49 192.168.5.4:0/21769 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.691873) v2 ==== 47+0+0 (1738927846 0 0) 0x2353ba00 con 0x249259a0 -3398> 2014-11-03 11:37:54.692810 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/21769 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.691873) v2 -- ?+0 0x2a7c4800 con 0x249259a0 -3397> 2014-11-03 11:37:54.692837 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.49 192.168.5.4:0/21769 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.691873) v2 ==== 47+0+0 (1738927846 0 0) 0x23510000 con 0x242296e0 -3396> 2014-11-03 11:37:54.692872 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/21769 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.691873) v2 -- ?+0 0x26bdba00 con 0x242296e0 -3395> 2014-11-03 11:37:54.698628 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(4e628eac/rbd_data.5007ab5e072ab6.00000000000003fe/head//6),PushReplyOp(d9649eac/rbd_data.89cd895a911542.0000000000005cde/head//6)]) v2 -- ?+0 0x252d7a00 con 0x2301f580 -3394> 2014-11-03 11:37:54.699292 7f2912507700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.699292, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3393> 2014-11-03 11:37:54.699343 7f2912507700 5 -- op tracker -- seq: 3109, time: 2014-11-03 11:37:54.699343, event: done, op: osd_sub_op(client.7873619.0:18867312 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219458 snapset=0=[]:[] snapc=0=[]) -3392> 2014-11-03 11:37:54.699415 7f2912507700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.699415, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3391> 2014-11-03 11:37:54.699444 7f2912507700 5 -- op tracker -- seq: 3110, time: 2014-11-03 11:37:54.699444, event: done, op: osd_sub_op(client.7873619.0:18867313 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219459 snapset=0=[]:[] snapc=0=[]) -3390> 2014-11-03 11:37:54.699592 7f2912507700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.699592, event: sub_op_applied, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3389> 2014-11-03 11:37:54.699644 7f2912507700 5 -- op tracker -- seq: 3111, time: 2014-11-03 11:37:54.699644, event: done, op: osd_sub_op(client.7873619.0:18867314 6.5a8 bddff5a8/rbd_data.4f62fb60cec91a.0000000000000098/head//6 [] v 105950'16219460 snapset=0=[]:[] snapc=0=[]) -3388> 2014-11-03 11:37:54.700066 7f2912507700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.700066, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3387> 2014-11-03 11:37:54.700112 7f2912507700 5 -- op tracker -- seq: 3112, time: 2014-11-03 11:37:54.700112, event: done, op: osd_sub_op(client.8703713.0:32254467 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953154 snapset=0=[]:[] snapc=0=[]) -3386> 2014-11-03 11:37:54.700242 7f2912507700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.700242, event: sub_op_applied, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3385> 2014-11-03 11:37:54.700286 7f2912507700 5 -- op tracker -- seq: 3113, time: 2014-11-03 11:37:54.700286, event: done, op: osd_sub_op(client.7890490.0:244504 24.3a1 36d1eba1/rbd_data.5743792ae8944a.0000000000000021/head//24 [] v 105950'222070 snapset=0=[]:[] snapc=0=[]) -3384> 2014-11-03 11:37:54.700424 7f2912507700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.700423, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3383> 2014-11-03 11:37:54.700475 7f2912507700 5 -- op tracker -- seq: 3114, time: 2014-11-03 11:37:54.700475, event: done, op: osd_sub_op(client.7868792.0:22003436 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362497 snapset=0=[]:[] snapc=0=[]) -3382> 2014-11-03 11:37:54.700652 7f2912507700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.700652, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3381> 2014-11-03 11:37:54.700715 7f2912507700 5 -- op tracker -- seq: 3115, time: 2014-11-03 11:37:54.700715, event: done, op: osd_sub_op(client.7868792.0:22003441 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649556 snapset=0=[]:[] snapc=0=[]) -3380> 2014-11-03 11:37:54.700817 7f2912507700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.700817, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3379> 2014-11-03 11:37:54.700852 7f2912507700 5 -- op tracker -- seq: 3116, time: 2014-11-03 11:37:54.700852, event: done, op: osd_sub_op(client.7868792.0:22003448 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100465 snapset=0=[]:[] snapc=0=[]) -3378> 2014-11-03 11:37:54.700936 7f2912507700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.700936, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3377> 2014-11-03 11:37:54.700974 7f2912507700 5 -- op tracker -- seq: 3117, time: 2014-11-03 11:37:54.700974, event: done, op: osd_sub_op(client.7868792.0:22003437 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362498 snapset=0=[]:[] snapc=0=[]) -3376> 2014-11-03 11:37:54.701104 7f2912507700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.701103, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3375> 2014-11-03 11:37:54.701143 7f2912507700 5 -- op tracker -- seq: 3118, time: 2014-11-03 11:37:54.701143, event: done, op: osd_sub_op(client.7868792.0:22003449 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100466 snapset=0=[]:[] snapc=0=[]) -3374> 2014-11-03 11:37:54.701267 7f2912507700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.701267, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3373> 2014-11-03 11:37:54.701321 7f2912507700 5 -- op tracker -- seq: 3119, time: 2014-11-03 11:37:54.701321, event: done, op: osd_sub_op(client.7868792.0:22003442 6.ad6 1f592ad6/rbd_data.4d4ddc4447765.00000000000000e6/head//6 [] v 105950'8649557 snapset=0=[]:[] snapc=0=[]) -3372> 2014-11-03 11:37:54.701415 7f2912507700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.701415, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3371> 2014-11-03 11:37:54.701451 7f2912507700 5 -- op tracker -- seq: 3120, time: 2014-11-03 11:37:54.701451, event: done, op: osd_sub_op(client.7868792.0:22003438 6.1f7 9c5d51f7/rbd_data.4d4ddc4447765.00000000000000db/head//6 [] v 105950'9362499 snapset=0=[]:[] snapc=0=[]) -3370> 2014-11-03 11:37:54.701567 7f2912507700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.701567, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3369> 2014-11-03 11:37:54.701601 7f2912507700 5 -- op tracker -- seq: 3121, time: 2014-11-03 11:37:54.701601, event: done, op: osd_sub_op(client.7868792.0:22003450 6.4f0 4b0ff4f0/rbd_data.4d4ddc4447765.0000000000000098/head//6 [] v 105950'13100467 snapset=0=[]:[] snapc=0=[]) -3368> 2014-11-03 11:37:54.701728 7f2912507700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.701727, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3367> 2014-11-03 11:37:54.701762 7f2912507700 5 -- op tracker -- seq: 3122, time: 2014-11-03 11:37:54.701762, event: done, op: osd_sub_op(client.7868792.0:22003487 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124375 snapset=0=[]:[] snapc=0=[]) -3366> 2014-11-03 11:37:54.701863 7f2912507700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.701863, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3365> 2014-11-03 11:37:54.701906 7f2912507700 5 -- op tracker -- seq: 3123, time: 2014-11-03 11:37:54.701906, event: done, op: osd_sub_op(client.7868792.0:22003488 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124376 snapset=0=[]:[] snapc=0=[]) -3364> 2014-11-03 11:37:54.714342 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.10 192.168.5.1:0/15344 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.713822) v2 ==== 47+0+0 (164048062 0 0) 0x2320b000 con 0x240ad000 -3363> 2014-11-03 11:37:54.714400 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/15344 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.713822) v2 -- ?+0 0x2353ba00 con 0x240ad000 -3362> 2014-11-03 11:37:54.714639 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.10 192.168.5.1:0/15344 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.713822) v2 ==== 47+0+0 (164048062 0 0) 0x253e5a00 con 0x2469edc0 -3361> 2014-11-03 11:37:54.714696 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/15344 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.713822) v2 -- ?+0 0x23510000 con 0x2469edc0 -3360> 2014-11-03 11:37:54.731665 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 44 ==== MOSDPGPush(6.6ac 105950 [PushOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6, version: 105946'9806056, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6@105946'9806056, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (2752489175 0 0) 0x2340c800 con 0x2301f580 -3359> 2014-11-03 11:37:54.731756 7f28f2db9700 5 -- op tracker -- seq: 3144, time: 2014-11-03 11:37:54.699907, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6, version: 105946'9806056, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6@105946'9806056, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3358> 2014-11-03 11:37:54.731854 7f28f2db9700 5 -- op tracker -- seq: 3144, time: 2014-11-03 11:37:54.699909, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6, version: 105946'9806056, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6@105946'9806056, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3357> 2014-11-03 11:37:54.731907 7f28f2db9700 5 -- op tracker -- seq: 3144, time: 2014-11-03 11:37:54.731640, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6, version: 105946'9806056, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6@105946'9806056, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3356> 2014-11-03 11:37:54.731930 7f28f2db9700 5 -- op tracker -- seq: 3144, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6, version: 105946'9806056, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6@105946'9806056, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3355> 2014-11-03 11:37:54.732092 7f290081c700 5 -- op tracker -- seq: 3144, time: 2014-11-03 11:37:54.732092, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6, version: 105946'9806056, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6@105946'9806056, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3354> 2014-11-03 11:37:54.732387 7f290081c700 5 -- op tracker -- seq: 3144, time: 2014-11-03 11:37:54.732387, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6, version: 105946'9806056, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6@105946'9806056, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3353> 2014-11-03 11:37:54.744219 7f291450b700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.744219, event: journaled_completion_queued, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3352> 2014-11-03 11:37:54.744297 7f291450b700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.744297, event: journaled_completion_queued, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3351> 2014-11-03 11:37:54.744299 7f2911d06700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:54.744298, event: commit_sent, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -3350> 2014-11-03 11:37:54.744350 7f291450b700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.744350, event: journaled_completion_queued, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3349> 2014-11-03 11:37:54.744374 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6810/5438 -- osd_sub_op_reply(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1d080 con 0x231ab6e0 -3348> 2014-11-03 11:37:54.744402 7f291450b700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.744402, event: journaled_completion_queued, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3347> 2014-11-03 11:37:54.744466 7f2911d06700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:54.744466, event: commit_sent, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -3346> 2014-11-03 11:37:54.744461 7f291450b700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.744461, event: journaled_completion_queued, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3345> 2014-11-03 11:37:54.744634 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6810/5438 -- osd_sub_op_reply(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcf9580 con 0x231ab6e0 -3344> 2014-11-03 11:37:54.744695 7f2911d06700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:54.744695, event: commit_sent, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -3343> 2014-11-03 11:37:54.744739 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6809/13676 -- osd_sub_op_reply(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2aa6c080 con 0x23157000 -3342> 2014-11-03 11:37:54.744796 7f2911d06700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:54.744795, event: commit_sent, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -3341> 2014-11-03 11:37:54.745011 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6809/13676 -- osd_sub_op_reply(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23d8d580 con 0x23157000 -3340> 2014-11-03 11:37:54.745085 7f2911d06700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.745084, event: op_commit, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3339> 2014-11-03 11:37:54.745129 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.2:0/1027858 -- osd_op_reply(459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] v105950'131096 uv131096 ondisk = 0) v6 -- ?+0 0x316d28c0 con 0x2311c9a0 -3338> 2014-11-03 11:37:54.745157 7f2911d06700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:54.745157, event: commit_sent, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -3337> 2014-11-03 11:37:54.746837 7f2914d0c700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.746836, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -3336> 2014-11-03 11:37:54.750942 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(e8c61eac/rbd_data.3a33ad2ae8944a.00000000000001bf/head//6)]) v2 -- ?+0 0x2b950200 con 0x2301f580 -3335> 2014-11-03 11:37:54.758656 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 100 ==== osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2654319755 0 3511520646) 0x27811000 con 0x22fbcb00 -3334> 2014-11-03 11:37:54.758724 7f28f6cf8700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.758516, event: header_read, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3333> 2014-11-03 11:37:54.758743 7f28f6cf8700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.758518, event: throttled, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3332> 2014-11-03 11:37:54.758762 7f28f6cf8700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.758640, event: all_read, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3331> 2014-11-03 11:37:54.758774 7f28f6cf8700 5 -- op tracker -- seq: 3145, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3330> 2014-11-03 11:37:54.758888 7f28fe017700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.758888, event: reached_pg, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3329> 2014-11-03 11:37:54.758923 7f28fe017700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.758923, event: started, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3328> 2014-11-03 11:37:54.759030 7f28fe017700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.759030, event: started, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3327> 2014-11-03 11:37:54.759068 7f28fe017700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.759068, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -3326> 2014-11-03 11:37:54.760701 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x29059000 sd=205 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bc420).accept sd=205 192.168.5.2:42965/0 -3325> 2014-11-03 11:37:54.761506 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450910 192.168.5.2:0/1003652 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0xada4200 con 0x1c9bc420 -3324> 2014-11-03 11:37:54.762080 7f28fb812700 0 osd.21 105950 do_command r=0 -3323> 2014-11-03 11:37:54.762123 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003652 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x6ef0e00 con 0x1c9bc420 -3322> 2014-11-03 11:37:54.774794 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450910 192.168.5.2:0/1003652 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x13defc00 con 0x1c9bc420 -3321> 2014-11-03 11:37:54.775629 7f28fb812700 0 osd.21 105950 do_command r=0 -3320> 2014-11-03 11:37:54.775648 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003652 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x23e5fc40 con 0x1c9bc420 -3319> 2014-11-03 11:37:54.776693 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 45 ==== MOSDPGPush(6.6ac 105950 [PushOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6, version: 105946'9806060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6@105946'9806060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (2061616405 0 0) 0x252d7a00 con 0x2301f580 -3318> 2014-11-03 11:37:54.776754 7f28f2db9700 5 -- op tracker -- seq: 3146, time: 2014-11-03 11:37:54.745733, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6, version: 105946'9806060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6@105946'9806060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3317> 2014-11-03 11:37:54.776784 7f28f2db9700 5 -- op tracker -- seq: 3146, time: 2014-11-03 11:37:54.745737, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6, version: 105946'9806060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6@105946'9806060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3316> 2014-11-03 11:37:54.776817 7f28f2db9700 5 -- op tracker -- seq: 3146, time: 2014-11-03 11:37:54.776662, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6, version: 105946'9806060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6@105946'9806060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3315> 2014-11-03 11:37:54.776840 7f28f2db9700 5 -- op tracker -- seq: 3146, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6, version: 105946'9806060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6@105946'9806060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3314> 2014-11-03 11:37:54.776905 7f290081c700 5 -- op tracker -- seq: 3146, time: 2014-11-03 11:37:54.776905, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6, version: 105946'9806060, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6@105946'9806060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3313> 2014-11-03 11:37:54.777119 7f290081c700 5 -- op tracker -- seq: 3146, time: 2014-11-03 11:37:54.777118, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6, version: 105946'9806060, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6@105946'9806060, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3312> 2014-11-03 11:37:54.782777 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003652 pipe(0x29059000 sd=205 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc420).reader couldn't read tag, (0) Success -3311> 2014-11-03 11:37:54.782846 7f290d72d700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003652 pipe(0x29059000 sd=205 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc420).fault (0) Success -3310> 2014-11-03 11:37:54.782994 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bc420 session 0x7c35280 -3309> 2014-11-03 11:37:54.800896 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 46 ==== MOSDPGPush(6.6ac 105950 [PushOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6, version: 105946'9806062, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6@105946'9806062, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1575159334 0 0) 0x2b950200 con 0x2301f580 -3308> 2014-11-03 11:37:54.800963 7f28f2db9700 5 -- op tracker -- seq: 3147, time: 2014-11-03 11:37:54.776914, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6, version: 105946'9806062, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6@105946'9806062, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3307> 2014-11-03 11:37:54.800997 7f28f2db9700 5 -- op tracker -- seq: 3147, time: 2014-11-03 11:37:54.776914, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6, version: 105946'9806062, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6@105946'9806062, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3306> 2014-11-03 11:37:54.801021 7f28f2db9700 5 -- op tracker -- seq: 3147, time: 2014-11-03 11:37:54.800873, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6, version: 105946'9806062, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6@105946'9806062, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3305> 2014-11-03 11:37:54.801044 7f28f2db9700 5 -- op tracker -- seq: 3147, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6, version: 105946'9806062, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6@105946'9806062, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3304> 2014-11-03 11:37:54.801121 7f28fe017700 5 -- op tracker -- seq: 3147, time: 2014-11-03 11:37:54.801121, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6, version: 105946'9806062, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6@105946'9806062, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3303> 2014-11-03 11:37:54.801304 7f28fe017700 5 -- op tracker -- seq: 3147, time: 2014-11-03 11:37:54.801303, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6, version: 105946'9806062, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6@105946'9806062, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3302> 2014-11-03 11:37:54.822196 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 47 ==== MOSDPGPush(6.6ac 105950 [PushOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6, version: 105946'9806067, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6@105946'9806067, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1521775448 0 0) 0x4d90800 con 0x2301f580 -3301> 2014-11-03 11:37:54.822264 7f28f2db9700 5 -- op tracker -- seq: 3148, time: 2014-11-03 11:37:54.801115, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6, version: 105946'9806067, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6@105946'9806067, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3300> 2014-11-03 11:37:54.822323 7f28f2db9700 5 -- op tracker -- seq: 3148, time: 2014-11-03 11:37:54.801116, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6, version: 105946'9806067, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6@105946'9806067, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3299> 2014-11-03 11:37:54.822351 7f28f2db9700 5 -- op tracker -- seq: 3148, time: 2014-11-03 11:37:54.822174, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6, version: 105946'9806067, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6@105946'9806067, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3298> 2014-11-03 11:37:54.822381 7f28f2db9700 5 -- op tracker -- seq: 3148, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6, version: 105946'9806067, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6@105946'9806067, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3297> 2014-11-03 11:37:54.822555 7f290081c700 5 -- op tracker -- seq: 3148, time: 2014-11-03 11:37:54.822555, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6, version: 105946'9806067, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6@105946'9806067, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3296> 2014-11-03 11:37:54.822834 7f290081c700 5 -- op tracker -- seq: 3148, time: 2014-11-03 11:37:54.822834, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6, version: 105946'9806067, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6@105946'9806067, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3295> 2014-11-03 11:37:54.829559 7f290d72d700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x27832340 sd=205 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bc580).accept sd=205 192.168.9.2:56799/0 -3294> 2014-11-03 11:37:54.830564 7f290d72d700 1 -- 192.168.5.2:6802/7606 <== client.7870547 192.168.9.2:0/1027018 1 ==== osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) v4 ==== 237+0+4096 (2849854758 0 665155927) 0x7c35c80 con 0x1c9bc580 -3293> 2014-11-03 11:37:54.830605 7f290d72d700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:54.830474, event: header_read, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -3292> 2014-11-03 11:37:54.830622 7f290d72d700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:54.830479, event: throttled, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -3291> 2014-11-03 11:37:54.830632 7f290d72d700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:54.830548, event: all_read, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -3290> 2014-11-03 11:37:54.830641 7f290d72d700 5 -- op tracker -- seq: 3149, time: 0.000000, event: dispatched, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -3289> 2014-11-03 11:37:54.830829 7f290001b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:54.830828, event: reached_pg, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -3288> 2014-11-03 11:37:54.830937 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- MOSDPGPull(24.761 105950 [PullOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x25835400 con 0x22fb76e0 -3287> 2014-11-03 11:37:54.830984 7f290001b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:54.830984, event: waiting for missing object, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -3286> 2014-11-03 11:37:54.844295 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 48 ==== MOSDPGPush(6.6ac 105950 [PushOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6, version: 105946'9806069, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6@105946'9806069, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1155199362 0 0) 0x23fdf200 con 0x2301f580 -3285> 2014-11-03 11:37:54.844365 7f28f2db9700 5 -- op tracker -- seq: 3150, time: 2014-11-03 11:37:54.822469, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6, version: 105946'9806069, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6@105946'9806069, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3284> 2014-11-03 11:37:54.844405 7f28f2db9700 5 -- op tracker -- seq: 3150, time: 2014-11-03 11:37:54.822470, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6, version: 105946'9806069, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6@105946'9806069, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3283> 2014-11-03 11:37:54.844433 7f28f2db9700 5 -- op tracker -- seq: 3150, time: 2014-11-03 11:37:54.844271, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6, version: 105946'9806069, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6@105946'9806069, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3282> 2014-11-03 11:37:54.844460 7f28f2db9700 5 -- op tracker -- seq: 3150, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6, version: 105946'9806069, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6@105946'9806069, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3281> 2014-11-03 11:37:54.844557 7f28fe017700 5 -- op tracker -- seq: 3150, time: 2014-11-03 11:37:54.844557, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6, version: 105946'9806069, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6@105946'9806069, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3280> 2014-11-03 11:37:54.844720 7f28fe017700 5 -- op tracker -- seq: 3150, time: 2014-11-03 11:37:54.844719, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6, version: 105946'9806069, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6@105946'9806069, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -3279> 2014-11-03 11:37:54.848405 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 52 ==== osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2100345089 0 3511520646) 0x26152800 con 0x23000b00 -3278> 2014-11-03 11:37:54.848521 7f28f21ad700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.848107, event: header_read, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3277> 2014-11-03 11:37:54.848571 7f28f21ad700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.848110, event: throttled, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3276> 2014-11-03 11:37:54.848602 7f28f21ad700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.848378, event: all_read, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3275> 2014-11-03 11:37:54.848639 7f28f21ad700 5 -- op tracker -- seq: 3151, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3274> 2014-11-03 11:37:54.848671 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 99 ==== osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (3729781949 0 3511520646) 0x25eb7c00 con 0x22fbc420 -3273> 2014-11-03 11:37:54.848740 7f28f72fe700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.848510, event: header_read, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3272> 2014-11-03 11:37:54.848756 7f28ff019700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.848756, event: reached_pg, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3271> 2014-11-03 11:37:54.848788 7f28f72fe700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.848513, event: throttled, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3270> 2014-11-03 11:37:54.848801 7f28ff019700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.848801, event: started, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3269> 2014-11-03 11:37:54.848823 7f28f72fe700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.848644, event: all_read, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3268> 2014-11-03 11:37:54.848855 7f28f72fe700 5 -- op tracker -- seq: 3152, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3267> 2014-11-03 11:37:54.849009 7f28ff019700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.849008, event: started, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3266> 2014-11-03 11:37:54.849087 7f28ff019700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.849087, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -3265> 2014-11-03 11:37:54.850297 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 100 ==== osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (916172310 0 3511520646) 0x27999000 con 0x22fbc420 -3264> 2014-11-03 11:37:54.850355 7f28f72fe700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.850026, event: header_read, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3263> 2014-11-03 11:37:54.850394 7f28f72fe700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.850029, event: throttled, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3262> 2014-11-03 11:37:54.850426 7f28f72fe700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.850273, event: all_read, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3261> 2014-11-03 11:37:54.850454 7f28f72fe700 5 -- op tracker -- seq: 3153, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3260> 2014-11-03 11:37:54.850544 7f28ff81a700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.850544, event: reached_pg, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3259> 2014-11-03 11:37:54.850582 7f28ff81a700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.850582, event: started, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3258> 2014-11-03 11:37:54.850686 7f28ff81a700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.850686, event: started, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3257> 2014-11-03 11:37:54.850714 7f28ff81a700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.850714, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -3256> 2014-11-03 11:37:54.850743 7f28ff81a700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.850742, event: reached_pg, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3255> 2014-11-03 11:37:54.850758 7f28ff81a700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.850758, event: started, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3254> 2014-11-03 11:37:54.850833 7f28ff81a700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.850832, event: started, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3253> 2014-11-03 11:37:54.850859 7f28ff81a700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.850859, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -3252> 2014-11-03 11:37:54.854444 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 53 ==== osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) v11 ==== 1402+0+39 (2869383447 0 3511520646) 0x26c42c00 con 0x23000b00 -3251> 2014-11-03 11:37:54.854476 7f28f21ad700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.854295, event: header_read, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3250> 2014-11-03 11:37:54.854492 7f28f21ad700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.854297, event: throttled, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3249> 2014-11-03 11:37:54.854505 7f28f21ad700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.854427, event: all_read, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3248> 2014-11-03 11:37:54.854517 7f28f21ad700 5 -- op tracker -- seq: 3154, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3247> 2014-11-03 11:37:54.854582 7f290181e700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.854582, event: reached_pg, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3246> 2014-11-03 11:37:54.854612 7f290181e700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.854611, event: started, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3245> 2014-11-03 11:37:54.854700 7f290181e700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.854700, event: started, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3244> 2014-11-03 11:37:54.854773 7f290181e700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.854773, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -3243> 2014-11-03 11:37:54.884540 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 101 ==== osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4925 (1671000934 0 2067311863) 0x26942c00 con 0x22fbcb00 -3242> 2014-11-03 11:37:54.884587 7f28f6cf8700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.884332, event: header_read, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3241> 2014-11-03 11:37:54.884603 7f28f6cf8700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.884336, event: throttled, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3240> 2014-11-03 11:37:54.884611 7f28f6cf8700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.884522, event: all_read, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3239> 2014-11-03 11:37:54.884619 7f28f6cf8700 5 -- op tracker -- seq: 3155, time: 0.000000, event: dispatched, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3238> 2014-11-03 11:37:54.884687 7f28ff81a700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.884687, event: reached_pg, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3237> 2014-11-03 11:37:54.884724 7f28ff81a700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.884724, event: started, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3236> 2014-11-03 11:37:54.884832 7f28ff81a700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.884831, event: started, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3235> 2014-11-03 11:37:54.884862 7f28ff81a700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.884862, event: commit_queued_for_journal_write, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -3234> 2014-11-03 11:37:54.908453 7f28fb011700 5 osd.21 105950 heartbeat: osd_stat(1844 GB used, 1867 GB avail, 3712 GB total, peers [1,3,4,6,7,8,10,12,13,20,23,28,30,31,32,33,35,37,40,42,43,44,45,46,48,49,50,54,55,61,66,68,69,70,75,76,77,81,82,83,90,93,97,98,99,102,103,104,105,107,108,109,110,112,113,114,116,118,121,122,123,124,125]/[] op hist [0,0,0,0,0,7,4,1,2,3]) -3233> 2014-11-03 11:37:54.908534 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6808/11054 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2a2e0800 con 0x239f4dc0 -3232> 2014-11-03 11:37:54.908619 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6809/11054 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6023c00 con 0x239f4c60 -3231> 2014-11-03 11:37:54.908682 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6858/28138 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6023200 con 0x2445a2c0 -3230> 2014-11-03 11:37:54.908730 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6860/28138 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8abf000 con 0x2445a160 -3229> 2014-11-03 11:37:54.908765 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6868/31636 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8abfa00 con 0x2439c000 -3228> 2014-11-03 11:37:54.908796 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6869/31636 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8abfc00 con 0x2445ac60 -3227> 2014-11-03 11:37:54.908839 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6882/1181 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8abfe00 con 0x233da580 -3226> 2014-11-03 11:37:54.908879 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6883/1181 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8abf600 con 0x233da420 -3225> 2014-11-03 11:37:54.908912 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6887/3511 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8abf200 con 0x233da000 -3224> 2014-11-03 11:37:54.908944 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6888/3511 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xa1a7a00 con 0x233da2c0 -3223> 2014-11-03 11:37:54.908979 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6893/8894 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xa1a7e00 con 0x23d636e0 -3222> 2014-11-03 11:37:54.909029 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6894/8894 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x41c2800 con 0x23d63580 -3221> 2014-11-03 11:37:54.909063 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6817/15344 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8664200 con 0x23d63c60 -3220> 2014-11-03 11:37:54.909098 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6821/15344 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8664400 con 0x23d63b00 -3219> 2014-11-03 11:37:54.909130 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6829/19844 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8664c00 con 0x2416eb00 -3218> 2014-11-03 11:37:54.909162 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6847/19844 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8664600 con 0x2416e9a0 -3217> 2014-11-03 11:37:54.909194 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6840/22281 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8664a00 con 0x23ea02c0 -3216> 2014-11-03 11:37:54.909222 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.1:6852/22281 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23250800 con 0x23ea0160 -3215> 2014-11-03 11:37:54.909255 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6834/4963 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23250e00 con 0x2342c420 -3214> 2014-11-03 11:37:54.909290 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6837/4963 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x218c4c00 con 0x2342c2c0 -3213> 2014-11-03 11:37:54.909331 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6852/14826 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x218c4800 con 0x2342c160 -3212> 2014-11-03 11:37:54.909354 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.2:6853/14826 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x25067400 con 0x2342c000 -3211> 2014-11-03 11:37:54.909378 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6813/13976 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x25067c00 con 0x2445ab00 -3210> 2014-11-03 11:37:54.909364 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.4 192.168.5.1:6868/31636 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x218c4600 con 0x2439c000 -3209> 2014-11-03 11:37:54.909408 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6817/13976 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2438f000 con 0x2445a9a0 -3208> 2014-11-03 11:37:54.909433 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6818/11504 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2a347400 con 0x2387dc60 -3207> 2014-11-03 11:37:54.909463 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6820/11504 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x24dcf800 con 0x2387db00 -3206> 2014-11-03 11:37:54.909496 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6826/20618 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23ba6000 con 0x233d2c60 -3205> 2014-11-03 11:37:54.909528 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6827/20618 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23ba6800 con 0x233d2b00 -3204> 2014-11-03 11:37:54.909566 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6830/25327 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x273fde00 con 0x233dac60 -3203> 2014-11-03 11:37:54.909610 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6831/25327 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x22fd4200 con 0x233dab00 -3202> 2014-11-03 11:37:54.909645 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6840/29378 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2324bc00 con 0x2387fb00 -3201> 2014-11-03 11:37:54.909679 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6841/29378 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2324b200 con 0x2387f9a0 -3200> 2014-11-03 11:37:54.909710 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6847/6455 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23305000 con 0x2387d160 -3199> 2014-11-03 11:37:54.909733 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6848/6455 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23305a00 con 0x2387d000 -3198> 2014-11-03 11:37:54.909762 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6858/13652 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x253e5000 con 0x2387fdc0 -3197> 2014-11-03 11:37:54.909801 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6859/13652 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x253e5c00 con 0x2387fc60 -3196> 2014-11-03 11:37:54.909841 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6869/25675 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x253e5200 con 0x235f99a0 -3195> 2014-11-03 11:37:54.909873 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.3:6870/25675 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x253e5800 con 0x235f9840 -3194> 2014-11-03 11:37:54.909905 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6808/9946 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x226b5800 con 0x23ea0580 -3193> 2014-11-03 11:37:54.909938 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6809/9946 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x226b5a00 con 0x23ea0420 -3192> 2014-11-03 11:37:54.909972 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6813/19461 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x226b5e00 con 0x233da160 -3191> 2014-11-03 11:37:54.910006 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6814/19461 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x226b5200 con 0x233da6e0 -3190> 2014-11-03 11:37:54.910031 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6817/27523 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6a83e00 con 0x2416e000 -3189> 2014-11-03 11:37:54.910069 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6818/27523 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6a83a00 con 0x24448420 -3188> 2014-11-03 11:37:54.910101 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6822/1781 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6a83000 con 0x2342c9a0 -3187> 2014-11-03 11:37:54.910158 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6823/1781 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6a83400 con 0x2342c840 -3186> 2014-11-03 11:37:54.910200 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6827/7678 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x31a6a00 con 0x245abc60 -3185> 2014-11-03 11:37:54.910226 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6828/7678 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x31a6400 con 0x245abb00 -3184> 2014-11-03 11:37:54.910258 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6837/17727 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d361200 con 0x23ea0000 -3183> 2014-11-03 11:37:54.910303 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6839/17727 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d361a00 con 0x2387ddc0 -3182> 2014-11-03 11:37:54.910339 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6843/21769 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d361c00 con 0x244482c0 -3181> 2014-11-03 11:37:54.910368 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6844/21769 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2355b400 con 0x23eb86e0 -3180> 2014-11-03 11:37:54.910390 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6848/26902 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2355b600 con 0x245ab9a0 -3179> 2014-11-03 11:37:54.910425 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6849/26902 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2355b200 con 0x245ab840 -3178> 2014-11-03 11:37:54.910459 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6869/8391 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8025600 con 0x242056e0 -3177> 2014-11-03 11:37:54.910494 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6870/8391 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8025400 con 0x9921000 -3176> 2014-11-03 11:37:54.910526 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6872/12368 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8025800 con 0x233d2580 -3175> 2014-11-03 11:37:54.910556 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.4:6873/12368 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8025000 con 0x233d2420 -3174> 2014-11-03 11:37:54.910593 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6832/12481 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8025a00 con 0x239f42c0 -3173> 2014-11-03 11:37:54.910642 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6833/12481 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x8025200 con 0x239f4160 -3172> 2014-11-03 11:37:54.910677 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6853/919 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6352400 con 0x233d22c0 -3171> 2014-11-03 11:37:54.910712 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6859/919 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6352c00 con 0x233d2160 -3170> 2014-11-03 11:37:54.910742 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6867/8133 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6352800 con 0x245ab420 -3169> 2014-11-03 11:37:54.910775 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6868/8133 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2a2e0600 con 0x245ab2c0 -3168> 2014-11-03 11:37:54.910845 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6872/11297 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xaf3b200 con 0x23eb8580 -3167> 2014-11-03 11:37:54.910883 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.5:6873/11297 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xaf3b400 con 0x23eb8420 -3166> 2014-11-03 11:37:54.910915 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6809/3622 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x22d40e00 con 0x239f4b00 -3165> 2014-11-03 11:37:54.910946 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6810/3622 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x22d40800 con 0x239f49a0 -3164> 2014-11-03 11:37:54.910967 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6832/12612 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xe5b5e00 con 0x2445a840 -3163> 2014-11-03 11:37:54.910994 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6833/12612 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xe5b5800 con 0x2445a6e0 -3162> 2014-11-03 11:37:54.911022 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6837/22493 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6c82e00 con 0x23d639a0 -3161> 2014-11-03 11:37:54.911056 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6838/22493 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6315800 con 0x23d63840 -3160> 2014-11-03 11:37:54.911119 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6842/32590 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x6315e00 con 0x239f4000 -3159> 2014-11-03 11:37:54.911146 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6843/32590 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x27969e00 con 0x23d63dc0 -3158> 2014-11-03 11:37:54.911198 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6863/27170 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x147be400 con 0x23d63160 -3157> 2014-11-03 11:37:54.911233 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6864/27170 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x147be600 con 0x23d63000 -3156> 2014-11-03 11:37:54.911258 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6867/31491 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x1ada2800 con 0x235f9c60 -3155> 2014-11-03 11:37:54.911282 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6869/31491 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2576c800 con 0x235f9b00 -3154> 2014-11-03 11:37:54.911306 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6872/4005 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2576ce00 con 0x2387d9a0 -3153> 2014-11-03 11:37:54.911337 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.6:6873/4005 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x24dce400 con 0x2387d840 -3152> 2014-11-03 11:37:54.911374 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6832/27918 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x24dce000 con 0x233d29a0 -3151> 2014-11-03 11:37:54.911411 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6833/27918 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x22a35e00 con 0x233d2840 -3150> 2014-11-03 11:37:54.911435 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6842/7095 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x22a35800 con 0x235f96e0 -3149> 2014-11-03 11:37:54.911454 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6843/7095 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xc02ec00 con 0x235f92c0 -3148> 2014-11-03 11:37:54.911485 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6862/21285 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xc02e800 con 0x233d2000 -3147> 2014-11-03 11:37:54.911507 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.7:6863/21285 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x1474a800 con 0x233d2dc0 -3146> 2014-11-03 11:37:54.911539 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6821/14722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x42a3800 con 0x239f4580 -3145> 2014-11-03 11:37:54.911587 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6823/14722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x256b9600 con 0x239f4420 -3144> 2014-11-03 11:37:54.911615 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6831/26930 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x256b9400 con 0x2387d420 -3143> 2014-11-03 11:37:54.911650 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6832/26930 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x256b9c00 con 0x2387d2c0 -3142> 2014-11-03 11:37:54.911672 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6812/21153 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2386ce00 con 0x23ea0b00 -3141> 2014-11-03 11:37:54.911698 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6814/21153 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2386c000 con 0x23ea09a0 -3140> 2014-11-03 11:37:54.911716 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6827/29722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2386c200 con 0x233da9a0 -3139> 2014-11-03 11:37:54.911740 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6830/29722 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x24092800 con 0x233da840 -3138> 2014-11-03 11:37:54.911764 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6843/2576 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x24092e00 con 0x245ab160 -3137> 2014-11-03 11:37:54.911799 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6844/2576 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xf637800 con 0x245ab000 -3136> 2014-11-03 11:37:54.911883 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6841/6777 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23f8c800 con 0x2387d6e0 -3135> 2014-11-03 11:37:54.911927 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6842/6777 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23561e00 con 0x2387d580 -3134> 2014-11-03 11:37:54.911955 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6839/17474 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23561400 con 0x2342c6e0 -3133> 2014-11-03 11:37:54.911988 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6855/17474 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23561000 con 0x2342c580 -3132> 2014-11-03 11:37:54.912015 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6820/21567 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23561200 con 0x2416e2c0 -3131> 2014-11-03 11:37:54.912056 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6857/21567 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23561c00 con 0x2416e160 -3130> 2014-11-03 11:37:54.912084 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6860/25883 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23ff5800 con 0x2416e580 -3129> 2014-11-03 11:37:54.912151 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6861/25883 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x54a3800 con 0x2416e420 -3128> 2014-11-03 11:37:54.912183 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6863/29518 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x54a3a00 con 0x2445a580 -3127> 2014-11-03 11:37:54.912206 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.8:6864/29518 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x54a3e00 con 0x2445a420 -3126> 2014-11-03 11:37:54.912239 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6810/24552 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x896d800 con 0x2387f840 -3125> 2014-11-03 11:37:54.912263 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6811/24552 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x896dc00 con 0x2387f6e0 -3124> 2014-11-03 11:37:54.912282 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6821/5555 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x896da00 con 0x23d63420 -3123> 2014-11-03 11:37:54.912316 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6829/5555 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d6ec800 con 0x23d632c0 -3122> 2014-11-03 11:37:54.912340 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6817/17408 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d6ec600 con 0x233dadc0 -3121> 2014-11-03 11:37:54.912361 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6818/17408 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d6ece00 con 0x233d26e0 -3120> 2014-11-03 11:37:54.912388 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6803/6451 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d6ec000 con 0x2416e840 -3119> 2014-11-03 11:37:54.912417 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6828/6451 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d6ecc00 con 0x2416e6e0 -3118> 2014-11-03 11:37:54.912439 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6839/23392 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xc2d1800 con 0x245ab6e0 -3117> 2014-11-03 11:37:54.912470 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6840/23392 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0xc2d1e00 con 0x245ab580 -3116> 2014-11-03 11:37:54.912494 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6850/12205 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d344800 con 0x2445a000 -3115> 2014-11-03 11:37:54.912517 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6851/12205 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d344e00 con 0x2445adc0 -3114> 2014-11-03 11:37:54.912568 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6855/17563 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x23821800 con 0x239f4840 -3113> 2014-11-03 11:37:54.912596 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6856/17563 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x27783800 con 0x239f46e0 -3112> 2014-11-03 11:37:54.912623 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6861/22948 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x242e2400 con 0x23ea0840 -3111> 2014-11-03 11:37:54.912638 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6862/22948 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x242e2e00 con 0x23ea06e0 -3110> 2014-11-03 11:37:54.912762 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6865/28288 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d164800 con 0x23ea0dc0 -3109> 2014-11-03 11:37:54.912814 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6866/28288 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2d164e00 con 0x23ea0c60 -3108> 2014-11-03 11:37:54.912842 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6871/31543 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2ba4d200 con 0x2416edc0 -3107> 2014-11-03 11:37:54.912867 7f28fb011700 1 -- 192.168.5.2:0/7606 --> 192.168.5.9:6872/31543 -- osd_ping(ping e105950 stamp 2014-11-03 11:37:54.908509) v2 -- ?+0 0x2ba4d800 con 0x2416ec60 -3106> 2014-11-03 11:37:54.912981 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.3 192.168.5.1:6858/28138 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xddd0600 con 0x2445a2c0 -3105> 2014-11-03 11:37:54.913021 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.3 192.168.5.1:6860/28138 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x6c82600 con 0x2445a160 -3104> 2014-11-03 11:37:54.913049 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.54 192.168.5.4:6869/8391 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x30db200 con 0x242056e0 -3103> 2014-11-03 11:37:54.913122 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.54 192.168.5.4:6870/8391 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xa053600 con 0x9921000 -3102> 2014-11-03 11:37:54.913288 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.121 192.168.5.9:0/12205 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.911521) v2 ==== 47+0+0 (3695665581 0 0) 0xaf2e400 con 0x24229b00 -3101> 2014-11-03 11:37:54.913328 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.121 192.168.5.9:0/12205 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.911521) v2 ==== 47+0+0 (3695665581 0 0) 0x23cbec00 con 0x24925dc0 -3100> 2014-11-03 11:37:54.913389 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.82 192.168.5.6:6867/31491 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x27969600 con 0x235f9c60 -3099> 2014-11-03 11:37:54.913400 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/12205 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.911521) v2 -- ?+0 0x253e5a00 con 0x24229b00 -3098> 2014-11-03 11:37:54.913537 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/12205 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.911521) v2 -- ?+0 0x2320b000 con 0x24925dc0 -3097> 2014-11-03 11:37:54.913637 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.82 192.168.5.6:6869/31491 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2ba4d600 con 0x235f9b00 -3096> 2014-11-03 11:37:54.913685 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.40 192.168.5.3:6869/25675 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23937c00 con 0x235f99a0 -3095> 2014-11-03 11:37:54.913725 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.40 192.168.5.3:6870/25675 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x5f22c00 con 0x235f9840 -3094> 2014-11-03 11:37:54.913749 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.93 192.168.5.7:6842/7095 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x24800400 con 0x235f96e0 -3093> 2014-11-03 11:37:54.913773 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.93 192.168.5.7:6843/7095 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xa5d2c00 con 0x235f92c0 -3092> 2014-11-03 11:37:54.913797 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.1 192.168.5.1:6808/11054 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x240f8400 con 0x239f4dc0 -3091> 2014-11-03 11:37:54.913823 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.1 192.168.5.1:6809/11054 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x226aaa00 con 0x239f4c60 -3090> 2014-11-03 11:37:54.913845 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.70 192.168.5.6:6809/3622 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xa248600 con 0x239f4b00 -3089> 2014-11-03 11:37:54.913867 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.70 192.168.5.6:6810/3622 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xa248c00 con 0x239f49a0 -3088> 2014-11-03 11:37:54.913892 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.122 192.168.5.9:6855/17563 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x226aa000 con 0x239f4840 -3087> 2014-11-03 11:37:54.913916 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.122 192.168.5.9:6856/17563 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23821a00 con 0x239f46e0 -3086> 2014-11-03 11:37:54.913936 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.98 192.168.5.8:6821/14722 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x24dcec00 con 0x239f4580 -3085> 2014-11-03 11:37:54.913960 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.98 192.168.5.8:6823/14722 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2576c600 con 0x239f4420 -3084> 2014-11-03 11:37:54.913982 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.61 192.168.5.5:6832/12481 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x1ada2600 con 0x239f42c0 -3083> 2014-11-03 11:37:54.914005 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.61 192.168.5.5:6833/12481 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x147bea00 con 0x239f4160 -3082> 2014-11-03 11:37:54.914026 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.77 192.168.5.6:6842/32590 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x27969c00 con 0x239f4000 -3081> 2014-11-03 11:37:54.914049 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.77 192.168.5.6:6843/32590 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2d164c00 con 0x23d63dc0 -3080> 2014-11-03 11:37:54.914071 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.10 192.168.5.1:6817/15344 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x6315600 con 0x23d63c60 -3079> 2014-11-03 11:37:54.914094 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.10 192.168.5.1:6821/15344 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x6c82c00 con 0x23d63b00 -3078> 2014-11-03 11:37:54.914145 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.76 192.168.5.6:6837/22493 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xe5b5c00 con 0x23d639a0 -3077> 2014-11-03 11:37:54.914172 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.76 192.168.5.6:6838/22493 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xecb2600 con 0x23d63840 -3076> 2014-11-03 11:37:54.914204 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.8 192.168.5.1:6893/8894 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x22d40c00 con 0x23d636e0 -3075> 2014-11-03 11:37:54.914226 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.8 192.168.5.1:6894/8894 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xaf3ba00 con 0x23d63580 -3074> 2014-11-03 11:37:54.914247 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.113 192.168.5.9:6821/5555 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x10a86600 con 0x23d63420 -3073> 2014-11-03 11:37:54.914268 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.113 192.168.5.9:6829/5555 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2438f200 con 0x23d632c0 -3072> 2014-11-03 11:37:54.914289 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.81 192.168.5.6:6863/27170 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2d164600 con 0x23d63160 -3071> 2014-11-03 11:37:54.914310 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.81 192.168.5.6:6864/27170 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x242e2c00 con 0x23d63000 -3070> 2014-11-03 11:37:54.914330 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.124 192.168.5.9:6865/28288 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x27783600 con 0x23ea0dc0 -3069> 2014-11-03 11:37:54.914351 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.124 192.168.5.9:6866/28288 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x242e2800 con 0x23ea0c60 -3068> 2014-11-03 11:37:54.914371 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.102 192.168.5.8:6812/21153 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2d344600 con 0x23ea0b00 -3067> 2014-11-03 11:37:54.914393 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.102 192.168.5.8:6814/21153 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xc2d1600 con 0x23ea09a0 -3066> 2014-11-03 11:37:54.914413 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.123 192.168.5.9:6861/22948 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x8664000 con 0x23ea0840 -3065> 2014-11-03 11:37:54.914449 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.123 192.168.5.9:6862/22948 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xaf2e000 con 0x23ea06e0 -3064> 2014-11-03 11:37:54.914472 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.42 192.168.5.4:6808/9946 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x54a3c00 con 0x23ea0580 -3063> 2014-11-03 11:37:54.914493 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.42 192.168.5.4:6809/9946 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23ff5600 con 0x23ea0420 -3062> 2014-11-03 11:37:54.914513 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.13 192.168.5.1:6840/22281 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23561600 con 0x23ea02c0 -3061> 2014-11-03 11:37:54.914535 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.13 192.168.5.1:6852/22281 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x73d3600 con 0x23ea0160 -3060> 2014-11-03 11:37:54.914556 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.48 192.168.5.4:6837/17727 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23f8ca00 con 0x23ea0000 -3059> 2014-11-03 11:37:54.914578 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.48 192.168.5.4:6839/17727 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23937600 con 0x2387ddc0 -3058> 2014-11-03 11:37:54.914598 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.30 192.168.5.3:6818/11504 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2de29800 con 0x2387dc60 -3057> 2014-11-03 11:37:54.914619 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.30 192.168.5.3:6820/11504 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xf637e00 con 0x2387db00 -3056> 2014-11-03 11:37:54.914641 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.83 192.168.5.6:6872/4005 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xecb2c00 con 0x2387d9a0 -3055> 2014-11-03 11:37:54.914662 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.83 192.168.5.6:6873/4005 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x70a2600 con 0x2387d840 -3054> 2014-11-03 11:37:54.914681 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.105 192.168.5.8:6841/6777 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x229e7a00 con 0x2387d6e0 -3053> 2014-11-03 11:37:54.914703 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.105 192.168.5.8:6842/6777 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2de29400 con 0x2387d580 -3052> 2014-11-03 11:37:54.914723 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.99 192.168.5.8:6831/26930 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x80ff600 con 0x2387d420 -3051> 2014-11-03 11:37:54.914745 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.99 192.168.5.8:6832/26930 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x328e600 con 0x2387d2c0 -3050> 2014-11-03 11:37:54.914765 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.35 192.168.5.3:6847/6455 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2295ac00 con 0x2387d160 -3049> 2014-11-03 11:37:54.914788 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.35 192.168.5.3:6848/6455 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x26fb1c00 con 0x2387d000 -3048> 2014-11-03 11:37:54.914809 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.46 192.168.5.4:6827/7678 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x25e25600 con 0x245abc60 -3047> 2014-11-03 11:37:54.914829 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.46 192.168.5.4:6828/7678 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x1adc4e00 con 0x245abb00 -3046> 2014-11-03 11:37:54.914848 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.50 192.168.5.4:6848/26902 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23d2b600 con 0x245ab9a0 -3045> 2014-11-03 11:37:54.914869 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.50 192.168.5.4:6849/26902 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x17cac600 con 0x245ab840 -3044> 2014-11-03 11:37:54.914891 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.118 192.168.5.9:6839/23392 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23821400 con 0x245ab6e0 -3043> 2014-11-03 11:37:54.914912 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.118 192.168.5.9:6840/23392 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x26a9f800 con 0x245ab580 -3042> 2014-11-03 11:37:54.914932 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.68 192.168.5.5:6867/8133 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x261a4600 con 0x245ab420 -3041> 2014-11-03 11:37:54.914954 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.68 192.168.5.5:6868/8133 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xc02e600 con 0x245ab2c0 -3040> 2014-11-03 11:37:54.914974 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.104 192.168.5.8:6843/2576 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x187f4c00 con 0x245ab160 -3039> 2014-11-03 11:37:54.914995 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.104 192.168.5.8:6844/2576 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x229e7600 con 0x245ab000 -3038> 2014-11-03 11:37:54.915014 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.125 192.168.5.9:6871/31543 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2d329600 con 0x2416edc0 -3037> 2014-11-03 11:37:54.915034 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.125 192.168.5.9:6872/31543 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x6023400 con 0x2416ec60 -3036> 2014-11-03 11:37:54.915053 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.12 192.168.5.1:6829/19844 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xddd0c00 con 0x2416eb00 -3035> 2014-11-03 11:37:54.915075 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.12 192.168.5.1:6847/19844 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x3a32600 con 0x2416e9a0 -3034> 2014-11-03 11:37:54.915094 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.116 192.168.5.9:6803/6451 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x26a9f200 con 0x2416e840 -3033> 2014-11-03 11:37:54.915124 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.116 192.168.5.9:6828/6451 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x24d78c00 con 0x2416e6e0 -3032> 2014-11-03 11:37:54.915144 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.109 192.168.5.8:6860/25883 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23a41600 con 0x2416e580 -3031> 2014-11-03 11:37:54.915165 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.109 192.168.5.8:6861/25883 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xf637600 con 0x2416e420 -3030> 2014-11-03 11:37:54.915185 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.108 192.168.5.8:6820/21567 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x24092600 con 0x2416e2c0 -3029> 2014-11-03 11:37:54.915205 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.108 192.168.5.8:6857/21567 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2386cc00 con 0x2416e160 -3028> 2014-11-03 11:37:54.915226 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.44 192.168.5.4:6817/27523 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x256b9000 con 0x2416e000 -3027> 2014-11-03 11:37:54.915248 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.44 192.168.5.4:6818/27523 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x42a3600 con 0x24448420 -3026> 2014-11-03 11:37:54.915268 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.49 192.168.5.4:6843/21769 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x1474a600 con 0x244482c0 -3025> 2014-11-03 11:37:54.915287 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.49 192.168.5.4:6844/21769 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xc02ea00 con 0x23eb86e0 -3024> 2014-11-03 11:37:54.915312 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.69 192.168.5.5:6872/11297 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xe5b5600 con 0x23eb8580 -3023> 2014-11-03 11:37:54.915333 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.69 192.168.5.5:6873/11297 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x22a35c00 con 0x23eb8420 -3022> 2014-11-03 11:37:54.915353 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.37 192.168.5.3:6858/13652 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x26fb1600 con 0x2387fdc0 -3021> 2014-11-03 11:37:54.915374 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.37 192.168.5.3:6859/13652 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23250c00 con 0x2387fc60 -3020> 2014-11-03 11:37:54.915400 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.33 192.168.5.3:6840/29378 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23250600 con 0x2387fb00 -3019> 2014-11-03 11:37:54.915420 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.33 192.168.5.3:6841/29378 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2a2e0400 con 0x2387f9a0 -3018> 2014-11-03 11:37:54.915440 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.112 192.168.5.9:6810/24552 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x187f4a00 con 0x2387f840 -3017> 2014-11-03 11:37:54.915461 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.112 192.168.5.9:6811/24552 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2893d200 con 0x2387f6e0 -3016> 2014-11-03 11:37:54.915481 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.7 192.168.5.1:6887/3511 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xaf3bc00 con 0x233da000 -3015> 2014-11-03 11:37:54.915502 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.7 192.168.5.1:6888/3511 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xa1a7800 con 0x233da2c0 -3014> 2014-11-03 11:37:54.915522 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.43 192.168.5.4:6813/19461 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23482600 con 0x233da160 -3013> 2014-11-03 11:37:54.915541 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.43 192.168.5.4:6814/19461 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23f8c600 con 0x233da6e0 -3012> 2014-11-03 11:37:54.915566 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.6 192.168.5.1:6882/1181 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x22d40600 con 0x233da580 -3011> 2014-11-03 11:37:54.915587 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.6 192.168.5.1:6883/1181 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x6315c00 con 0x233da420 -3010> 2014-11-03 11:37:54.915607 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.32 192.168.5.3:6830/25327 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x1ae44c00 con 0x233dac60 -3009> 2014-11-03 11:37:54.915627 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.32 192.168.5.3:6831/25327 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x286bb600 con 0x233dab00 -3008> 2014-11-03 11:37:54.915650 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.103 192.168.5.8:6827/29722 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x328ec00 con 0x233da9a0 -3007> 2014-11-03 11:37:54.915671 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.103 192.168.5.8:6830/29722 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2d344c00 con 0x233da840 -3006> 2014-11-03 11:37:54.915692 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.114 192.168.5.9:6817/17408 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x1a22c600 con 0x233dadc0 -3005> 2014-11-03 11:37:54.915713 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.114 192.168.5.9:6818/17408 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x1a22cc00 con 0x233d26e0 -3004> 2014-11-03 11:37:54.915737 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.55 192.168.5.4:6872/12368 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x9591600 con 0x233d2580 -3003> 2014-11-03 11:37:54.915759 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.55 192.168.5.4:6873/12368 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x25835200 con 0x233d2420 -3002> 2014-11-03 11:37:54.915779 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.66 192.168.5.5:6853/919 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x261a4c00 con 0x233d22c0 -3001> 2014-11-03 11:37:54.915811 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.66 192.168.5.5:6859/919 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x22a35600 con 0x233d2160 -3000> 2014-11-03 11:37:54.915832 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.97 192.168.5.7:6862/21285 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xf637c00 con 0x233d2000 -2999> 2014-11-03 11:37:54.915853 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.97 192.168.5.7:6863/21285 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2d329800 con 0x233d2dc0 -2998> 2014-11-03 11:37:54.915872 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.31 192.168.5.3:6826/20618 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x405b600 con 0x233d2c60 -2997> 2014-11-03 11:37:54.915893 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.31 192.168.5.3:6827/20618 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2295a600 con 0x233d2b00 -2996> 2014-11-03 11:37:54.915916 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.90 192.168.5.7:6832/27918 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xc2d1c00 con 0x233d29a0 -2995> 2014-11-03 11:37:54.915937 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.90 192.168.5.7:6833/27918 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x2576cc00 con 0x233d2840 -2994> 2014-11-03 11:37:54.915957 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.45 192.168.5.4:6822/1781 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x54a3600 con 0x2342c9a0 -2993> 2014-11-03 11:37:54.915978 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.45 192.168.5.4:6823/1781 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x23482c00 con 0x2342c840 -2992> 2014-11-03 11:37:54.916000 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.107 192.168.5.8:6839/17474 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x24092c00 con 0x2342c6e0 -2991> 2014-11-03 11:37:54.916020 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.107 192.168.5.8:6855/17474 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x5d49600 con 0x2342c580 -2990> 2014-11-03 11:37:54.916040 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.20 192.168.5.2:6834/4963 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x78cbc00 con 0x2342c420 -2989> 2014-11-03 11:37:54.916063 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.20 192.168.5.2:6837/4963 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x3a32c00 con 0x2342c2c0 -2988> 2014-11-03 11:37:54.916087 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.23 192.168.5.2:6852/14826 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x78cb600 con 0x2342c160 -2987> 2014-11-03 11:37:54.916108 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.23 192.168.5.2:6853/14826 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x6526600 con 0x2342c000 -2986> 2014-11-03 11:37:54.916128 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.4 192.168.5.1:6869/31636 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xa1a7400 con 0x2445ac60 -2985> 2014-11-03 11:37:54.916147 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.28 192.168.5.3:6813/13976 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x1ae44600 con 0x2445ab00 -2984> 2014-11-03 11:37:54.916167 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.28 192.168.5.3:6817/13976 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x73d3a00 con 0x2445a9a0 -2983> 2014-11-03 11:37:54.916187 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.75 192.168.5.6:6832/12612 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x42a3c00 con 0x2445a840 -2982> 2014-11-03 11:37:54.916207 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.75 192.168.5.6:6833/12612 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x619f600 con 0x2445a6e0 -2981> 2014-11-03 11:37:54.916226 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.110 192.168.5.8:6863/29518 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0xedfd600 con 0x2445a580 -2980> 2014-11-03 11:37:54.916258 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.110 192.168.5.8:6864/29518 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x25bd9600 con 0x2445a420 -2979> 2014-11-03 11:37:54.917978 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.121 192.168.5.9:6850/12205 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x24d78600 con 0x2445a000 -2978> 2014-11-03 11:37:54.918005 7f290982e700 1 -- 192.168.5.2:0/7606 <== osd.121 192.168.5.9:6851/12205 12 ==== osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.908509) v2 ==== 47+0+0 (1718769406 0 0) 0x27783c00 con 0x2445adc0 -2977> 2014-11-03 11:37:54.926733 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x25592b80 sd=537 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bc6e0).accept sd=537 192.168.5.2:42968/0 -2976> 2014-11-03 11:37:54.927696 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419964 192.168.5.2:0/1003767 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x24d78c00 con 0x1c9bc6e0 -2975> 2014-11-03 11:37:54.928387 7f28fb812700 0 osd.21 105950 do_command r=0 -2974> 2014-11-03 11:37:54.928425 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003767 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x2b23e1c0 con 0x1c9bc6e0 -2973> 2014-11-03 11:37:54.939026 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419964 192.168.5.2:0/1003767 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0xf637600 con 0x1c9bc6e0 -2972> 2014-11-03 11:37:54.939780 7f28fb812700 0 osd.21 105950 do_command r=0 -2971> 2014-11-03 11:37:54.939796 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003767 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x260e1380 con 0x1c9bc6e0 -2970> 2014-11-03 11:37:54.941261 7f28f4ad6700 1 -- 192.168.5.2:6803/7606 <== osd.89 192.168.5.7:6812/24271 45 ==== osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1202+0+0 (2770770704 0 0) 0x24450400 con 0x22fe06e0 -2969> 2014-11-03 11:37:54.941294 7f28f4ad6700 5 -- op tracker -- seq: 3156, time: 2014-11-03 11:37:54.941158, event: header_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2968> 2014-11-03 11:37:54.941311 7f28f4ad6700 5 -- op tracker -- seq: 3156, time: 2014-11-03 11:37:54.941160, event: throttled, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2967> 2014-11-03 11:37:54.941323 7f28f4ad6700 5 -- op tracker -- seq: 3156, time: 2014-11-03 11:37:54.941244, event: all_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2966> 2014-11-03 11:37:54.941338 7f28f4ad6700 5 -- op tracker -- seq: 3156, time: 0.000000, event: dispatched, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2965> 2014-11-03 11:37:54.941394 7f28fd816700 5 -- op tracker -- seq: 3156, time: 2014-11-03 11:37:54.941394, event: reached_pg, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2964> 2014-11-03 11:37:54.941426 7f28fd816700 5 -- op tracker -- seq: 3156, time: 2014-11-03 11:37:54.941426, event: started, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2963> 2014-11-03 11:37:54.941447 7f28fd816700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6812/24271 -- osd_sub_op_reply(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] ack, result = 0) v2 -- ?+1 0x23e43600 con 0x22fe06e0 -2962> 2014-11-03 11:37:54.941472 7f28fd816700 5 -- op tracker -- seq: 3156, time: 2014-11-03 11:37:54.941472, event: done, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2961> 2014-11-03 11:37:54.943133 7f28f4ad6700 1 -- 192.168.5.2:6803/7606 <== osd.89 192.168.5.7:6812/24271 46 ==== osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1202+0+0 (2566622843 0 0) 0x2444f800 con 0x22fe06e0 -2960> 2014-11-03 11:37:54.943160 7f28f4ad6700 5 -- op tracker -- seq: 3157, time: 2014-11-03 11:37:54.943062, event: header_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2959> 2014-11-03 11:37:54.943173 7f28f4ad6700 5 -- op tracker -- seq: 3157, time: 2014-11-03 11:37:54.943063, event: throttled, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2958> 2014-11-03 11:37:54.943181 7f28f4ad6700 5 -- op tracker -- seq: 3157, time: 2014-11-03 11:37:54.943123, event: all_read, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2957> 2014-11-03 11:37:54.943189 7f28f4ad6700 5 -- op tracker -- seq: 3157, time: 0.000000, event: dispatched, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2956> 2014-11-03 11:37:54.943275 7f290001b700 5 -- op tracker -- seq: 3157, time: 2014-11-03 11:37:54.943274, event: reached_pg, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2955> 2014-11-03 11:37:54.943326 7f290001b700 5 -- op tracker -- seq: 3157, time: 2014-11-03 11:37:54.943326, event: started, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2954> 2014-11-03 11:37:54.943340 7f290001b700 5 -- op tracker -- seq: 3157, time: 2014-11-03 11:37:54.943340, event: done, op: osd_sub_op(unknown.0.0:0 26.63a 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) -2953> 2014-11-03 11:37:54.945249 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003767 pipe(0x25592b80 sd=537 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc6e0).reader couldn't read tag, (0) Success -2952> 2014-11-03 11:37:54.945295 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003767 pipe(0x25592b80 sd=537 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc6e0).fault (0) Success -2951> 2014-11-03 11:37:54.945390 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bc6e0 session 0xa758000 -2950> 2014-11-03 11:37:54.949648 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.31 192.168.5.3:0/20618 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.948753) v2 ==== 47+0+0 (1930073259 0 0) 0x2340dc00 con 0x24925840 -2949> 2014-11-03 11:37:54.949691 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.948753) v2 -- ?+0 0x23cbec00 con 0x24925840 -2948> 2014-11-03 11:37:54.949760 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.31 192.168.5.3:0/20618 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:54.948753) v2 ==== 47+0+0 (1930073259 0 0) 0x2318ae00 con 0x24229580 -2947> 2014-11-03 11:37:54.949830 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:54.948753) v2 -- ?+0 0xaf2e400 con 0x24229580 -2946> 2014-11-03 11:37:54.952958 7f291450b700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.952957, event: journaled_completion_queued, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -2945> 2014-11-03 11:37:54.953128 7f2914d0c700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.953128, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -2944> 2014-11-03 11:37:54.953176 7f2914d0c700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.953176, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -2943> 2014-11-03 11:37:54.953232 7f2914d0c700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.953232, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -2942> 2014-11-03 11:37:54.953270 7f2912507700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.953270, event: sub_op_applied, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -2941> 2014-11-03 11:37:54.953345 7f2912507700 5 -- op tracker -- seq: 3125, time: 2014-11-03 11:37:54.953345, event: done, op: osd_sub_op(client.7868792.0:22003489 6.6fe c77a86fe/rbd_data.4d4ddc4447765.00000000000000b6/head//6 [] v 105950'9124377 snapset=0=[]:[] snapc=0=[]) -2940> 2014-11-03 11:37:54.954225 7f2911d06700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:54.954225, event: commit_sent, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -2939> 2014-11-03 11:37:54.954267 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] ondisk, result = 0) v2 -- ?+0 0x239268c0 con 0x22fefdc0 -2938> 2014-11-03 11:37:54.954348 7f2914d0c700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.954348, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -2937> 2014-11-03 11:37:54.971062 7f291450b700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.971061, event: journaled_completion_queued, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -2936> 2014-11-03 11:37:54.971136 7f291450b700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.971135, event: journaled_completion_queued, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -2935> 2014-11-03 11:37:54.971187 7f291450b700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.971186, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -2934> 2014-11-03 11:37:54.971250 7f291450b700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.971250, event: journaled_completion_queued, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -2933> 2014-11-03 11:37:54.971234 7f2911d06700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:54.971234, event: commit_sent, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -2932> 2014-11-03 11:37:54.971316 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6871/4005 -- osd_sub_op_reply(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23925dc0 con 0x22fef2c0 -2931> 2014-11-03 11:37:54.971368 7f2911d06700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:54.971367, event: commit_sent, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -2930> 2014-11-03 11:37:54.971398 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23925b00 con 0x230c2c60 -2929> 2014-11-03 11:37:54.971429 7f2911d06700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:54.971429, event: commit_sent, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -2928> 2014-11-03 11:37:54.971455 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x269af000 con 0x22fefdc0 -2927> 2014-11-03 11:37:54.971494 7f2911d06700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:54.971494, event: commit_sent, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -2926> 2014-11-03 11:37:54.971520 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269b08c0 con 0x22fbcb00 -2925> 2014-11-03 11:37:54.973339 7f2914d0c700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:54.973339, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -2924> 2014-11-03 11:37:54.973413 7f2914d0c700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:54.973413, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -2923> 2014-11-03 11:37:54.973457 7f2914d0c700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:54.973457, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -2922> 2014-11-03 11:37:54.973494 7f2914d0c700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:54.973494, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -2921> 2014-11-03 11:37:54.973538 7f2914d0c700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:54.973538, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -2920> 2014-11-03 11:37:54.979128 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(35ee8eac/rbd_data.885435b2bbeeb.00000000000015db/head//6),PushReplyOp(8fcf46ac/rbd_data.3e5d6a2ae8944a.00000000000018b7/head//6)]) v2 -- ?+0 0xa6b5400 con 0x2301f580 -2919> 2014-11-03 11:37:55.001378 7f291450b700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:55.001377, event: journaled_completion_queued, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -2918> 2014-11-03 11:37:55.001427 7f291450b700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:55.001427, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -2917> 2014-11-03 11:37:55.001444 7f291450b700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:55.001444, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -2916> 2014-11-03 11:37:55.001482 7f291450b700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:55.001482, event: journaled_completion_queued, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -2915> 2014-11-03 11:37:55.001466 7f2911d06700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:55.001466, event: commit_sent, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -2914> 2014-11-03 11:37:55.001513 7f291450b700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:55.001513, event: journaled_completion_queued, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -2913> 2014-11-03 11:37:55.001527 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- osd_sub_op_reply(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269af840 con 0x23000b00 -2912> 2014-11-03 11:37:55.001574 7f2911d06700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:55.001574, event: commit_sent, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -2911> 2014-11-03 11:37:55.001602 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269b0080 con 0x22fbc420 -2910> 2014-11-03 11:37:55.001635 7f2911d06700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:55.001635, event: commit_sent, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -2909> 2014-11-03 11:37:55.001739 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269af580 con 0x22fbc420 -2908> 2014-11-03 11:37:55.001775 7f2911d06700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:55.001775, event: commit_sent, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -2907> 2014-11-03 11:37:55.001797 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- osd_sub_op_reply(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] ondisk, result = 0) v2 -- ?+0 0x269b0600 con 0x23000b00 -2906> 2014-11-03 11:37:55.001824 7f2911d06700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:55.001824, event: commit_sent, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -2905> 2014-11-03 11:37:55.001848 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] ondisk, result = 0) v2 -- ?+0 0x269afb00 con 0x22fbcb00 -2904> 2014-11-03 11:37:55.002001 7f290c834700 1 -- 192.168.5.2:6802/7606 <== mon.0 192.168.5.201:6789/0 45 ==== pg_stats_ack(4 pgs tid 6) v1 ==== 120+0+0 (2625101931 0 0) 0x23299000 con 0x22c29160 -2903> 2014-11-03 11:37:55.006266 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(a72a1eac/rbd_data.885435b2bbeeb.00000000000026c4/head//6),PushReplyOp(959fc6ac/rbd_data.4eeda241ca92c8.0000000000003992/head//6)]) v2 -- ?+0 0x2310a800 con 0x2301f580 -2902> 2014-11-03 11:37:55.006616 7f2912507700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:55.006615, event: sub_op_applied, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -2901> 2014-11-03 11:37:55.006667 7f2912507700 5 -- op tracker -- seq: 3128, time: 2014-11-03 11:37:55.006667, event: done, op: osd_sub_op(client.7867846.0:19188828 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653966 snapset=0=[]:[] snapc=0=[]) -2900> 2014-11-03 11:37:55.006851 7f2912507700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:55.006850, event: sub_op_applied, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -2899> 2014-11-03 11:37:55.006896 7f2912507700 5 -- op tracker -- seq: 3129, time: 2014-11-03 11:37:55.006896, event: done, op: osd_sub_op(client.7867846.0:19188829 6.a72 70278a72/rbd_data.4d61bf45a3ae1e.00000000000000d0/head//6 [] v 105950'6653967 snapset=0=[]:[] snapc=0=[]) -2898> 2014-11-03 11:37:55.007050 7f2912507700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:55.007050, event: sub_op_applied, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -2897> 2014-11-03 11:37:55.007121 7f2912507700 5 -- op tracker -- seq: 3130, time: 2014-11-03 11:37:55.007121, event: done, op: osd_sub_op(client.7869095.0:16207383 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973394 snapset=0=[]:[] snapc=0=[]) -2896> 2014-11-03 11:37:55.007198 7f2912507700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:55.007198, event: sub_op_applied, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -2895> 2014-11-03 11:37:55.007239 7f2912507700 5 -- op tracker -- seq: 3131, time: 2014-11-03 11:37:55.007239, event: done, op: osd_sub_op(client.7869095.0:16207384 6.9f0 5325c9f0/rbd_data.4f51ee466e8d14.000000000000025d/head//6 [] v 105950'6973395 snapset=0=[]:[] snapc=0=[]) -2894> 2014-11-03 11:37:55.007523 7f2912507700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:55.007523, event: op_applied, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -2893> 2014-11-03 11:37:55.007598 7f2912507700 5 -- op tracker -- seq: 3132, time: 2014-11-03 11:37:55.007598, event: done, op: osd_op(client.9598152.0:459476 rbd_data.92d20c2ae8944a.0000000000000084 [set-alloc-hint object_size 4194304 write_size 4194304,write 3096576~4096] 24.8f258d3e ack+ondisk+write e105950) -2892> 2014-11-03 11:37:55.009118 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.83 192.168.5.6:0/4005 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.008237) v2 ==== 47+0+0 (133995945 0 0) 0x4d90600 con 0x2445c2c0 -2891> 2014-11-03 11:37:55.009177 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.008237) v2 -- ?+0 0x2340dc00 con 0x2445c2c0 -2890> 2014-11-03 11:37:55.009272 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.83 192.168.5.6:0/4005 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.008237) v2 ==== 47+0+0 (133995945 0 0) 0x399fc00 con 0x23ea4420 -2889> 2014-11-03 11:37:55.009338 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.008237) v2 -- ?+0 0x2318ae00 con 0x23ea4420 -2888> 2014-11-03 11:37:55.030771 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 176 ==== osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2132389420 0 3511520646) 0x29056400 con 0x22fefdc0 -2887> 2014-11-03 11:37:55.030841 7f28f36c2700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.030438, event: header_read, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2886> 2014-11-03 11:37:55.030905 7f28f36c2700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.030441, event: throttled, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2885> 2014-11-03 11:37:55.030929 7f28f36c2700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.030663, event: all_read, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2884> 2014-11-03 11:37:55.030952 7f28f36c2700 5 -- op tracker -- seq: 3158, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2883> 2014-11-03 11:37:55.031054 7f28fe818700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.031053, event: reached_pg, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2882> 2014-11-03 11:37:55.031103 7f28fe818700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.031103, event: started, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2881> 2014-11-03 11:37:55.031298 7f28fe818700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.031298, event: started, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2880> 2014-11-03 11:37:55.031350 7f28fe818700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.031350, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2879> 2014-11-03 11:37:55.031477 7f2914d0c700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.031477, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2878> 2014-11-03 11:37:55.031848 7f291450b700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.031847, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2877> 2014-11-03 11:37:55.031928 7f2911d06700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.031927, event: commit_sent, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2876> 2014-11-03 11:37:55.031967 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x922cb00 con 0x22fefdc0 -2875> 2014-11-03 11:37:55.043872 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 49 ==== MOSDPGPush(6.6ac 105950 [PushOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6, version: 105946'9806072, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6@105946'9806072, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (3505366501 0 0) 0x2310a800 con 0x2301f580 -2874> 2014-11-03 11:37:55.043987 7f28f2db9700 5 -- op tracker -- seq: 3159, time: 2014-11-03 11:37:55.023068, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6, version: 105946'9806072, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6@105946'9806072, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2873> 2014-11-03 11:37:55.044028 7f28f2db9700 5 -- op tracker -- seq: 3159, time: 2014-11-03 11:37:55.023070, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6, version: 105946'9806072, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6@105946'9806072, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2872> 2014-11-03 11:37:55.044056 7f28f2db9700 5 -- op tracker -- seq: 3159, time: 2014-11-03 11:37:55.043858, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6, version: 105946'9806072, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6@105946'9806072, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2871> 2014-11-03 11:37:55.044076 7f28f2db9700 5 -- op tracker -- seq: 3159, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6, version: 105946'9806072, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6@105946'9806072, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2870> 2014-11-03 11:37:55.044154 7f290081c700 5 -- op tracker -- seq: 3159, time: 2014-11-03 11:37:55.044154, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6, version: 105946'9806072, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6@105946'9806072, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2869> 2014-11-03 11:37:55.044368 7f290081c700 5 -- op tracker -- seq: 3159, time: 2014-11-03 11:37:55.044368, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6, version: 105946'9806072, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6@105946'9806072, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2868> 2014-11-03 11:37:55.089016 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 50 ==== MOSDPGPush(6.6ac 105950 [PushOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6, version: 105946'9806081, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6@105946'9806081, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (975280927 0 0) 0xa6b5400 con 0x2301f580 -2867> 2014-11-03 11:37:55.089116 7f28f2db9700 5 -- op tracker -- seq: 3160, time: 2014-11-03 11:37:55.067603, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6, version: 105946'9806081, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6@105946'9806081, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2866> 2014-11-03 11:37:55.089167 7f28f2db9700 5 -- op tracker -- seq: 3160, time: 2014-11-03 11:37:55.067608, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6, version: 105946'9806081, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6@105946'9806081, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2865> 2014-11-03 11:37:55.089196 7f28f2db9700 5 -- op tracker -- seq: 3160, time: 2014-11-03 11:37:55.088990, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6, version: 105946'9806081, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6@105946'9806081, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2864> 2014-11-03 11:37:55.089223 7f28f2db9700 5 -- op tracker -- seq: 3160, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6, version: 105946'9806081, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6@105946'9806081, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2863> 2014-11-03 11:37:55.089341 7f28fe017700 5 -- op tracker -- seq: 3160, time: 2014-11-03 11:37:55.089341, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6, version: 105946'9806081, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6@105946'9806081, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2862> 2014-11-03 11:37:55.089604 7f28fe017700 5 -- op tracker -- seq: 3160, time: 2014-11-03 11:37:55.089604, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6, version: 105946'9806081, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6@105946'9806081, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2861> 2014-11-03 11:37:55.100602 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.71 192.168.5.6:0/14378 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.099793) v2 ==== 47+0+0 (2942352870 0 0) 0x286bba00 con 0x234ed2c0 -2860> 2014-11-03 11:37:55.100664 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.099793) v2 -- ?+0 0x399fc00 con 0x234ed2c0 -2859> 2014-11-03 11:37:55.100655 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.71 192.168.5.6:0/14378 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.099793) v2 ==== 47+0+0 (2942352870 0 0) 0x13def400 con 0x233352c0 -2858> 2014-11-03 11:37:55.100941 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/14378 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.099793) v2 -- ?+0 0x4d90600 con 0x233352c0 -2857> 2014-11-03 11:37:55.118716 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 51 ==== MOSDPGPush(6.6ac 105950 [PushOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6, version: 105946'9806082, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6@105946'9806082, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (1582448614 0 0) 0xcd57000 con 0x2301f580 -2856> 2014-11-03 11:37:55.118799 7f28f2db9700 5 -- op tracker -- seq: 3161, time: 2014-11-03 11:37:55.101141, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6, version: 105946'9806082, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6@105946'9806082, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2855> 2014-11-03 11:37:55.118834 7f28f2db9700 5 -- op tracker -- seq: 3161, time: 2014-11-03 11:37:55.101143, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6, version: 105946'9806082, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6@105946'9806082, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2854> 2014-11-03 11:37:55.118880 7f28f2db9700 5 -- op tracker -- seq: 3161, time: 2014-11-03 11:37:55.118692, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6, version: 105946'9806082, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6@105946'9806082, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2853> 2014-11-03 11:37:55.118907 7f28f2db9700 5 -- op tracker -- seq: 3161, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6, version: 105946'9806082, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6@105946'9806082, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2852> 2014-11-03 11:37:55.119057 7f290081c700 5 -- op tracker -- seq: 3161, time: 2014-11-03 11:37:55.119056, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6, version: 105946'9806082, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6@105946'9806082, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2851> 2014-11-03 11:37:55.119237 7f290081c700 5 -- op tracker -- seq: 3161, time: 2014-11-03 11:37:55.119237, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6, version: 105946'9806082, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6@105946'9806082, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2850> 2014-11-03 11:37:55.123548 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.81 192.168.5.6:0/27170 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.121795) v2 ==== 47+0+0 (887511471 0 0) 0x2579a000 con 0x24925b00 -2849> 2014-11-03 11:37:55.123617 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.121795) v2 -- ?+0 0x13def400 con 0x24925b00 -2848> 2014-11-03 11:37:55.123674 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.81 192.168.5.6:0/27170 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.121795) v2 ==== 47+0+0 (887511471 0 0) 0x2470a200 con 0x24229840 -2847> 2014-11-03 11:37:55.123740 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.121795) v2 -- ?+0 0x286bba00 con 0x24229840 -2846> 2014-11-03 11:37:55.138348 7f28f2db9700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 52 ==== MOSDPGPush(6.6ac 105950 [PushOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6, version: 105946'9806084, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6@105946'9806084, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4196581+0+0 (2583696554 0 0) 0x27256e00 con 0x2301f580 -2845> 2014-11-03 11:37:55.138420 7f28f2db9700 5 -- op tracker -- seq: 3162, time: 2014-11-03 11:37:55.118996, event: header_read, op: MOSDPGPush(6.6ac 105950 [PushOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6, version: 105946'9806084, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6@105946'9806084, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2844> 2014-11-03 11:37:55.138461 7f28f2db9700 5 -- op tracker -- seq: 3162, time: 2014-11-03 11:37:55.119003, event: throttled, op: MOSDPGPush(6.6ac 105950 [PushOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6, version: 105946'9806084, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6@105946'9806084, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2843> 2014-11-03 11:37:55.138492 7f28f2db9700 5 -- op tracker -- seq: 3162, time: 2014-11-03 11:37:55.138324, event: all_read, op: MOSDPGPush(6.6ac 105950 [PushOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6, version: 105946'9806084, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6@105946'9806084, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2842> 2014-11-03 11:37:55.138522 7f28f2db9700 5 -- op tracker -- seq: 3162, time: 0.000000, event: dispatched, op: MOSDPGPush(6.6ac 105950 [PushOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6, version: 105946'9806084, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6@105946'9806084, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2841> 2014-11-03 11:37:55.138642 7f28fe017700 5 -- op tracker -- seq: 3162, time: 2014-11-03 11:37:55.138642, event: reached_pg, op: MOSDPGPush(6.6ac 105950 [PushOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6, version: 105946'9806084, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6@105946'9806084, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2840> 2014-11-03 11:37:55.138827 7f28fe017700 5 -- op tracker -- seq: 3162, time: 2014-11-03 11:37:55.138827, event: done, op: MOSDPGPush(6.6ac 105950 [PushOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6, version: 105946'9806084, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6@105946'9806084, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2839> 2014-11-03 11:37:55.144616 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.13 192.168.5.1:0/22281 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.144167) v2 ==== 47+0+0 (3989687590 0 0) 0x2342e000 con 0x2445c420 -2838> 2014-11-03 11:37:55.144691 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/22281 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.144167) v2 -- ?+0 0x2579a000 con 0x2445c420 -2837> 2014-11-03 11:37:55.144744 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.13 192.168.5.1:0/22281 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.144167) v2 ==== 47+0+0 (3989687590 0 0) 0x2342ea00 con 0x23ea4580 -2836> 2014-11-03 11:37:55.144812 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/22281 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.144167) v2 -- ?+0 0x2470a200 con 0x23ea4580 -2835> 2014-11-03 11:37:55.174834 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- pg_trim(6.6ac to 105950'9806086 e105950) v1 -- ?+0 0xc157600 con 0x2301f580 -2834> 2014-11-03 11:37:55.180208 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(78d13eac/rbd_data.3ea57474ca1fcf.0000000000001521/head//6)]) v2 -- ?+0 0x244ab000 con 0x2301f580 -2833> 2014-11-03 11:37:55.200039 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x25592340 sd=563 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bc840).accept sd=563 192.168.5.2:42972/0 -2832> 2014-11-03 11:37:55.200923 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450806 192.168.5.2:0/1003890 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2a6e8a00 con 0x1c9bc840 -2831> 2014-11-03 11:37:55.201568 7f28fb812700 0 osd.21 105950 do_command r=0 -2830> 2014-11-03 11:37:55.201606 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003890 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x2b23e1c0 con 0x1c9bc840 -2829> 2014-11-03 11:37:55.209925 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450806 192.168.5.2:0/1003890 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x1a327200 con 0x1c9bc840 -2828> 2014-11-03 11:37:55.210010 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(d62e3eac/rbd_data.4e480c2ae8944a.00000000000003a4/head//6),PushReplyOp(688c0eac/rbd_data.89cd895a911542.0000000000006e78/head//6),PushReplyOp(7666c6ac/rbd_data.89cd895a911542.00000000000094c3/head//6)]) v2 -- ?+0 0x2350a800 con 0x2301f580 -2827> 2014-11-03 11:37:55.211000 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.68 192.168.5.5:0/8133 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.209373) v2 ==== 47+0+0 (2080511755 0 0) 0x17ae8200 con 0x24321420 -2826> 2014-11-03 11:37:55.211047 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.68 192.168.5.5:0/8133 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.209373) v2 ==== 47+0+0 (2080511755 0 0) 0x2b1ffe00 con 0x24292420 -2825> 2014-11-03 11:37:55.211078 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/8133 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.209373) v2 -- ?+0 0x2342ea00 con 0x24321420 -2824> 2014-11-03 11:37:55.211241 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/8133 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.209373) v2 -- ?+0 0x2342e000 con 0x24292420 -2823> 2014-11-03 11:37:55.211699 7f28fb812700 0 osd.21 105950 do_command r=0 -2822> 2014-11-03 11:37:55.211726 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1003890 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x23416c40 con 0x1c9bc840 -2821> 2014-11-03 11:37:55.213900 7f2912507700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:55.213899, event: sub_op_applied, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -2820> 2014-11-03 11:37:55.213970 7f2912507700 5 -- op tracker -- seq: 3139, time: 2014-11-03 11:37:55.213970, event: done, op: osd_sub_op(client.7870481.0:43904207 24.787 f4144f87/rbd_data.57d78c2ae8944a.0000000000000223/head//24 [] v 105950'4953155 snapset=0=[]:[] snapc=0=[]) -2819> 2014-11-03 11:37:55.221177 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 102 ==== osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1914854432 0 3511520646) 0x2d47d400 con 0x22fbcb00 -2818> 2014-11-03 11:37:55.221247 7f28f6cf8700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.220976, event: header_read, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2817> 2014-11-03 11:37:55.221264 7f28f6cf8700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.220979, event: throttled, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2816> 2014-11-03 11:37:55.221278 7f28f6cf8700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.221154, event: all_read, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2815> 2014-11-03 11:37:55.221287 7f28f6cf8700 5 -- op tracker -- seq: 3163, time: 0.000000, event: dispatched, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2814> 2014-11-03 11:37:55.221410 7f290081c700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.221410, event: reached_pg, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2813> 2014-11-03 11:37:55.221465 7f290081c700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.221465, event: started, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2812> 2014-11-03 11:37:55.221504 7f2918d14700 5 osd.21 105950 tick -2811> 2014-11-03 11:37:55.221590 7f290081c700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.221590, event: started, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2810> 2014-11-03 11:37:55.221629 7f290081c700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.221629, event: commit_queued_for_journal_write, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2809> 2014-11-03 11:37:55.221785 7f2914d0c700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.221785, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2808> 2014-11-03 11:37:55.222473 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003890 pipe(0x25592340 sd=563 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc840).reader couldn't read tag, (0) Success -2807> 2014-11-03 11:37:55.222522 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1003890 pipe(0x25592340 sd=563 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc840).fault (0) Success -2806> 2014-11-03 11:37:55.222631 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bc840 session 0x251d4000 -2805> 2014-11-03 11:37:55.261024 7f291450b700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.261023, event: journaled_completion_queued, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2804> 2014-11-03 11:37:55.261087 7f2911d06700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.261087, event: commit_sent, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2803> 2014-11-03 11:37:55.261144 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] ondisk, result = 0) v2 -- ?+0 0x922c000 con 0x22fbcb00 -2802> 2014-11-03 11:37:55.269340 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 177 ==== osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3454866918 0 3511520646) 0xd4b8400 con 0x22fefdc0 -2801> 2014-11-03 11:37:55.269378 7f28f36c2700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269122, event: header_read, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2800> 2014-11-03 11:37:55.269398 7f28f36c2700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269125, event: throttled, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2799> 2014-11-03 11:37:55.269410 7f28f36c2700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269325, event: all_read, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2798> 2014-11-03 11:37:55.269418 7f28f36c2700 5 -- op tracker -- seq: 3164, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2797> 2014-11-03 11:37:55.269481 7f290201f700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269481, event: reached_pg, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2796> 2014-11-03 11:37:55.269519 7f290201f700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269519, event: started, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2795> 2014-11-03 11:37:55.269684 7f290201f700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269684, event: started, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2794> 2014-11-03 11:37:55.269725 7f290201f700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269725, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2793> 2014-11-03 11:37:55.269800 7f2914d0c700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.269800, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2792> 2014-11-03 11:37:55.270105 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 178 ==== osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (522004132 0 3511520646) 0x2bd4e000 con 0x22fefdc0 -2791> 2014-11-03 11:37:55.270251 7f28f36c2700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.270007, event: header_read, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2790> 2014-11-03 11:37:55.270266 7f28f36c2700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.270009, event: throttled, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2789> 2014-11-03 11:37:55.270279 7f28f36c2700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.270095, event: all_read, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2788> 2014-11-03 11:37:55.270305 7f28f36c2700 5 -- op tracker -- seq: 3165, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2787> 2014-11-03 11:37:55.270384 7f28ff81a700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.270384, event: reached_pg, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2786> 2014-11-03 11:37:55.270418 7f28ff81a700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.270418, event: started, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2785> 2014-11-03 11:37:55.270564 7f28ff81a700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.270563, event: started, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2784> 2014-11-03 11:37:55.270624 7f28ff81a700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.270624, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2783> 2014-11-03 11:37:55.271084 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 179 ==== osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1348065515 0 3511520646) 0x26c43800 con 0x22fefdc0 -2782> 2014-11-03 11:37:55.271126 7f28f36c2700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.270939, event: header_read, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2781> 2014-11-03 11:37:55.271145 7f28f36c2700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.270941, event: throttled, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2780> 2014-11-03 11:37:55.271158 7f28f36c2700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.271072, event: all_read, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2779> 2014-11-03 11:37:55.271188 7f28f36c2700 5 -- op tracker -- seq: 3166, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2778> 2014-11-03 11:37:55.271235 7f290201f700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.271235, event: reached_pg, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2777> 2014-11-03 11:37:55.271267 7f290201f700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.271267, event: started, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2776> 2014-11-03 11:37:55.271371 7f290201f700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.271371, event: started, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2775> 2014-11-03 11:37:55.271400 7f290201f700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.271400, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2774> 2014-11-03 11:37:55.272012 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 180 ==== osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3456909259 0 3511520646) 0x27999c00 con 0x22fefdc0 -2773> 2014-11-03 11:37:55.272068 7f28f36c2700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.271884, event: header_read, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2772> 2014-11-03 11:37:55.272103 7f28f36c2700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.271887, event: throttled, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2771> 2014-11-03 11:37:55.272112 7f28f36c2700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.271999, event: all_read, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2770> 2014-11-03 11:37:55.272120 7f28f36c2700 5 -- op tracker -- seq: 3167, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2769> 2014-11-03 11:37:55.272188 7f28ff81a700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.272188, event: reached_pg, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2768> 2014-11-03 11:37:55.272219 7f28ff81a700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.272219, event: started, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2767> 2014-11-03 11:37:55.272309 7f28ff81a700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.272309, event: started, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2766> 2014-11-03 11:37:55.272339 7f28ff81a700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.272339, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2765> 2014-11-03 11:37:55.272873 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 181 ==== osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1926768344 0 3511520646) 0xd4b6000 con 0x22fefdc0 -2764> 2014-11-03 11:37:55.272898 7f28f36c2700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.272743, event: header_read, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2763> 2014-11-03 11:37:55.272914 7f28f36c2700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.272746, event: throttled, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2762> 2014-11-03 11:37:55.272925 7f28f36c2700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.272863, event: all_read, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2761> 2014-11-03 11:37:55.272954 7f28f36c2700 5 -- op tracker -- seq: 3168, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2760> 2014-11-03 11:37:55.273025 7f290201f700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.273025, event: reached_pg, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2759> 2014-11-03 11:37:55.273058 7f290201f700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.273058, event: started, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2758> 2014-11-03 11:37:55.273146 7f290201f700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.273146, event: started, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2757> 2014-11-03 11:37:55.273174 7f290201f700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.273174, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2756> 2014-11-03 11:37:55.283721 7f28f3eca700 1 -- 192.168.5.2:6803/7606 <== osd.83 192.168.5.6:6871/4005 41 ==== osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (277788247 0 3511520646) 0x2493cc00 con 0x22fef2c0 -2755> 2014-11-03 11:37:55.283757 7f28f3eca700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.283623, event: header_read, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2754> 2014-11-03 11:37:55.283772 7f28f3eca700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.283625, event: throttled, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2753> 2014-11-03 11:37:55.283785 7f28f3eca700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.283704, event: all_read, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2752> 2014-11-03 11:37:55.283794 7f28f3eca700 5 -- op tracker -- seq: 3169, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2751> 2014-11-03 11:37:55.283869 7f28fe017700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.283869, event: reached_pg, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2750> 2014-11-03 11:37:55.283909 7f28fe017700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.283908, event: started, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2749> 2014-11-03 11:37:55.284045 7f28fe017700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.284045, event: started, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2748> 2014-11-03 11:37:55.284087 7f28fe017700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.284087, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2747> 2014-11-03 11:37:55.292132 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 44 ==== osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1586091762 0 3511520646) 0x26612800 con 0x230c2c60 -2746> 2014-11-03 11:37:55.292193 7f28ee16d700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.291974, event: header_read, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2745> 2014-11-03 11:37:55.292216 7f28ee16d700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.291982, event: throttled, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2744> 2014-11-03 11:37:55.292233 7f28ee16d700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.292112, event: all_read, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2743> 2014-11-03 11:37:55.292255 7f28ee16d700 5 -- op tracker -- seq: 3170, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2742> 2014-11-03 11:37:55.292316 7f28fd816700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.292315, event: reached_pg, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2741> 2014-11-03 11:37:55.292348 7f28fd816700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.292348, event: started, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2740> 2014-11-03 11:37:55.292516 7f28fd816700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.292515, event: started, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2739> 2014-11-03 11:37:55.292578 7f28fd816700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.292577, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2738> 2014-11-03 11:37:55.293344 7f291450b700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.293344, event: journaled_completion_queued, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2737> 2014-11-03 11:37:55.293389 7f2911d06700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.293388, event: commit_sent, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2736> 2014-11-03 11:37:55.293416 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] ondisk, result = 0) v2 -- ?+0 0x316d2600 con 0x22fefdc0 -2735> 2014-11-03 11:37:55.293421 7f2914d0c700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.293421, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2734> 2014-11-03 11:37:55.293459 7f2914d0c700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.293459, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2733> 2014-11-03 11:37:55.293488 7f2914d0c700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.293488, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2732> 2014-11-03 11:37:55.293518 7f2914d0c700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.293518, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2731> 2014-11-03 11:37:55.293545 7f2914d0c700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.293545, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2730> 2014-11-03 11:37:55.293575 7f2914d0c700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.293575, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2729> 2014-11-03 11:37:55.297269 7f291450b700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.297268, event: journaled_completion_queued, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2728> 2014-11-03 11:37:55.297303 7f291450b700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.297303, event: journaled_completion_queued, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2727> 2014-11-03 11:37:55.297301 7f2911d06700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.297301, event: commit_sent, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2726> 2014-11-03 11:37:55.297328 7f291450b700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.297328, event: journaled_completion_queued, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2725> 2014-11-03 11:37:55.297352 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b2e1340 con 0x22fefdc0 -2724> 2014-11-03 11:37:55.297360 7f291450b700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.297360, event: journaled_completion_queued, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2723> 2014-11-03 11:37:55.297376 7f291450b700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.297376, event: journaled_completion_queued, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2722> 2014-11-03 11:37:55.297380 7f2911d06700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.297380, event: commit_sent, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2721> 2014-11-03 11:37:55.297394 7f291450b700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.297394, event: journaled_completion_queued, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2720> 2014-11-03 11:37:55.297465 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] ondisk, result = 0) v2 -- ?+0 0x274188c0 con 0x22fefdc0 -2719> 2014-11-03 11:37:55.297489 7f2911d06700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.297489, event: commit_sent, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2718> 2014-11-03 11:37:55.297557 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] ondisk, result = 0) v2 -- ?+0 0x27417000 con 0x22fefdc0 -2717> 2014-11-03 11:37:55.297607 7f2911d06700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.297607, event: commit_sent, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2716> 2014-11-03 11:37:55.297650 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcfa340 con 0x22fefdc0 -2715> 2014-11-03 11:37:55.297682 7f2911d06700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.297682, event: commit_sent, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2714> 2014-11-03 11:37:55.297710 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6871/4005 -- osd_sub_op_reply(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1c580 con 0x22fef2c0 -2713> 2014-11-03 11:37:55.297744 7f2911d06700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.297743, event: commit_sent, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2712> 2014-11-03 11:37:55.297782 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1cdc0 con 0x230c2c60 -2711> 2014-11-03 11:37:55.303420 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.95 192.168.5.7:0/14919 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.301542) v2 ==== 47+0+0 (2057112744 0 0) 0x6425c00 con 0x24321000 -2710> 2014-11-03 11:37:55.303434 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.95 192.168.5.7:0/14919 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.301542) v2 ==== 47+0+0 (2057112744 0 0) 0x6425800 con 0x2484a580 -2709> 2014-11-03 11:37:55.303528 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/14919 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.301542) v2 -- ?+0 0x17ae8200 con 0x24321000 -2708> 2014-11-03 11:37:55.303594 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/14919 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.301542) v2 -- ?+0 0x2b1ffe00 con 0x2484a580 -2707> 2014-11-03 11:37:55.342422 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 182 ==== osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1557249913 0 3511520646) 0x2d47bc00 con 0x22fefdc0 -2706> 2014-11-03 11:37:55.342464 7f28f36c2700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342259, event: header_read, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2705> 2014-11-03 11:37:55.342489 7f28f36c2700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342262, event: throttled, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2704> 2014-11-03 11:37:55.342508 7f28f36c2700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342407, event: all_read, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2703> 2014-11-03 11:37:55.342536 7f28f36c2700 5 -- op tracker -- seq: 3171, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2702> 2014-11-03 11:37:55.342620 7f28ff81a700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342619, event: reached_pg, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2701> 2014-11-03 11:37:55.342654 7f28ff81a700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342654, event: started, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2700> 2014-11-03 11:37:55.342775 7f28ff81a700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342775, event: started, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2699> 2014-11-03 11:37:55.342818 7f28ff81a700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342818, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2698> 2014-11-03 11:37:55.342913 7f2914d0c700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.342913, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2697> 2014-11-03 11:37:55.343899 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 183 ==== osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3050077708 0 3511520646) 0x2d47b000 con 0x22fefdc0 -2696> 2014-11-03 11:37:55.343929 7f28f36c2700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.343821, event: header_read, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2695> 2014-11-03 11:37:55.343947 7f28f36c2700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.343822, event: throttled, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2694> 2014-11-03 11:37:55.343960 7f28f36c2700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.343891, event: all_read, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2693> 2014-11-03 11:37:55.343972 7f28f36c2700 5 -- op tracker -- seq: 3172, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2692> 2014-11-03 11:37:55.344029 7f290201f700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.344029, event: reached_pg, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2691> 2014-11-03 11:37:55.344075 7f290201f700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.344075, event: started, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2690> 2014-11-03 11:37:55.344174 7f290201f700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.344174, event: started, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2689> 2014-11-03 11:37:55.344206 7f290201f700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.344205, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2688> 2014-11-03 11:37:55.344354 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 184 ==== osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1251020766 0 3511520646) 0x24bec800 con 0x22fefdc0 -2687> 2014-11-03 11:37:55.344384 7f28f36c2700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344275, event: header_read, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2686> 2014-11-03 11:37:55.344402 7f28f36c2700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344277, event: throttled, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2685> 2014-11-03 11:37:55.344415 7f28f36c2700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344346, event: all_read, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2684> 2014-11-03 11:37:55.344428 7f28f36c2700 5 -- op tracker -- seq: 3173, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2683> 2014-11-03 11:37:55.344473 7f28ff81a700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344473, event: reached_pg, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2682> 2014-11-03 11:37:55.344493 7f28ff81a700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344493, event: started, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2681> 2014-11-03 11:37:55.344543 7f291450b700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.344542, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2680> 2014-11-03 11:37:55.344585 7f28ff81a700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344585, event: started, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2679> 2014-11-03 11:37:55.344655 7f28ff81a700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344655, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2678> 2014-11-03 11:37:55.344675 7f2914d0c700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.344674, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2677> 2014-11-03 11:37:55.344697 7f2911d06700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.344697, event: commit_sent, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2676> 2014-11-03 11:37:55.344717 7f2914d0c700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.344717, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2675> 2014-11-03 11:37:55.344723 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a9b80 con 0x22fefdc0 -2674> 2014-11-03 11:37:55.347771 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 185 ==== osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2675497586 0 3511520646) 0x1c94a400 con 0x22fefdc0 -2673> 2014-11-03 11:37:55.347805 7f28f36c2700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.347601, event: header_read, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2672> 2014-11-03 11:37:55.347834 7f28f36c2700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.347602, event: throttled, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2671> 2014-11-03 11:37:55.347849 7f28f36c2700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.347719, event: all_read, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2670> 2014-11-03 11:37:55.347862 7f28f36c2700 5 -- op tracker -- seq: 3174, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2669> 2014-11-03 11:37:55.347942 7f290201f700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.347942, event: reached_pg, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2668> 2014-11-03 11:37:55.347985 7f290201f700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.347985, event: started, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2667> 2014-11-03 11:37:55.348002 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 186 ==== osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2643416175 0 3511520646) 0x1c949800 con 0x22fefdc0 -2666> 2014-11-03 11:37:55.348033 7f28f36c2700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.347916, event: header_read, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2665> 2014-11-03 11:37:55.348049 7f28f36c2700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.347917, event: throttled, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2664> 2014-11-03 11:37:55.348057 7f28f36c2700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.347995, event: all_read, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2663> 2014-11-03 11:37:55.348067 7f28f36c2700 5 -- op tracker -- seq: 3175, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2662> 2014-11-03 11:37:55.348129 7f290201f700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.348129, event: started, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2661> 2014-11-03 11:37:55.348175 7f290201f700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.348174, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2660> 2014-11-03 11:37:55.348246 7f28ff81a700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.348246, event: reached_pg, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2659> 2014-11-03 11:37:55.348279 7f28ff81a700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.348279, event: started, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2658> 2014-11-03 11:37:55.348327 7f28f45d1700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 48 ==== osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1459173130 0 3511520646) 0x22f9f400 con 0x22fef580 -2657> 2014-11-03 11:37:55.348365 7f28f45d1700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.348178, event: header_read, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2656> 2014-11-03 11:37:55.348383 7f28f45d1700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.348181, event: throttled, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2655> 2014-11-03 11:37:55.348391 7f28f45d1700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.348311, event: all_read, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2654> 2014-11-03 11:37:55.348391 7f28ff81a700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.348391, event: started, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2653> 2014-11-03 11:37:55.348379 7f28ead39700 1 -- 192.168.5.2:6803/7606 <== osd.28 192.168.5.3:6812/13976 33 ==== osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1638717181 0 3511520646) 0x2e168800 con 0x23157dc0 -2652> 2014-11-03 11:37:55.348409 7f28f45d1700 5 -- op tracker -- seq: 3176, time: 0.000000, event: dispatched, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2651> 2014-11-03 11:37:55.348417 7f28ead39700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.348191, event: header_read, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2650> 2014-11-03 11:37:55.348427 7f28ff81a700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.348427, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2649> 2014-11-03 11:37:55.348433 7f28ead39700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.348193, event: throttled, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2648> 2014-11-03 11:37:55.348445 7f28ead39700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.348358, event: all_read, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2647> 2014-11-03 11:37:55.348452 7f28ead39700 5 -- op tracker -- seq: 3177, time: 0.000000, event: dispatched, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2646> 2014-11-03 11:37:55.348468 7f28ff019700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.348468, event: reached_pg, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2645> 2014-11-03 11:37:55.348493 7f28ff019700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.348493, event: started, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2644> 2014-11-03 11:37:55.348521 7f290201f700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.348521, event: reached_pg, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2643> 2014-11-03 11:37:55.348555 7f290201f700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.348555, event: started, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2642> 2014-11-03 11:37:55.348563 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 187 ==== osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (228654257 0 3511520646) 0x1c948c00 con 0x22fefdc0 -2641> 2014-11-03 11:37:55.348595 7f28ff019700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.348595, event: started, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2640> 2014-11-03 11:37:55.348595 7f28f36c2700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.348433, event: header_read, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2639> 2014-11-03 11:37:55.348636 7f28f36c2700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.348435, event: throttled, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2638> 2014-11-03 11:37:55.348638 7f28ff019700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.348638, event: commit_queued_for_journal_write, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2637> 2014-11-03 11:37:55.348667 7f28f36c2700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.348552, event: all_read, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2636> 2014-11-03 11:37:55.348685 7f28f36c2700 5 -- op tracker -- seq: 3178, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2635> 2014-11-03 11:37:55.348681 7f290201f700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.348680, event: started, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2634> 2014-11-03 11:37:55.348718 7f290201f700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.348718, event: commit_queued_for_journal_write, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2633> 2014-11-03 11:37:55.348732 7f28ff81a700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.348732, event: reached_pg, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2632> 2014-11-03 11:37:55.348754 7f28ff81a700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.348754, event: started, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2631> 2014-11-03 11:37:55.348837 7f28ff81a700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.348837, event: started, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2630> 2014-11-03 11:37:55.348866 7f28ff81a700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.348866, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2629> 2014-11-03 11:37:55.350241 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 188 ==== osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3489390332 0 3511520646) 0x1c948000 con 0x22fefdc0 -2628> 2014-11-03 11:37:55.350267 7f28f36c2700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.350169, event: header_read, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2627> 2014-11-03 11:37:55.350294 7f28f36c2700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.350170, event: throttled, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2626> 2014-11-03 11:37:55.350306 7f28f36c2700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.350233, event: all_read, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2625> 2014-11-03 11:37:55.350318 7f28f36c2700 5 -- op tracker -- seq: 3179, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2624> 2014-11-03 11:37:55.350371 7f290201f700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.350371, event: reached_pg, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2623> 2014-11-03 11:37:55.350412 7f290201f700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.350412, event: started, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2622> 2014-11-03 11:37:55.350537 7f290201f700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.350536, event: started, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2621> 2014-11-03 11:37:55.350590 7f290201f700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.350590, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2620> 2014-11-03 11:37:55.352285 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 189 ==== osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1650915265 0 3511520646) 0x1c935400 con 0x22fefdc0 -2619> 2014-11-03 11:37:55.352317 7f28f36c2700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.352203, event: header_read, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2618> 2014-11-03 11:37:55.352335 7f28f36c2700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.352205, event: throttled, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2617> 2014-11-03 11:37:55.352347 7f28f36c2700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.352274, event: all_read, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2616> 2014-11-03 11:37:55.352366 7f28f36c2700 5 -- op tracker -- seq: 3180, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2615> 2014-11-03 11:37:55.352412 7f28ff81a700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.352412, event: reached_pg, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2614> 2014-11-03 11:37:55.352435 7f28ff81a700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.352435, event: started, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2613> 2014-11-03 11:37:55.352523 7f28ff81a700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.352523, event: started, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2612> 2014-11-03 11:37:55.352552 7f28ff81a700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.352552, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2611> 2014-11-03 11:37:55.353414 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.58 192.168.5.5:0/24563 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.351949) v2 ==== 47+0+0 (3230913997 0 0) 0x23414600 con 0x2469eb00 -2610> 2014-11-03 11:37:55.353406 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 190 ==== osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2975736288 0 3511520646) 0x1c933c00 con 0x22fefdc0 -2609> 2014-11-03 11:37:55.353473 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/24563 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.351949) v2 -- ?+0 0x6425c00 con 0x2469eb00 -2608> 2014-11-03 11:37:55.353474 7f28f36c2700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.353112, event: header_read, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2607> 2014-11-03 11:37:55.353511 7f28f36c2700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.353114, event: throttled, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2606> 2014-11-03 11:37:55.353534 7f28f36c2700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.353391, event: all_read, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2605> 2014-11-03 11:37:55.353552 7f28f36c2700 5 -- op tracker -- seq: 3181, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2604> 2014-11-03 11:37:55.353614 7f290201f700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.353613, event: reached_pg, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2603> 2014-11-03 11:37:55.353640 7f290201f700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.353639, event: started, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2602> 2014-11-03 11:37:55.353638 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.58 192.168.5.5:0/24563 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.351949) v2 ==== 47+0+0 (3230913997 0 0) 0x2743ac00 con 0x244cbdc0 -2601> 2014-11-03 11:37:55.353686 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/24563 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.351949) v2 -- ?+0 0x6425800 con 0x244cbdc0 -2600> 2014-11-03 11:37:55.353745 7f290201f700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.353745, event: started, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2599> 2014-11-03 11:37:55.353772 7f290201f700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.353772, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2598> 2014-11-03 11:37:55.355091 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 191 ==== osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2328499510 0 3511520646) 0x1c933000 con 0x22fefdc0 -2597> 2014-11-03 11:37:55.355134 7f28f36c2700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.354985, event: header_read, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2596> 2014-11-03 11:37:55.355159 7f28f36c2700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.354991, event: throttled, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2595> 2014-11-03 11:37:55.355178 7f28f36c2700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.355076, event: all_read, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2594> 2014-11-03 11:37:55.355196 7f28f36c2700 5 -- op tracker -- seq: 3182, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2593> 2014-11-03 11:37:55.355262 7f28ff81a700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.355262, event: reached_pg, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2592> 2014-11-03 11:37:55.355296 7f28ff81a700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.355296, event: started, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2591> 2014-11-03 11:37:55.355426 7f28ff81a700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.355426, event: started, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2590> 2014-11-03 11:37:55.355474 7f28ff81a700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.355474, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2589> 2014-11-03 11:37:55.358255 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 192 ==== osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3631422823 0 3511520646) 0x1c934800 con 0x22fefdc0 -2588> 2014-11-03 11:37:55.358288 7f28f36c2700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.358140, event: header_read, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2587> 2014-11-03 11:37:55.358301 7f28f36c2700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.358142, event: throttled, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2586> 2014-11-03 11:37:55.358309 7f28f36c2700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.358240, event: all_read, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2585> 2014-11-03 11:37:55.358320 7f28f36c2700 5 -- op tracker -- seq: 3183, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2584> 2014-11-03 11:37:55.358378 7f290201f700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.358378, event: reached_pg, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2583> 2014-11-03 11:37:55.358414 7f290201f700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.358414, event: started, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2582> 2014-11-03 11:37:55.358658 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2650ddc0 sd=66 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bc9a0).accept sd=66 192.168.5.2:42975/0 -2581> 2014-11-03 11:37:55.358674 7f290201f700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.358674, event: started, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2580> 2014-11-03 11:37:55.358721 7f290201f700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.358720, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2579> 2014-11-03 11:37:55.359020 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 193 ==== osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2672986297 0 3511520646) 0x23d9b400 con 0x22fefdc0 -2578> 2014-11-03 11:37:55.359049 7f28f36c2700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.358930, event: header_read, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2577> 2014-11-03 11:37:55.359066 7f28f36c2700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.358932, event: throttled, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2576> 2014-11-03 11:37:55.359075 7f28f36c2700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.359012, event: all_read, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2575> 2014-11-03 11:37:55.359088 7f28f36c2700 5 -- op tracker -- seq: 3184, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2574> 2014-11-03 11:37:55.359130 7f28ff81a700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.359130, event: reached_pg, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2573> 2014-11-03 11:37:55.359158 7f28ff81a700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.359158, event: started, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2572> 2014-11-03 11:37:55.359259 7f28ff81a700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.359259, event: started, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2571> 2014-11-03 11:37:55.359286 7f28ff81a700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.359285, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2570> 2014-11-03 11:37:55.359564 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450919 192.168.5.2:0/1004025 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23a9e000 con 0x1c9bc9a0 -2569> 2014-11-03 11:37:55.360186 7f28fb812700 0 osd.21 105950 do_command r=0 -2568> 2014-11-03 11:37:55.360207 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004025 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x2dcb08c0 con 0x1c9bc9a0 -2567> 2014-11-03 11:37:55.360237 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 194 ==== osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (357825422 0 3511520646) 0x23d9a800 con 0x22fefdc0 -2566> 2014-11-03 11:37:55.360259 7f28f36c2700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.360081, event: header_read, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2565> 2014-11-03 11:37:55.360276 7f28f36c2700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.360083, event: throttled, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2564> 2014-11-03 11:37:55.360288 7f28f36c2700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.360228, event: all_read, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2563> 2014-11-03 11:37:55.360300 7f28f36c2700 5 -- op tracker -- seq: 3185, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2562> 2014-11-03 11:37:55.360351 7f290201f700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.360351, event: reached_pg, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2561> 2014-11-03 11:37:55.360380 7f290201f700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.360379, event: started, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2560> 2014-11-03 11:37:55.360477 7f290201f700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.360477, event: started, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2559> 2014-11-03 11:37:55.360504 7f290201f700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.360504, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2558> 2014-11-03 11:37:55.367973 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450919 192.168.5.2:0/1004025 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23a92800 con 0x1c9bc9a0 -2557> 2014-11-03 11:37:55.368743 7f28fb812700 0 osd.21 105950 do_command r=0 -2556> 2014-11-03 11:37:55.368761 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004025 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x23637a80 con 0x1c9bc9a0 -2555> 2014-11-03 11:37:55.376396 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004025 pipe(0x2650ddc0 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc9a0).reader couldn't read tag, (0) Success -2554> 2014-11-03 11:37:55.376484 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004025 pipe(0x2650ddc0 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bc9a0).fault (0) Success -2553> 2014-11-03 11:37:55.376680 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bc9a0 session 0x2d281280 -2552> 2014-11-03 11:37:55.399211 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 51 ==== osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (1457640387 0 3511520646) 0x2de0e800 con 0x22fbc000 -2551> 2014-11-03 11:37:55.399271 7f28f71fd700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.399021, event: header_read, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2550> 2014-11-03 11:37:55.399312 7f28f71fd700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.399024, event: throttled, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2549> 2014-11-03 11:37:55.399339 7f28f71fd700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.399189, event: all_read, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2548> 2014-11-03 11:37:55.399355 7f28f71fd700 5 -- op tracker -- seq: 3186, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2547> 2014-11-03 11:37:55.399439 7f290001b700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.399439, event: reached_pg, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2546> 2014-11-03 11:37:55.399474 7f290001b700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.399474, event: started, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2545> 2014-11-03 11:37:55.399572 7f290001b700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.399572, event: started, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2544> 2014-11-03 11:37:55.399610 7f290001b700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.399610, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2543> 2014-11-03 11:37:55.404079 7f291450b700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.404079, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2542> 2014-11-03 11:37:55.404148 7f291450b700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.404148, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2541> 2014-11-03 11:37:55.404138 7f2911d06700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.404138, event: commit_sent, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2540> 2014-11-03 11:37:55.404258 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x258be600 con 0x22fefdc0 -2539> 2014-11-03 11:37:55.404299 7f2914d0c700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.404299, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2538> 2014-11-03 11:37:55.404312 7f2911d06700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.404312, event: commit_sent, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2537> 2014-11-03 11:37:55.404372 7f2914d0c700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.404372, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2536> 2014-11-03 11:37:55.404412 7f2914d0c700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.404412, event: write_thread_in_journal_buffer, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2535> 2014-11-03 11:37:55.404413 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2f94db80 con 0x22fefdc0 -2534> 2014-11-03 11:37:55.404442 7f2914d0c700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.404442, event: write_thread_in_journal_buffer, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2533> 2014-11-03 11:37:55.404473 7f2914d0c700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.404473, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2532> 2014-11-03 11:37:55.404498 7f2914d0c700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.404498, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2531> 2014-11-03 11:37:55.404528 7f2914d0c700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.404528, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2530> 2014-11-03 11:37:55.404553 7f2914d0c700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.404553, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2529> 2014-11-03 11:37:55.404580 7f2914d0c700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.404580, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2528> 2014-11-03 11:37:55.404605 7f2914d0c700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.404605, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2527> 2014-11-03 11:37:55.404632 7f2914d0c700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.404632, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2526> 2014-11-03 11:37:55.404657 7f2914d0c700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.404657, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2525> 2014-11-03 11:37:55.404684 7f2914d0c700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.404684, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2524> 2014-11-03 11:37:55.405328 7f291450b700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.405328, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2523> 2014-11-03 11:37:55.405372 7f2911d06700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.405372, event: commit_sent, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2522> 2014-11-03 11:37:55.405388 7f291450b700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.405388, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2521> 2014-11-03 11:37:55.405406 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2f94c580 con 0x22fefdc0 -2520> 2014-11-03 11:37:55.405414 7f291450b700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.405414, event: journaled_completion_queued, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2519> 2014-11-03 11:37:55.405443 7f2911d06700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.405443, event: commit_sent, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2518> 2014-11-03 11:37:55.405444 7f291450b700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.405444, event: journaled_completion_queued, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2517> 2014-11-03 11:37:55.405469 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2f94d8c0 con 0x22fefdc0 -2516> 2014-11-03 11:37:55.405496 7f291450b700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.405496, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2515> 2014-11-03 11:37:55.405530 7f291450b700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.405530, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2514> 2014-11-03 11:37:55.405573 7f291450b700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.405573, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2513> 2014-11-03 11:37:55.405580 7f2911d06700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.405580, event: commit_sent, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2512> 2014-11-03 11:37:55.405606 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- osd_sub_op_reply(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2f94c2c0 con 0x22fef580 -2511> 2014-11-03 11:37:55.405606 7f291450b700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.405606, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2510> 2014-11-03 11:37:55.405648 7f291450b700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.405648, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2509> 2014-11-03 11:37:55.405660 7f2911d06700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.405660, event: commit_sent, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2508> 2014-11-03 11:37:55.405676 7f291450b700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.405676, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2507> 2014-11-03 11:37:55.405682 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6812/13976 -- osd_sub_op_reply(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2f94d080 con 0x23157dc0 -2506> 2014-11-03 11:37:55.405708 7f291450b700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.405707, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2505> 2014-11-03 11:37:55.405731 7f2911d06700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.405731, event: commit_sent, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2504> 2014-11-03 11:37:55.405735 7f291450b700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.405735, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2503> 2014-11-03 11:37:55.405758 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2f94c840 con 0x22fefdc0 -2502> 2014-11-03 11:37:55.405761 7f291450b700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.405761, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2501> 2014-11-03 11:37:55.405793 7f2911d06700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.405793, event: commit_sent, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2500> 2014-11-03 11:37:55.405853 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9adc0 con 0x22fefdc0 -2499> 2014-11-03 11:37:55.405885 7f2911d06700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.405884, event: commit_sent, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2498> 2014-11-03 11:37:55.405928 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9b600 con 0x22fefdc0 -2497> 2014-11-03 11:37:55.405996 7f2911d06700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.405996, event: commit_sent, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2496> 2014-11-03 11:37:55.406017 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9a580 con 0x22fefdc0 -2495> 2014-11-03 11:37:55.406101 7f2911d06700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.406101, event: commit_sent, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2494> 2014-11-03 11:37:55.406183 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9b080 con 0x22fefdc0 -2493> 2014-11-03 11:37:55.406213 7f2911d06700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.406213, event: commit_sent, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2492> 2014-11-03 11:37:55.406270 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240a9b80 con 0x22fefdc0 -2491> 2014-11-03 11:37:55.406311 7f2911d06700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.406311, event: commit_sent, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2490> 2014-11-03 11:37:55.406336 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fcfa340 con 0x22fefdc0 -2489> 2014-11-03 11:37:55.406358 7f2911d06700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.406357, event: commit_sent, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2488> 2014-11-03 11:37:55.406535 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x27417000 con 0x22fefdc0 -2487> 2014-11-03 11:37:55.406576 7f2911d06700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.406576, event: commit_sent, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2486> 2014-11-03 11:37:55.406602 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] ondisk, result = 0) v2 -- ?+0 0x274188c0 con 0x22fbc000 -2485> 2014-11-03 11:37:55.407626 7f2912507700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:55.407626, event: sub_op_applied, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -2484> 2014-11-03 11:37:55.407672 7f2912507700 5 -- op tracker -- seq: 3141, time: 2014-11-03 11:37:55.407672, event: done, op: osd_sub_op(client.7914905.0:3652009 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090626 snapset=0=[]:[] snapc=0=[]) -2483> 2014-11-03 11:37:55.407846 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(f5affeac/rbd_data.4eeda241ca92c8.000000000000373b/head//6),PushReplyOp(ad7d46ac/rbd_data.7b544f6e3a6632.000000000000a9d4/head//6)]) v2 -- ?+0 0x23247000 con 0x2301f580 -2482> 2014-11-03 11:37:55.407901 7f2912507700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:55.407901, event: sub_op_applied, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -2481> 2014-11-03 11:37:55.407933 7f2912507700 5 -- op tracker -- seq: 3142, time: 2014-11-03 11:37:55.407933, event: done, op: osd_sub_op(client.7869019.0:16926853 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132376 snapset=0=[]:[] snapc=0=[]) -2480> 2014-11-03 11:37:55.408333 7f2912507700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:55.408333, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -2479> 2014-11-03 11:37:55.408365 7f2912507700 5 -- op tracker -- seq: 3143, time: 2014-11-03 11:37:55.408365, event: done, op: osd_sub_op(client.8703713.0:32254469 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953156 snapset=0=[]:[] snapc=0=[]) -2478> 2014-11-03 11:37:55.408510 7f2912507700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:55.408510, event: sub_op_applied, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -2477> 2014-11-03 11:37:55.408541 7f2912507700 5 -- op tracker -- seq: 3145, time: 2014-11-03 11:37:55.408541, event: done, op: osd_sub_op(client.7868624.0:4468361 6.576 52051d76/rbd_data.3d4749199c94d5.0000000000000681/head//6 [] v 105950'17262141 snapset=0=[]:[] snapc=0=[]) -2476> 2014-11-03 11:37:55.408956 7f28f45d1700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 49 ==== osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1477450672 0 3511520646) 0x2d576400 con 0x22fef580 -2475> 2014-11-03 11:37:55.408988 7f28f45d1700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.408870, event: header_read, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2474> 2014-11-03 11:37:55.409006 7f28f45d1700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.408872, event: throttled, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2473> 2014-11-03 11:37:55.409025 7f28f45d1700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.408945, event: all_read, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2472> 2014-11-03 11:37:55.409037 7f28f45d1700 5 -- op tracker -- seq: 3187, time: 0.000000, event: dispatched, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2471> 2014-11-03 11:37:55.409099 7f290181e700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.409098, event: reached_pg, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2470> 2014-11-03 11:37:55.409139 7f290181e700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.409138, event: started, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2469> 2014-11-03 11:37:55.409265 7f290181e700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.409265, event: started, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2468> 2014-11-03 11:37:55.409309 7f290181e700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.409309, event: commit_queued_for_journal_write, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2467> 2014-11-03 11:37:55.409377 7f2914d0c700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.409377, event: write_thread_in_journal_buffer, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2466> 2014-11-03 11:37:55.410834 7f291450b700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.410834, event: journaled_completion_queued, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2465> 2014-11-03 11:37:55.410908 7f2911d06700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.410908, event: commit_sent, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2464> 2014-11-03 11:37:55.410955 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- osd_sub_op_reply(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650ddc0 con 0x22fef580 -2463> 2014-11-03 11:37:55.413608 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(35cbf6ac/rbd_data.3b14122ae8944a.0000000000001a6a/head//6)]) v2 -- ?+0 0x27830600 con 0x2301f580 -2462> 2014-11-03 11:37:55.434663 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 100 ==== osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3195774127 0 3511520646) 0x27806c00 con 0x22fbc580 -2461> 2014-11-03 11:37:55.434706 7f28f70fc700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.434518, event: header_read, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2460> 2014-11-03 11:37:55.434731 7f28f70fc700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.434521, event: throttled, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2459> 2014-11-03 11:37:55.434754 7f28f70fc700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.434648, event: all_read, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2458> 2014-11-03 11:37:55.434771 7f28f70fc700 5 -- op tracker -- seq: 3188, time: 0.000000, event: dispatched, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2457> 2014-11-03 11:37:55.434850 7f290081c700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.434849, event: reached_pg, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2456> 2014-11-03 11:37:55.434885 7f290081c700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.434885, event: started, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2455> 2014-11-03 11:37:55.435082 7f290081c700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.435081, event: started, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2454> 2014-11-03 11:37:55.435146 7f290081c700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.435146, event: commit_queued_for_journal_write, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2453> 2014-11-03 11:37:55.435234 7f2914d0c700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.435234, event: write_thread_in_journal_buffer, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2452> 2014-11-03 11:37:55.435431 7f291450b700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.435431, event: journaled_completion_queued, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2451> 2014-11-03 11:37:55.435519 7f2911d06700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.435519, event: commit_sent, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2450> 2014-11-03 11:37:55.435563 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] ondisk, result = 0) v2 -- ?+0 0x315242c0 con 0x22fbc580 -2449> 2014-11-03 11:37:55.452018 7f2904023700 10 monclient: tick -2448> 2014-11-03 11:37:55.452041 7f2904023700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2014-11-03 11:37:25.452040) -2447> 2014-11-03 11:37:55.452064 7f2904023700 10 monclient: renew subs? (now: 2014-11-03 11:37:55.452064; renew after: 2014-11-03 11:39:54.017667) -- no -2446> 2014-11-03 11:37:55.464739 7f28f3eca700 1 -- 192.168.5.2:6803/7606 <== osd.83 192.168.5.6:6871/4005 42 ==== osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1724855870 0 3511520646) 0x2493d800 con 0x22fef2c0 -2445> 2014-11-03 11:37:55.464778 7f28f3eca700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.464631, event: header_read, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2444> 2014-11-03 11:37:55.464793 7f28f3eca700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.464633, event: throttled, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2443> 2014-11-03 11:37:55.464802 7f28f3eca700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.464723, event: all_read, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2442> 2014-11-03 11:37:55.464810 7f28f3eca700 5 -- op tracker -- seq: 3189, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2441> 2014-11-03 11:37:55.464887 7f28fe017700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.464887, event: reached_pg, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2440> 2014-11-03 11:37:55.464930 7f28fe017700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.464930, event: started, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2439> 2014-11-03 11:37:55.465042 7f28fe017700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.465042, event: started, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2438> 2014-11-03 11:37:55.465071 7f28fe017700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.465071, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2437> 2014-11-03 11:37:55.465179 7f2914d0c700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.465178, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2436> 2014-11-03 11:37:55.465440 7f291450b700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.465440, event: journaled_completion_queued, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2435> 2014-11-03 11:37:55.465488 7f2911d06700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.465488, event: commit_sent, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2434> 2014-11-03 11:37:55.465523 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6871/4005 -- osd_sub_op_reply(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] ondisk, result = 0) v2 -- ?+0 0x315258c0 con 0x22fef2c0 -2433> 2014-11-03 11:37:55.470809 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.112 192.168.5.9:0/24552 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.469514) v2 ==== 47+0+0 (1136190332 0 0) 0x6a83600 con 0x2484a160 -2432> 2014-11-03 11:37:55.470858 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.112 192.168.5.9:0/24552 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.469514) v2 ==== 47+0+0 (1136190332 0 0) 0x6a83c00 con 0x24537b00 -2431> 2014-11-03 11:37:55.470920 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/24552 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.469514) v2 -- ?+0 0x2743ac00 con 0x2484a160 -2430> 2014-11-03 11:37:55.471042 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/24552 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.469514) v2 -- ?+0 0x23414600 con 0x24537b00 -2429> 2014-11-03 11:37:55.497908 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 45 ==== osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (502906376 0 3511520646) 0x26611c00 con 0x230c2c60 -2428> 2014-11-03 11:37:55.497951 7f28ee16d700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.497805, event: header_read, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2427> 2014-11-03 11:37:55.497970 7f28ee16d700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.497807, event: throttled, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2426> 2014-11-03 11:37:55.497982 7f28ee16d700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.497894, event: all_read, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2425> 2014-11-03 11:37:55.497994 7f28ee16d700 5 -- op tracker -- seq: 3190, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2424> 2014-11-03 11:37:55.498080 7f28fd816700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.498079, event: reached_pg, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2423> 2014-11-03 11:37:55.498205 7f28fd816700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.498205, event: started, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2422> 2014-11-03 11:37:55.498394 7f28fd816700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.498393, event: started, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2421> 2014-11-03 11:37:55.498459 7f28fd816700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.498459, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2420> 2014-11-03 11:37:55.498566 7f2914d0c700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.498566, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2419> 2014-11-03 11:37:55.498759 7f291450b700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.498759, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2418> 2014-11-03 11:37:55.498866 7f2911d06700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.498866, event: commit_sent, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2417> 2014-11-03 11:37:55.498922 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] ondisk, result = 0) v2 -- ?+0 0x31524b00 con 0x230c2c60 -2416> 2014-11-03 11:37:55.499174 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2d424b80 sd=231 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bcb00).accept sd=231 192.168.5.201:58703/0 -2415> 2014-11-03 11:37:55.499774 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 46 ==== osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2232943010 0 3511520646) 0x275f4400 con 0x230c2c60 -2414> 2014-11-03 11:37:55.499807 7f28ee16d700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.499672, event: header_read, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2413> 2014-11-03 11:37:55.499830 7f28ee16d700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.499673, event: throttled, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2412> 2014-11-03 11:37:55.499843 7f28ee16d700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.499747, event: all_read, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2411> 2014-11-03 11:37:55.499855 7f28ee16d700 5 -- op tracker -- seq: 3191, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2410> 2014-11-03 11:37:55.499866 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419970 192.168.5.201:0/1029991 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "24.7d7"}) v1 ==== 80+0+0 (3756934197 0 0) 0x27b41000 con 0x1c9bcb00 -2409> 2014-11-03 11:37:55.499896 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 54 ==== osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1240418779 0 3511520646) 0x29e11c00 con 0x23000b00 -2408> 2014-11-03 11:37:55.499924 7f290001b700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.499924, event: reached_pg, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2407> 2014-11-03 11:37:55.499931 7f28f21ad700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.499792, event: header_read, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2406> 2014-11-03 11:37:55.499979 7f290001b700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.499979, event: started, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2405> 2014-11-03 11:37:55.499987 7f28f21ad700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.499795, event: throttled, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2404> 2014-11-03 11:37:55.500013 7f28f21ad700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.499880, event: all_read, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2403> 2014-11-03 11:37:55.500027 7f28f21ad700 5 -- op tracker -- seq: 3192, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2402> 2014-11-03 11:37:55.500090 7f28ff019700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.500090, event: reached_pg, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2401> 2014-11-03 11:37:55.500121 7f28ff019700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.500121, event: started, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2400> 2014-11-03 11:37:55.500125 7f290001b700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.500124, event: started, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2399> 2014-11-03 11:37:55.500178 7f290001b700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.500178, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2398> 2014-11-03 11:37:55.500213 7f28ff019700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.500213, event: started, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2397> 2014-11-03 11:37:55.500251 7f28ff019700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.500251, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2396> 2014-11-03 11:37:55.500253 7f2914d0c700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.500253, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2395> 2014-11-03 11:37:55.500308 7f2914d0c700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.500308, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2394> 2014-11-03 11:37:55.500507 7f28fb812700 0 osd.21 105950 do_command r=0 -2393> 2014-11-03 11:37:55.500512 7f291450b700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.500512, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2392> 2014-11-03 11:37:55.500529 7f291450b700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.500529, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2391> 2014-11-03 11:37:55.500534 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029991 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x936ae00 con 0x1c9bcb00 -2390> 2014-11-03 11:37:55.500549 7f2911d06700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.500549, event: commit_sent, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2389> 2014-11-03 11:37:55.500575 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] ondisk, result = 0) v2 -- ?+0 0x31525080 con 0x230c2c60 -2388> 2014-11-03 11:37:55.500601 7f2911d06700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.500601, event: commit_sent, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2387> 2014-11-03 11:37:55.500617 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- osd_sub_op_reply(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23c1d340 con 0x23000b00 -2386> 2014-11-03 11:37:55.501485 7f28ee16d700 1 -- 192.168.5.2:6803/7606 <== osd.68 192.168.5.5:6866/8133 47 ==== osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2019968246 0 3511520646) 0xd4b6c00 con 0x230c2c60 -2385> 2014-11-03 11:37:55.501556 7f28ee16d700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.501336, event: header_read, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2384> 2014-11-03 11:37:55.501613 7f28ee16d700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.501339, event: throttled, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2383> 2014-11-03 11:37:55.501646 7f28ee16d700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.501467, event: all_read, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2382> 2014-11-03 11:37:55.501677 7f28ee16d700 5 -- op tracker -- seq: 3193, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2381> 2014-11-03 11:37:55.501765 7f28fd816700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.501765, event: reached_pg, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2380> 2014-11-03 11:37:55.501817 7f28fd816700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.501816, event: started, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2379> 2014-11-03 11:37:55.501974 7f28fd816700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.501974, event: started, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2378> 2014-11-03 11:37:55.502018 7f28fd816700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.502018, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2377> 2014-11-03 11:37:55.502190 7f2914d0c700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.502190, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2376> 2014-11-03 11:37:55.502838 7f291450b700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.502838, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2375> 2014-11-03 11:37:55.502950 7f2911d06700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.502950, event: commit_sent, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2374> 2014-11-03 11:37:55.503022 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6866/8133 -- osd_sub_op_reply(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240d3600 con 0x230c2c60 -2373> 2014-11-03 11:37:55.507482 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419970 192.168.5.201:0/1029991 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "24.7d7"}) v1 ==== 74+0+0 (774687807 0 0) 0x2c015e00 con 0x1c9bcb00 -2372> 2014-11-03 11:37:55.508473 7f28fb812700 0 osd.21 105950 do_command r=0 -2371> 2014-11-03 11:37:55.508498 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1029991 -- command_reply(tid 2: 0 ) v1 -- ?+13413 0x24500700 con 0x1c9bcb00 -2370> 2014-11-03 11:37:55.513164 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029991 pipe(0x2d424b80 sd=231 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bcb00).reader couldn't read tag, (0) Success -2369> 2014-11-03 11:37:55.513257 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1029991 pipe(0x2d424b80 sd=231 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bcb00).fault (0) Success -2368> 2014-11-03 11:37:55.513408 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bcb00 session 0x32be000 -2367> 2014-11-03 11:37:55.517435 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 101 ==== osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (228547376 0 3511520646) 0x2c336c00 con 0x2308b840 -2366> 2014-11-03 11:37:55.517491 7f28ef985700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.517290, event: header_read, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2365> 2014-11-03 11:37:55.517521 7f28ef985700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.517293, event: throttled, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2364> 2014-11-03 11:37:55.517541 7f28ef985700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.517415, event: all_read, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2363> 2014-11-03 11:37:55.517561 7f28ef985700 5 -- op tracker -- seq: 3194, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2362> 2014-11-03 11:37:55.517650 7f28ff81a700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.517649, event: reached_pg, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2361> 2014-11-03 11:37:55.517702 7f28ff81a700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.517702, event: started, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2360> 2014-11-03 11:37:55.517837 7f28ff81a700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.517837, event: started, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2359> 2014-11-03 11:37:55.517884 7f28ff81a700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.517883, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2358> 2014-11-03 11:37:55.518202 7f2914d0c700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.518202, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2357> 2014-11-03 11:37:55.518725 7f291450b700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.518725, event: journaled_completion_queued, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2356> 2014-11-03 11:37:55.518781 7f2911d06700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.518780, event: commit_sent, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2355> 2014-11-03 11:37:55.518827 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] ondisk, result = 0) v2 -- ?+0 0x240d22c0 con 0x2308b840 -2354> 2014-11-03 11:37:55.563457 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x27831b00 sd=232 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bcc60).accept sd=232 192.168.5.2:42979/0 -2353> 2014-11-03 11:37:55.564350 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419973 192.168.5.2:0/1004176 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23414600 con 0x1c9bcc60 -2352> 2014-11-03 11:37:55.564768 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(15c97eac/rbd_data.7f01592ae8944a.000000000000007c/head//6)]) v2 -- ?+0 0x2340c800 con 0x2301f580 -2351> 2014-11-03 11:37:55.565173 7f28fb812700 0 osd.21 105950 do_command r=0 -2350> 2014-11-03 11:37:55.565209 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004176 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x23637a80 con 0x1c9bcc60 -2349> 2014-11-03 11:37:55.570452 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.34 192.168.5.3:0/2034 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.570056) v2 ==== 47+0+0 (1655384600 0 0) 0x23cbe800 con 0x239f89a0 -2348> 2014-11-03 11:37:55.570482 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.34 192.168.5.3:0/2034 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.570056) v2 ==== 47+0+0 (1655384600 0 0) 0x1a327800 con 0x238b8b00 -2347> 2014-11-03 11:37:55.570502 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/2034 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.570056) v2 -- ?+0 0x6a83600 con 0x239f89a0 -2346> 2014-11-03 11:37:55.570598 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/2034 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.570056) v2 -- ?+0 0x6a83c00 con 0x238b8b00 -2345> 2014-11-03 11:37:55.573363 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(395256ac/rbd_data.89cd895a911542.000000000000808e/head//6)]) v2 -- ?+0 0x2340c000 con 0x2301f580 -2344> 2014-11-03 11:37:55.578518 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(8616deac/rbd_data.89cd895a911542.0000000000008e96/head//6)]) v2 -- ?+0 0x252d7a00 con 0x2301f580 -2343> 2014-11-03 11:37:55.579236 7f2912507700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:55.579236, event: sub_op_applied, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -2342> 2014-11-03 11:37:55.579322 7f2912507700 5 -- op tracker -- seq: 3151, time: 2014-11-03 11:37:55.579321, event: done, op: osd_sub_op(client.7867990.0:23840794 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703422 snapset=0=[]:[] snapc=0=[]) -2341> 2014-11-03 11:37:55.579376 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419973 192.168.5.2:0/1004176 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23561600 con 0x1c9bcc60 -2340> 2014-11-03 11:37:55.579428 7f2912507700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:55.579427, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -2339> 2014-11-03 11:37:55.579476 7f2912507700 5 -- op tracker -- seq: 3152, time: 2014-11-03 11:37:55.579476, event: done, op: osd_sub_op(client.7867942.0:137198052 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944777 snapset=0=[]:[] snapc=0=[]) -2338> 2014-11-03 11:37:55.579804 7f2912507700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:55.579803, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -2337> 2014-11-03 11:37:55.579861 7f2912507700 5 -- op tracker -- seq: 3153, time: 2014-11-03 11:37:55.579861, event: done, op: osd_sub_op(client.7867942.0:137198053 6.4c4 c60ab4c4/rbd_data.43c40916a10.000000000000041c/head//6 [] v 105950'10944778 snapset=0=[]:[] snapc=0=[]) -2336> 2014-11-03 11:37:55.580036 7f2912507700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:55.580035, event: sub_op_applied, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -2335> 2014-11-03 11:37:55.580114 7f2912507700 5 -- op tracker -- seq: 3154, time: 2014-11-03 11:37:55.580114, event: done, op: osd_sub_op(client.7867990.0:23840795 6.a42 8fa74a42/rbd_data.4f641e4f2c279.00000000000000e8/head//6 [] v 105950'6703423 snapset=0=[]:[] snapc=0=[]) -2334> 2014-11-03 11:37:55.580532 7f2912507700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:55.580532, event: sub_op_applied, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -2333> 2014-11-03 11:37:55.580599 7f2912507700 5 -- op tracker -- seq: 3155, time: 2014-11-03 11:37:55.580599, event: done, op: osd_sub_op(client.7871531.0:413984 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114068 snapset=0=[]:[] snapc=0=[]) -2332> 2014-11-03 11:37:55.580774 7f2912507700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.580774, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2331> 2014-11-03 11:37:55.580814 7f2912507700 5 -- op tracker -- seq: 3158, time: 2014-11-03 11:37:55.580814, event: done, op: osd_sub_op(client.8703713.0:32254484 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953157 snapset=0=[]:[] snapc=0=[]) -2330> 2014-11-03 11:37:55.581056 7f28fb812700 0 osd.21 105950 do_command r=0 -2329> 2014-11-03 11:37:55.581093 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004176 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x936ae00 con 0x1c9bcc60 -2328> 2014-11-03 11:37:55.584816 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(e4ce6ac/rbd_data.3c3dbc2ae8944a.000000000000020b/head//6)]) v2 -- ?+0 0x2b950200 con 0x2301f580 -2327> 2014-11-03 11:37:55.587840 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004176 pipe(0x27831b00 sd=232 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bcc60).reader couldn't read tag, (0) Success -2326> 2014-11-03 11:37:55.587885 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004176 pipe(0x27831b00 sd=232 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bcc60).fault (0) Success -2325> 2014-11-03 11:37:55.588015 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bcc60 session 0x32be000 -2324> 2014-11-03 11:37:55.605147 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 118 ==== MOSDPGPush(24.761 105950 [PushOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, version: 105946'346662, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195286+0+0 (3939328073 0 0) 0x25835400 con 0x22fb76e0 -2323> 2014-11-03 11:37:55.605215 7f28edd69700 5 -- op tracker -- seq: 3195, time: 2014-11-03 11:37:55.592446, event: header_read, op: MOSDPGPush(24.761 105950 [PushOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, version: 105946'346662, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2322> 2014-11-03 11:37:55.605251 7f28edd69700 5 -- op tracker -- seq: 3195, time: 2014-11-03 11:37:55.592449, event: throttled, op: MOSDPGPush(24.761 105950 [PushOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, version: 105946'346662, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2321> 2014-11-03 11:37:55.605269 7f28edd69700 5 -- op tracker -- seq: 3195, time: 2014-11-03 11:37:55.605130, event: all_read, op: MOSDPGPush(24.761 105950 [PushOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, version: 105946'346662, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2320> 2014-11-03 11:37:55.605311 7f28edd69700 5 -- op tracker -- seq: 3195, time: 0.000000, event: dispatched, op: MOSDPGPush(24.761 105950 [PushOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, version: 105946'346662, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2319> 2014-11-03 11:37:55.605406 7f290001b700 5 -- op tracker -- seq: 3195, time: 2014-11-03 11:37:55.605406, event: reached_pg, op: MOSDPGPush(24.761 105950 [PushOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, version: 105946'346662, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2318> 2014-11-03 11:37:55.605486 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 119 ==== osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1087915704 0 3511520646) 0x191b7400 con 0x22fb76e0 -2317> 2014-11-03 11:37:55.605519 7f28edd69700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.605378, event: header_read, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2316> 2014-11-03 11:37:55.605545 7f28edd69700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.605379, event: throttled, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2315> 2014-11-03 11:37:55.605562 7f28edd69700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.605476, event: all_read, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2314> 2014-11-03 11:37:55.605573 7f28edd69700 5 -- op tracker -- seq: 3196, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2313> 2014-11-03 11:37:55.605626 7f28fd816700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.605626, event: reached_pg, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2312> 2014-11-03 11:37:55.605663 7f28fd816700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.605663, event: started, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2311> 2014-11-03 11:37:55.605707 7f290001b700 5 -- op tracker -- seq: 3195, time: 2014-11-03 11:37:55.605707, event: done, op: MOSDPGPush(24.761 105950 [PushOp(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24, version: 105946'346662, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24@105946'346662, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2310> 2014-11-03 11:37:55.605770 7f290001b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.605770, event: reached_pg, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2309> 2014-11-03 11:37:55.605814 7f28fd816700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.605814, event: started, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2308> 2014-11-03 11:37:55.605839 7f290001b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.605839, event: started, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2307> 2014-11-03 11:37:55.605862 7f28fd816700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.605862, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2306> 2014-11-03 11:37:55.605874 7f290001b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.605874, event: started, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2305> 2014-11-03 11:37:55.605945 7f290001b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.605944, event: waiting for subops from 31,90, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2304> 2014-11-03 11:37:55.605986 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6824/20618 -- osd_sub_op(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] v 105950'346663 snapset=0=[]:[] snapc=0=[]) v11 -- ?+5001 0x24e87400 con 0x230c2000 -2303> 2014-11-03 11:37:55.606027 7f290001b700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] v 105950'346663 snapset=0=[]:[] snapc=0=[]) v11 -- ?+5001 0x24e85000 con 0x22fb76e0 -2302> 2014-11-03 11:37:55.606176 7f290001b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.606176, event: commit_queued_for_journal_write, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2301> 2014-11-03 11:37:55.606631 7f2914d0c700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.606631, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2300> 2014-11-03 11:37:55.606673 7f2914d0c700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.606672, event: write_thread_in_journal_buffer, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2299> 2014-11-03 11:37:55.607625 7f28ebc48700 1 -- 192.168.5.2:6803/7606 <== osd.31 192.168.5.3:6824/20618 78 ==== osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (2210868875 0 0) 0x23613840 con 0x230c2000 -2298> 2014-11-03 11:37:55.607674 7f28ebc48700 5 -- op tracker -- seq: 3197, time: 2014-11-03 11:37:55.607509, event: header_read, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2297> 2014-11-03 11:37:55.607710 7f28ebc48700 5 -- op tracker -- seq: 3197, time: 2014-11-03 11:37:55.607512, event: throttled, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2296> 2014-11-03 11:37:55.607729 7f28ebc48700 5 -- op tracker -- seq: 3197, time: 2014-11-03 11:37:55.607605, event: all_read, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2295> 2014-11-03 11:37:55.607755 7f28ebc48700 5 -- op tracker -- seq: 3197, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2294> 2014-11-03 11:37:55.607839 7f28fd816700 5 -- op tracker -- seq: 3197, time: 2014-11-03 11:37:55.607839, event: reached_pg, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2293> 2014-11-03 11:37:55.607876 7f28fd816700 5 -- op tracker -- seq: 3197, time: 2014-11-03 11:37:55.607876, event: started, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2292> 2014-11-03 11:37:55.607896 7f28fd816700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.607896, event: sub_op_commit_rec, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2291> 2014-11-03 11:37:55.607923 7f28fd816700 5 -- op tracker -- seq: 3197, time: 2014-11-03 11:37:55.607923, event: done, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2290> 2014-11-03 11:37:55.608127 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(d5d30eac/rbd_data.89cd895a911542.0000000000009a55/head//6)]) v2 -- ?+0 0x4d90800 con 0x2301f580 -2289> 2014-11-03 11:37:55.611949 7f291450b700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.611949, event: journaled_completion_queued, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2288> 2014-11-03 11:37:55.612011 7f291450b700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.612011, event: journaled_completion_queued, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2287> 2014-11-03 11:37:55.612011 7f2911d06700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.612010, event: commit_sent, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2286> 2014-11-03 11:37:55.612056 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa23f340 con 0x22fb76e0 -2285> 2014-11-03 11:37:55.612117 7f2911d06700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.612117, event: op_commit, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2284> 2014-11-03 11:37:55.614136 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(73d956ac/rbd_data.798e3b6cf11b25.000000000000719b/head//6)]) v2 -- ?+0 0x23fdf200 con 0x2301f580 -2283> 2014-11-03 11:37:55.615885 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 120 ==== osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (389544281 0 3511520646) 0x24e85000 con 0x22fb76e0 -2282> 2014-11-03 11:37:55.615931 7f28edd69700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.615801, event: header_read, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2281> 2014-11-03 11:37:55.615946 7f28edd69700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.615803, event: throttled, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2280> 2014-11-03 11:37:55.615955 7f28edd69700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.615869, event: all_read, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2279> 2014-11-03 11:37:55.615975 7f28edd69700 5 -- op tracker -- seq: 3198, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2278> 2014-11-03 11:37:55.616333 7f290001b700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.616333, event: reached_pg, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2277> 2014-11-03 11:37:55.616371 7f290001b700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.616371, event: started, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2276> 2014-11-03 11:37:55.616455 7f290001b700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.616455, event: started, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2275> 2014-11-03 11:37:55.616484 7f290001b700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.616483, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2274> 2014-11-03 11:37:55.616555 7f2914d0c700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.616555, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2273> 2014-11-03 11:37:55.618976 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 121 ==== osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (954219652 0 3511520646) 0x29055800 con 0x22fb76e0 -2272> 2014-11-03 11:37:55.619011 7f28edd69700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.618883, event: header_read, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2271> 2014-11-03 11:37:55.619025 7f28edd69700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.618885, event: throttled, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2270> 2014-11-03 11:37:55.619035 7f28edd69700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.618962, event: all_read, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2269> 2014-11-03 11:37:55.619043 7f28edd69700 5 -- op tracker -- seq: 3199, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2268> 2014-11-03 11:37:55.619121 7f290101d700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.619121, event: reached_pg, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2267> 2014-11-03 11:37:55.619163 7f290101d700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.619162, event: started, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2266> 2014-11-03 11:37:55.619300 7f290101d700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.619300, event: started, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2265> 2014-11-03 11:37:55.619343 7f290101d700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.619343, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2264> 2014-11-03 11:37:55.623734 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 122 ==== osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (3490064654 0 0) 0xa23f340 con 0x22fb76e0 -2263> 2014-11-03 11:37:55.623763 7f28edd69700 5 -- op tracker -- seq: 3200, time: 2014-11-03 11:37:55.623673, event: header_read, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2262> 2014-11-03 11:37:55.623774 7f28edd69700 5 -- op tracker -- seq: 3200, time: 2014-11-03 11:37:55.623675, event: throttled, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2261> 2014-11-03 11:37:55.623782 7f28edd69700 5 -- op tracker -- seq: 3200, time: 2014-11-03 11:37:55.623720, event: all_read, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2260> 2014-11-03 11:37:55.623789 7f28edd69700 5 -- op tracker -- seq: 3200, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2259> 2014-11-03 11:37:55.623857 7f28fd816700 5 -- op tracker -- seq: 3200, time: 2014-11-03 11:37:55.623857, event: reached_pg, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2258> 2014-11-03 11:37:55.623928 7f28fd816700 5 -- op tracker -- seq: 3200, time: 2014-11-03 11:37:55.623928, event: started, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2257> 2014-11-03 11:37:55.623942 7f28fd816700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.623942, event: sub_op_commit_rec, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2256> 2014-11-03 11:37:55.624005 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.2:0/1027018 -- osd_op_reply(3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] v105950'346663 uv346663 ondisk = 0) v6 -- ?+0 0x29ce2000 con 0x1c9bc580 -2255> 2014-11-03 11:37:55.624046 7f28fd816700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.624046, event: commit_sent, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2254> 2014-11-03 11:37:55.624086 7f28fd816700 5 -- op tracker -- seq: 3200, time: 2014-11-03 11:37:55.624085, event: done, op: osd_sub_op_reply(client.7870547.0:3301683 24.761 e1d92f61/rbd_data.47bd962ae8944a.0000000000000820/head//24 [] ondisk, result = 0) -2253> 2014-11-03 11:37:55.633859 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 123 ==== osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3320778835 0 3511520646) 0x24bed400 con 0x22fb76e0 -2252> 2014-11-03 11:37:55.633890 7f28edd69700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.633694, event: header_read, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2251> 2014-11-03 11:37:55.633903 7f28edd69700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.633696, event: throttled, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2250> 2014-11-03 11:37:55.633912 7f28edd69700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.633846, event: all_read, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2249> 2014-11-03 11:37:55.633925 7f28edd69700 5 -- op tracker -- seq: 3201, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2248> 2014-11-03 11:37:55.633978 7f290001b700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.633978, event: reached_pg, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2247> 2014-11-03 11:37:55.634012 7f290001b700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.634011, event: started, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2246> 2014-11-03 11:37:55.634092 7f290001b700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.634091, event: started, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2245> 2014-11-03 11:37:55.634130 7f290001b700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.634130, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2244> 2014-11-03 11:37:55.655013 7f291450b700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.655013, event: journaled_completion_queued, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2243> 2014-11-03 11:37:55.655083 7f2911d06700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.655083, event: commit_sent, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2242> 2014-11-03 11:37:55.655126 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa23e000 con 0x22fb76e0 -2241> 2014-11-03 11:37:55.655135 7f2914d0c700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.655135, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2240> 2014-11-03 11:37:55.655201 7f2914d0c700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.655201, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2239> 2014-11-03 11:37:55.657261 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(9b7b1eac/rbd_data.89cd895a911542.0000000000005729/head//6)]) v2 -- ?+0 0x2310a800 con 0x2301f580 -2238> 2014-11-03 11:37:55.657763 7f2912507700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.657763, event: sub_op_applied, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2237> 2014-11-03 11:37:55.657807 7f291450b700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.657807, event: journaled_completion_queued, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2236> 2014-11-03 11:37:55.657814 7f2912507700 5 -- op tracker -- seq: 3163, time: 2014-11-03 11:37:55.657814, event: done, op: osd_sub_op(client.9561337.0:516034 6.323 10d32323/rbd_data.43ef922ae8944a.0000000000000849/head//6 [] v 105950'14134375 snapset=0=[]:[] snapc=0=[]) -2235> 2014-11-03 11:37:55.657848 7f291450b700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.657848, event: journaled_completion_queued, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2234> 2014-11-03 11:37:55.657866 7f2911d06700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.657865, event: commit_sent, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2233> 2014-11-03 11:37:55.657901 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] ondisk, result = 0) v2 -- ?+0 0xa23fb80 con 0x22fb76e0 -2232> 2014-11-03 11:37:55.657919 7f2912507700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.657919, event: sub_op_applied, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2231> 2014-11-03 11:37:55.657930 7f2911d06700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.657929, event: commit_sent, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2230> 2014-11-03 11:37:55.657965 7f2912507700 5 -- op tracker -- seq: 3164, time: 2014-11-03 11:37:55.657965, event: done, op: osd_sub_op(client.7869019.0:16926943 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033670 snapset=0=[]:[] snapc=0=[]) -2229> 2014-11-03 11:37:55.657997 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa23f600 con 0x22fb76e0 -2228> 2014-11-03 11:37:55.658272 7f2912507700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.658272, event: sub_op_applied, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2227> 2014-11-03 11:37:55.658309 7f2912507700 5 -- op tracker -- seq: 3165, time: 2014-11-03 11:37:55.658309, event: done, op: osd_sub_op(client.7869019.0:16926944 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033671 snapset=0=[]:[] snapc=0=[]) -2226> 2014-11-03 11:37:55.658364 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.77 192.168.5.6:0/32590 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.656966) v2 ==== 47+0+0 (4235232539 0 0) 0x252d7200 con 0x2484a420 -2225> 2014-11-03 11:37:55.658411 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/32590 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.656966) v2 -- ?+0 0x23cbe800 con 0x2484a420 -2224> 2014-11-03 11:37:55.658432 7f2912507700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.658432, event: sub_op_applied, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2223> 2014-11-03 11:37:55.658433 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.77 192.168.5.6:0/32590 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.656966) v2 ==== 47+0+0 (4235232539 0 0) 0x2d6ec200 con 0x24205840 -2222> 2014-11-03 11:37:55.658485 7f2912507700 5 -- op tracker -- seq: 3166, time: 2014-11-03 11:37:55.658485, event: done, op: osd_sub_op(client.7869019.0:16926945 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033672 snapset=0=[]:[] snapc=0=[]) -2221> 2014-11-03 11:37:55.658498 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/32590 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.656966) v2 -- ?+0 0x1a327800 con 0x24205840 -2220> 2014-11-03 11:37:55.658623 7f2912507700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.658623, event: sub_op_applied, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2219> 2014-11-03 11:37:55.658661 7f2912507700 5 -- op tracker -- seq: 3167, time: 2014-11-03 11:37:55.658661, event: done, op: osd_sub_op(client.7869019.0:16926946 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033673 snapset=0=[]:[] snapc=0=[]) -2218> 2014-11-03 11:37:55.658820 7f2912507700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.658819, event: sub_op_applied, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2217> 2014-11-03 11:37:55.658870 7f2912507700 5 -- op tracker -- seq: 3168, time: 2014-11-03 11:37:55.658870, event: done, op: osd_sub_op(client.7869019.0:16926947 6.799 b1905f99/rbd_data.451e822ae8944a.00000000000000cf/head//6 [] v 105950'15033674 snapset=0=[]:[] snapc=0=[]) -2216> 2014-11-03 11:37:55.659108 7f2912507700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.659108, event: sub_op_applied, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2215> 2014-11-03 11:37:55.659196 7f2912507700 5 -- op tracker -- seq: 3169, time: 2014-11-03 11:37:55.659195, event: done, op: osd_sub_op(client.7914905.0:3652014 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090627 snapset=0=[]:[] snapc=0=[]) -2214> 2014-11-03 11:37:55.659372 7f2912507700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.659372, event: sub_op_applied, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2213> 2014-11-03 11:37:55.659446 7f2912507700 5 -- op tracker -- seq: 3170, time: 2014-11-03 11:37:55.659446, event: done, op: osd_sub_op(client.7869019.0:16927003 6.1c1 905191c1/rbd_data.451e822ae8944a.000000000000026d/head//6 [] v 105950'8132377 snapset=0=[]:[] snapc=0=[]) -2212> 2014-11-03 11:37:55.659643 7f2912507700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.659642, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2211> 2014-11-03 11:37:55.659712 7f2912507700 5 -- op tracker -- seq: 3171, time: 2014-11-03 11:37:55.659711, event: done, op: osd_sub_op(client.7914641.0:33713947 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033675 snapset=0=[]:[] snapc=0=[]) -2210> 2014-11-03 11:37:55.659920 7f2912507700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.659920, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2209> 2014-11-03 11:37:55.659986 7f2912507700 5 -- op tracker -- seq: 3172, time: 2014-11-03 11:37:55.659985, event: done, op: osd_sub_op(client.7914641.0:33713948 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033676 snapset=0=[]:[] snapc=0=[]) -2208> 2014-11-03 11:37:55.660187 7f2912507700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.660186, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2207> 2014-11-03 11:37:55.660301 7f2912507700 5 -- op tracker -- seq: 3173, time: 2014-11-03 11:37:55.660301, event: done, op: osd_sub_op(client.7914641.0:33713949 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033677 snapset=0=[]:[] snapc=0=[]) -2206> 2014-11-03 11:37:55.660466 7f2912507700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.660465, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2205> 2014-11-03 11:37:55.660526 7f2912507700 5 -- op tracker -- seq: 3174, time: 2014-11-03 11:37:55.660526, event: done, op: osd_sub_op(client.7914641.0:33713950 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033678 snapset=0=[]:[] snapc=0=[]) -2204> 2014-11-03 11:37:55.660710 7f2912507700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.660709, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2203> 2014-11-03 11:37:55.660753 7f2912507700 5 -- op tracker -- seq: 3175, time: 2014-11-03 11:37:55.660753, event: done, op: osd_sub_op(client.7914641.0:33713951 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033679 snapset=0=[]:[] snapc=0=[]) -2202> 2014-11-03 11:37:55.661004 7f2912507700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.661004, event: sub_op_applied, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2201> 2014-11-03 11:37:55.661054 7f2912507700 5 -- op tracker -- seq: 3176, time: 2014-11-03 11:37:55.661054, event: done, op: osd_sub_op(client.4838051.0:1120281 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320280 snapset=0=[]:[] snapc=0=[]) -2200> 2014-11-03 11:37:55.661325 7f2912507700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.661325, event: sub_op_applied, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2199> 2014-11-03 11:37:55.661407 7f2912507700 5 -- op tracker -- seq: 3177, time: 2014-11-03 11:37:55.661407, event: done, op: osd_sub_op(client.4838051.0:1120282 6.7e9 bdd827e9/rbd_data.49d2642ae8944a.0000000000000105/head//6 [] v 105950'8627706 snapset=0=[]:[] snapc=0=[]) -2198> 2014-11-03 11:37:55.661572 7f2912507700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.661572, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2197> 2014-11-03 11:37:55.661640 7f2912507700 5 -- op tracker -- seq: 3178, time: 2014-11-03 11:37:55.661640, event: done, op: osd_sub_op(client.7914641.0:33713952 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033680 snapset=0=[]:[] snapc=0=[]) -2196> 2014-11-03 11:37:55.661839 7f2912507700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.661839, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2195> 2014-11-03 11:37:55.661882 7f2912507700 5 -- op tracker -- seq: 3179, time: 2014-11-03 11:37:55.661882, event: done, op: osd_sub_op(client.7914641.0:33713953 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033681 snapset=0=[]:[] snapc=0=[]) -2194> 2014-11-03 11:37:55.662087 7f2912507700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.662086, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2193> 2014-11-03 11:37:55.662156 7f2912507700 5 -- op tracker -- seq: 3180, time: 2014-11-03 11:37:55.662156, event: done, op: osd_sub_op(client.7914641.0:33713954 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033682 snapset=0=[]:[] snapc=0=[]) -2192> 2014-11-03 11:37:55.662497 7f2912507700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.662497, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2191> 2014-11-03 11:37:55.662554 7f2912507700 5 -- op tracker -- seq: 3181, time: 2014-11-03 11:37:55.662554, event: done, op: osd_sub_op(client.7914641.0:33713955 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033683 snapset=0=[]:[] snapc=0=[]) -2190> 2014-11-03 11:37:55.662730 7f2912507700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.662730, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2189> 2014-11-03 11:37:55.662773 7f2912507700 5 -- op tracker -- seq: 3182, time: 2014-11-03 11:37:55.662773, event: done, op: osd_sub_op(client.7914641.0:33713956 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033684 snapset=0=[]:[] snapc=0=[]) -2188> 2014-11-03 11:37:55.663100 7f2912507700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.663100, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2187> 2014-11-03 11:37:55.663142 7f2912507700 5 -- op tracker -- seq: 3183, time: 2014-11-03 11:37:55.663142, event: done, op: osd_sub_op(client.7914641.0:33713957 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033685 snapset=0=[]:[] snapc=0=[]) -2186> 2014-11-03 11:37:55.663365 7f2912507700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.663365, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2185> 2014-11-03 11:37:55.663408 7f2912507700 5 -- op tracker -- seq: 3184, time: 2014-11-03 11:37:55.663408, event: done, op: osd_sub_op(client.7914641.0:33713958 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033686 snapset=0=[]:[] snapc=0=[]) -2184> 2014-11-03 11:37:55.663613 7f2912507700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.663613, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2183> 2014-11-03 11:37:55.663663 7f2912507700 5 -- op tracker -- seq: 3185, time: 2014-11-03 11:37:55.663663, event: done, op: osd_sub_op(client.7914641.0:33713959 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033687 snapset=0=[]:[] snapc=0=[]) -2182> 2014-11-03 11:37:55.663898 7f2912507700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.663898, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2181> 2014-11-03 11:37:55.663941 7f2912507700 5 -- op tracker -- seq: 3186, time: 2014-11-03 11:37:55.663941, event: done, op: osd_sub_op(client.7867942.0:137197995 6.4f0 ff0fc4f0/rbd_data.43c40916a10.0000000000000569/head//6 [] v 105950'13100468 snapset=0=[]:[] snapc=0=[]) -2180> 2014-11-03 11:37:55.664150 7f2912507700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.664150, event: sub_op_applied, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2179> 2014-11-03 11:37:55.664189 7f2912507700 5 -- op tracker -- seq: 3187, time: 2014-11-03 11:37:55.664189, event: done, op: osd_sub_op(client.4838051.0:1120283 6.54c 3e0bdd4c/rbd_data.49d2642ae8944a.0000000000000087/head//6 [] v 105950'16320281 snapset=0=[]:[] snapc=0=[]) -2178> 2014-11-03 11:37:55.664420 7f2912507700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.664420, event: sub_op_applied, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2177> 2014-11-03 11:37:55.664464 7f2912507700 5 -- op tracker -- seq: 3188, time: 2014-11-03 11:37:55.664463, event: done, op: osd_sub_op(client.9841255.0:636761 6.1f7 5878e1f7/rbd_data.4d3b3c2ae8944a.0000000000001ffe/head//6 [] v 105950'9362500 snapset=0=[]:[] snapc=0=[]) -2176> 2014-11-03 11:37:55.664661 7f2912507700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.664661, event: sub_op_applied, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2175> 2014-11-03 11:37:55.664704 7f2912507700 5 -- op tracker -- seq: 3189, time: 2014-11-03 11:37:55.664703, event: done, op: osd_sub_op(client.7914905.0:3652015 6.a8 e2d630a8/rbd_data.76480a6dd10759.00000000000006f0/head//6 [] v 105950'6090628 snapset=0=[]:[] snapc=0=[]) -2174> 2014-11-03 11:37:55.664918 7f2912507700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.664918, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2173> 2014-11-03 11:37:55.664957 7f2912507700 5 -- op tracker -- seq: 3190, time: 2014-11-03 11:37:55.664957, event: done, op: osd_sub_op(client.7868828.0:25297342 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132378 snapset=0=[]:[] snapc=0=[]) -2172> 2014-11-03 11:37:55.665164 7f2912507700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.665164, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2171> 2014-11-03 11:37:55.665208 7f2912507700 5 -- op tracker -- seq: 3191, time: 2014-11-03 11:37:55.665208, event: done, op: osd_sub_op(client.7868828.0:25297343 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132379 snapset=0=[]:[] snapc=0=[]) -2170> 2014-11-03 11:37:55.665400 7f2912507700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.665400, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2169> 2014-11-03 11:37:55.665446 7f2912507700 5 -- op tracker -- seq: 3192, time: 2014-11-03 11:37:55.665446, event: done, op: osd_sub_op(client.7868828.0:25297354 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703424 snapset=0=[]:[] snapc=0=[]) -2168> 2014-11-03 11:37:55.665662 7f2912507700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.665662, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2167> 2014-11-03 11:37:55.665716 7f2912507700 5 -- op tracker -- seq: 3193, time: 2014-11-03 11:37:55.665715, event: done, op: osd_sub_op(client.7868828.0:25297344 6.1c1 246731c1/rbd_data.4d48f961268364.0000000000000118/head//6 [] v 105950'8132380 snapset=0=[]:[] snapc=0=[]) -2166> 2014-11-03 11:37:55.665923 7f2912507700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.665923, event: sub_op_applied, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2165> 2014-11-03 11:37:55.665962 7f2912507700 5 -- op tracker -- seq: 3194, time: 2014-11-03 11:37:55.665962, event: done, op: osd_sub_op(client.7869041.0:12286877 6.690 ad5d5690/rbd_data.3b6c8f2ae8944a.000000000000025d/head//6 [] v 105950'28900016 snapset=0=[]:[] snapc=0=[]) -2164> 2014-11-03 11:37:55.673189 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.107 192.168.5.8:0/17474 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.671693) v2 ==== 47+0+0 (3544092739 0 0) 0x2a8ef800 con 0x239f8580 -2163> 2014-11-03 11:37:55.673241 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.107 192.168.5.8:0/17474 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.671693) v2 ==== 47+0+0 (3544092739 0 0) 0x23305600 con 0x238b86e0 -2162> 2014-11-03 11:37:55.673261 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.671693) v2 -- ?+0 0x252d7200 con 0x239f8580 -2161> 2014-11-03 11:37:55.673377 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.671693) v2 -- ?+0 0x2d6ec200 con 0x238b86e0 -2160> 2014-11-03 11:37:55.690408 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.75 192.168.5.6:0/12612 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.689068) v2 ==== 47+0+0 (4236435436 0 0) 0x245ff400 con 0x238b8580 -2159> 2014-11-03 11:37:55.690413 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.75 192.168.5.6:0/12612 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.689068) v2 ==== 47+0+0 (4236435436 0 0) 0x286bbe00 con 0x239f8420 -2158> 2014-11-03 11:37:55.690464 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/12612 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.689068) v2 -- ?+0 0x23305600 con 0x238b8580 -2157> 2014-11-03 11:37:55.690530 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/12612 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.689068) v2 -- ?+0 0x2a8ef800 con 0x239f8420 -2156> 2014-11-03 11:37:55.710325 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x236132c0 sd=569 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9bcdc0).accept sd=569 192.168.5.2:42982/0 -2155> 2014-11-03 11:37:55.711215 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419979 192.168.5.2:0/1004303 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23a92e00 con 0x1c9bcdc0 -2154> 2014-11-03 11:37:55.711988 7f28fb812700 0 osd.21 105950 do_command r=0 -2153> 2014-11-03 11:37:55.712036 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004303 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x6ef0c40 con 0x1c9bcdc0 -2152> 2014-11-03 11:37:55.714693 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.110 192.168.5.8:0/29518 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.712661) v2 ==== 47+0+0 (1721528767 0 0) 0x29fcda00 con 0x239f82c0 -2151> 2014-11-03 11:37:55.714777 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.110 192.168.5.8:0/29518 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.712661) v2 ==== 47+0+0 (1721528767 0 0) 0x2353b400 con 0x238b8420 -2150> 2014-11-03 11:37:55.714832 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/29518 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.712661) v2 -- ?+0 0x286bbe00 con 0x239f82c0 -2149> 2014-11-03 11:37:55.714961 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/29518 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.712661) v2 -- ?+0 0x245ff400 con 0x238b8420 -2148> 2014-11-03 11:37:55.716654 7f28f71fd700 1 -- 192.168.5.2:6803/7606 <== osd.48 192.168.5.4:6836/17727 52 ==== osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1844555161 0 3511520646) 0x96ea000 con 0x22fbc000 -2147> 2014-11-03 11:37:55.716690 7f28f71fd700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716548, event: header_read, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2146> 2014-11-03 11:37:55.716704 7f28f71fd700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716551, event: throttled, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2145> 2014-11-03 11:37:55.716713 7f28f71fd700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716640, event: all_read, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2144> 2014-11-03 11:37:55.716727 7f28f71fd700 5 -- op tracker -- seq: 3202, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2143> 2014-11-03 11:37:55.716785 7f28fd816700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716785, event: reached_pg, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2142> 2014-11-03 11:37:55.716817 7f28fd816700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716817, event: started, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2141> 2014-11-03 11:37:55.716907 7f28fd816700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716907, event: started, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2140> 2014-11-03 11:37:55.716938 7f28fd816700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716938, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2139> 2014-11-03 11:37:55.716995 7f2914d0c700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.716995, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2138> 2014-11-03 11:37:55.717275 7f291450b700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.717275, event: journaled_completion_queued, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2137> 2014-11-03 11:37:55.717310 7f2911d06700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.717310, event: commit_sent, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2136> 2014-11-03 11:37:55.717337 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6836/17727 -- osd_sub_op_reply(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3110e340 con 0x22fbc000 -2135> 2014-11-03 11:37:55.729036 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419979 192.168.5.2:0/1004303 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2372bc00 con 0x1c9bcdc0 -2134> 2014-11-03 11:37:55.730451 7f28fb812700 0 osd.21 105950 do_command r=0 -2133> 2014-11-03 11:37:55.730497 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004303 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x32f7700 con 0x1c9bcdc0 -2132> 2014-11-03 11:37:55.736837 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004303 pipe(0x236132c0 sd=569 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bcdc0).reader couldn't read tag, (0) Success -2131> 2014-11-03 11:37:55.736910 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004303 pipe(0x236132c0 sd=569 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9bcdc0).fault (0) Success -2130> 2014-11-03 11:37:55.737060 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9bcdc0 session 0x32be000 -2129> 2014-11-03 11:37:55.746714 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 103 ==== osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (675928157 0 3511520646) 0x29054c00 con 0x22fbcb00 -2128> 2014-11-03 11:37:55.746772 7f28f6cf8700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.746542, event: header_read, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2127> 2014-11-03 11:37:55.746795 7f28f6cf8700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.746546, event: throttled, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2126> 2014-11-03 11:37:55.746810 7f28f6cf8700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.746681, event: all_read, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2125> 2014-11-03 11:37:55.746823 7f28f6cf8700 5 -- op tracker -- seq: 3203, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2124> 2014-11-03 11:37:55.746903 7f290081c700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.746903, event: reached_pg, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2123> 2014-11-03 11:37:55.746958 7f290081c700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.746958, event: started, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2122> 2014-11-03 11:37:55.747031 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 92 ==== osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (649238853 0 3511520646) 0x26059000 con 0x22ff3b00 -2121> 2014-11-03 11:37:55.747064 7f28f0d99700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.746899, event: header_read, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2120> 2014-11-03 11:37:55.747078 7f28f0d99700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.746903, event: throttled, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2119> 2014-11-03 11:37:55.747087 7f28f0d99700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747016, event: all_read, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2118> 2014-11-03 11:37:55.747094 7f28f0d99700 5 -- op tracker -- seq: 3204, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2117> 2014-11-03 11:37:55.747124 7f290081c700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.747123, event: started, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2116> 2014-11-03 11:37:55.747148 7f290001b700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747148, event: reached_pg, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2115> 2014-11-03 11:37:55.747165 7f290081c700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.747165, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2114> 2014-11-03 11:37:55.747173 7f290001b700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747173, event: started, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2113> 2014-11-03 11:37:55.747303 7f2914d0c700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.747303, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2112> 2014-11-03 11:37:55.747322 7f290001b700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747322, event: started, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2111> 2014-11-03 11:37:55.747372 7f290001b700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747372, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2110> 2014-11-03 11:37:55.747599 7f291450b700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.747599, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2109> 2014-11-03 11:37:55.747652 7f2911d06700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.747652, event: commit_sent, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2108> 2014-11-03 11:37:55.747675 7f2914d0c700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747674, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2107> 2014-11-03 11:37:55.747690 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3110e8c0 con 0x22fbcb00 -2106> 2014-11-03 11:37:55.747865 7f291450b700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747865, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2105> 2014-11-03 11:37:55.747935 7f2911d06700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.747935, event: commit_sent, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2104> 2014-11-03 11:37:55.747971 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x316d12c0 con 0x22ff3b00 -2103> 2014-11-03 11:37:55.748443 7f28f0d99700 1 -- 192.168.5.2:6803/7606 <== osd.121 192.168.5.9:6849/12205 93 ==== osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2476391575 0 3511520646) 0x26611c00 con 0x22ff3b00 -2102> 2014-11-03 11:37:55.748483 7f28f0d99700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748328, event: header_read, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2101> 2014-11-03 11:37:55.748504 7f28f0d99700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748330, event: throttled, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2100> 2014-11-03 11:37:55.748518 7f28f0d99700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748432, event: all_read, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2099> 2014-11-03 11:37:55.748533 7f28f0d99700 5 -- op tracker -- seq: 3205, time: 0.000000, event: dispatched, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2098> 2014-11-03 11:37:55.748605 7f28fd816700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748605, event: reached_pg, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2097> 2014-11-03 11:37:55.748650 7f28fd816700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748650, event: started, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2096> 2014-11-03 11:37:55.748728 7f28fd816700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748728, event: started, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2095> 2014-11-03 11:37:55.748755 7f28fd816700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748755, event: commit_queued_for_journal_write, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2094> 2014-11-03 11:37:55.748808 7f2914d0c700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.748808, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2093> 2014-11-03 11:37:55.749016 7f291450b700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.749016, event: journaled_completion_queued, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2092> 2014-11-03 11:37:55.749047 7f2911d06700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.749047, event: commit_sent, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2091> 2014-11-03 11:37:55.749092 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6849/12205 -- osd_sub_op_reply(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23e43b80 con 0x22ff3b00 -2090> 2014-11-03 11:37:55.789258 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6838/23392 -- MOSDPGPushReply(6.6ac 105950 [PushReplyOp(de5be6ac/rbd_data.4af49c2ae8944a.0000000000000401/head//6)]) v2 -- ?+0 0xa6b5400 con 0x2301f580 -2089> 2014-11-03 11:37:55.790488 7f2912507700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.790488, event: sub_op_applied, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2088> 2014-11-03 11:37:55.790535 7f2912507700 5 -- op tracker -- seq: 3196, time: 2014-11-03 11:37:55.790535, event: done, op: osd_sub_op(client.7867870.0:20391846 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187825 snapset=0=[]:[] snapc=0=[]) -2087> 2014-11-03 11:37:55.791044 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 53 ==== MRecoveryReserve RELEASE pgid: 6.6ac, query_epoch: 105950 v2 ==== 26+0+0 (565612114 0 0) 0xa6b5400 con 0x2301f580 -2086> 2014-11-03 11:37:55.791088 7f290b031700 5 -- op tracker -- seq: 3206, time: 2014-11-03 11:37:55.790878, event: header_read, op: MRecoveryReserve RELEASE pgid: 6.6ac, query_epoch: 105950 -2085> 2014-11-03 11:37:55.791102 7f290b031700 5 -- op tracker -- seq: 3206, time: 2014-11-03 11:37:55.790880, event: throttled, op: MRecoveryReserve RELEASE pgid: 6.6ac, query_epoch: 105950 -2084> 2014-11-03 11:37:55.791108 7f290b031700 5 -- op tracker -- seq: 3206, time: 2014-11-03 11:37:55.790951, event: all_read, op: MRecoveryReserve RELEASE pgid: 6.6ac, query_epoch: 105950 -2083> 2014-11-03 11:37:55.791113 7f290b031700 5 -- op tracker -- seq: 3206, time: 2014-11-03 11:37:55.791083, event: dispatched, op: MRecoveryReserve RELEASE pgid: 6.6ac, query_epoch: 105950 -2082> 2014-11-03 11:37:55.791119 7f290b031700 5 -- op tracker -- seq: 3206, time: 2014-11-03 11:37:55.791119, event: waiting_for_osdmap, op: MRecoveryReserve RELEASE pgid: 6.6ac, query_epoch: 105950 -2081> 2014-11-03 11:37:55.791151 7f290b031700 5 -- op tracker -- seq: 3206, time: 2014-11-03 11:37:55.791151, event: done, op: MRecoveryReserve RELEASE pgid: 6.6ac, query_epoch: 105950 -2080> 2014-11-03 11:37:55.791174 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.118 192.168.5.9:6838/23392 54 ==== pg_info(1 pgs e105950:6.6ac) v4 ==== 1743+0+0 (2649943701 0 0) 0x2310a800 con 0x2301f580 -2079> 2014-11-03 11:37:55.791195 7f290b031700 5 -- op tracker -- seq: 3207, time: 2014-11-03 11:37:55.791017, event: header_read, op: pg_info(1 pgs e105950:6.6ac) -2078> 2014-11-03 11:37:55.791202 7f290b031700 5 -- op tracker -- seq: 3207, time: 2014-11-03 11:37:55.791018, event: throttled, op: pg_info(1 pgs e105950:6.6ac) -2077> 2014-11-03 11:37:55.791207 7f290b031700 5 -- op tracker -- seq: 3207, time: 2014-11-03 11:37:55.791123, event: all_read, op: pg_info(1 pgs e105950:6.6ac) -2076> 2014-11-03 11:37:55.791212 7f290b031700 5 -- op tracker -- seq: 3207, time: 2014-11-03 11:37:55.791193, event: dispatched, op: pg_info(1 pgs e105950:6.6ac) -2075> 2014-11-03 11:37:55.791217 7f290b031700 5 -- op tracker -- seq: 3207, time: 2014-11-03 11:37:55.791217, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:6.6ac) -2074> 2014-11-03 11:37:55.791193 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.6ac( v 105950'9806086 (105913'9802746,105950'9806086] local-les=105948 n=9072 ec=164 les/c 105948/105943 105947/105947/105366) [118,106,21] r=2 lpr=105947 pi=79477-105946/417 luod=0'0 crt=105946'9806084 active] exit Started/ReplicaActive/RepRecovering 2.743547 1 0.000128 -2073> 2014-11-03 11:37:55.791224 7f290b031700 5 -- op tracker -- seq: 3207, time: 2014-11-03 11:37:55.791224, event: started, op: pg_info(1 pgs e105950:6.6ac) -2072> 2014-11-03 11:37:55.791256 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.6ac( v 105950'9806086 (105913'9802746,105950'9806086] local-les=105948 n=9072 ec=164 les/c 105948/105943 105947/105947/105366) [118,106,21] r=2 lpr=105947 pi=79477-105946/417 luod=0'0 crt=105946'9806084 active] enter Started/ReplicaActive/RepNotRecovering -2071> 2014-11-03 11:37:55.791339 7f290b031700 5 -- op tracker -- seq: 3207, time: 2014-11-03 11:37:55.791339, event: done, op: pg_info(1 pgs e105950:6.6ac) -2070> 2014-11-03 11:37:55.791443 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- MRecoveryReserve GRANT pgid: 24.74d, query_epoch: 105950 v2 -- ?+0 0x2a6e8400 con 0x230a92c0 -2069> 2014-11-03 11:37:55.791477 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.74d( v 105946'157340 lc 105932'157336 (103730'153529,105946'157340] local-les=105948 n=165 ec=25000 les/c 105948/105943 105947/105947/105344) [114,21,85] r=1 lpr=105947 pi=94556-105946/311 luod=0'0 crt=105946'157340 lcod 0'0 active m=2] exit Started/ReplicaActive/RepWaitRecoveryReserved 29.136797 5 0.000072 -2068> 2014-11-03 11:37:55.791516 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.74d( v 105946'157340 lc 105932'157336 (103730'153529,105946'157340] local-les=105948 n=165 ec=25000 les/c 105948/105943 105947/105947/105344) [114,21,85] r=1 lpr=105947 pi=94556-105946/311 luod=0'0 crt=105946'157340 lcod 0'0 active m=2] enter Started/ReplicaActive/RepRecovering -2067> 2014-11-03 11:37:55.795701 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- MRecoveryReserve RELEASE pgid: 26.4bd, query_epoch: 105950 v2 -- ?+0 0x2a6e8800 con 0x22fe0c60 -2066> 2014-11-03 11:37:55.795762 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6816/27523 -- MRecoveryReserve RELEASE pgid: 26.4bd, query_epoch: 105950 v2 -- ?+0 0x91aea00 con 0x230c2160 -2065> 2014-11-03 11:37:55.795792 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 mlcod 0'0 active+degraded] exit Started/Primary/Active/Recovering 1.328961 1 0.000082 -2064> 2014-11-03 11:37:55.795827 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 mlcod 0'0 active+degraded] enter Started/Primary/Active/Recovered -2063> 2014-11-03 11:37:55.795842 7f2912507700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.795841, event: sub_op_applied, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2062> 2014-11-03 11:37:55.795873 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000047 0 0.000000 -2061> 2014-11-03 11:37:55.795890 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.4bd( v 105946'5300 (57369'2203,105946'5300] local-les=105948 n=12 ec=25085 les/c 105948/105943 105947/105947/105947) [21,6,44] r=0 lpr=105947 pi=105939-105946/2 crt=105946'5300 mlcod 0'0 active] enter Started/Primary/Active/Clean -2060> 2014-11-03 11:37:55.795895 7f2912507700 5 -- op tracker -- seq: 3198, time: 2014-11-03 11:37:55.795895, event: done, op: osd_sub_op(client.7867870.0:20391847 6.24d 7858e24d/rbd_data.43c0d35f998907.0000000000000161/head//6 [] v 105950'12187826 snapset=0=[]:[] snapc=0=[]) -2059> 2014-11-03 11:37:55.795942 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- pg_info(1 pgs e105950:26.4bd) v4 -- ?+0 0x91ae400 con 0x22fe0c60 -2058> 2014-11-03 11:37:55.795942 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 lc 105946'131027 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105946'131027 active+recovery_wait+degraded m=1] exit Started/Primary/Active/WaitLocalRecoveryReserved 27.736033 2 0.000086 -2057> 2014-11-03 11:37:55.795978 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6816/27523 -- pg_info(1 pgs e105950:26.4bd) v4 -- ?+0 0x2b1ff000 con 0x230c2160 -2056> 2014-11-03 11:37:55.795985 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 lc 105946'131027 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105946'131027 active+recovery_wait+degraded m=1] enter Started/Primary/Active/WaitRemoteRecoveryReserved -2055> 2014-11-03 11:37:55.796014 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- MRecoveryReserve REQUEST pgid: 24.53e, query_epoch: 105950 v2 -- ?+0 0x9459c00 con 0x22ff3420 -2054> 2014-11-03 11:37:55.796017 7f2912507700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.796017, event: sub_op_applied, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2053> 2014-11-03 11:37:55.796068 7f2912507700 5 -- op tracker -- seq: 3199, time: 2014-11-03 11:37:55.796068, event: done, op: osd_sub_op(client.12404444.0:39610 26.75f 515bdf5f/rbd_data.4744cf2ae8944a.0000000000000042/head//26 [] v 105950'1374243 snapset=0=[]:[] snapc=0=[]) -2052> 2014-11-03 11:37:55.796160 7f2912507700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.796159, event: op_applied, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2051> 2014-11-03 11:37:55.796304 7f2912507700 5 -- op tracker -- seq: 3149, time: 2014-11-03 11:37:55.796303, event: done, op: osd_op(client.7870547.0:3301683 rbd_data.47bd962ae8944a.0000000000000820 [set-alloc-hint object_size 4194304 write_size 4194304,write 1048576~4096] 24.e1d92f61 snapc 2de=[2de] ack+ondisk+write e105950) -2050> 2014-11-03 11:37:55.796473 7f2912507700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.796472, event: sub_op_applied, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2049> 2014-11-03 11:37:55.796522 7f2912507700 5 -- op tracker -- seq: 3201, time: 2014-11-03 11:37:55.796522, event: done, op: osd_sub_op(client.7912219.0:1154964 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187827 snapset=0=[]:[] snapc=0=[]) -2048> 2014-11-03 11:37:55.796587 7f2912507700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.796587, event: sub_op_applied, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2047> 2014-11-03 11:37:55.796624 7f2912507700 5 -- op tracker -- seq: 3202, time: 2014-11-03 11:37:55.796624, event: done, op: osd_sub_op(client.7912181.0:1550571 6.4f0 95d764f0/rbd_data.75551c509c613c.00000000000009f5/head//6 [] v 105950'13100469 snapset=0=[]:[] snapc=0=[]) -2046> 2014-11-03 11:37:55.796825 7f2912507700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.796825, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2045> 2014-11-03 11:37:55.796900 7f2912507700 5 -- op tracker -- seq: 3203, time: 2014-11-03 11:37:55.796900, event: done, op: osd_sub_op(client.7869068.0:21687452 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134376 snapset=0=[]:[] snapc=0=[]) -2044> 2014-11-03 11:37:55.796942 7f2912507700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.796942, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2043> 2014-11-03 11:37:55.797018 7f2912507700 5 -- op tracker -- seq: 3204, time: 2014-11-03 11:37:55.797018, event: done, op: osd_sub_op(client.12146205.0:439318 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222071 snapset=0=[]:[] snapc=0=[]) -2042> 2014-11-03 11:37:55.797462 7f2912507700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.797462, event: sub_op_applied, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2041> 2014-11-03 11:37:55.797504 7f2912507700 5 -- op tracker -- seq: 3205, time: 2014-11-03 11:37:55.797504, event: done, op: osd_sub_op(client.12146205.0:439319 24.3a1 20803a1/rbd_data.b9a55d6dc18057.0000000000000165/head//24 [] v 105950'222072 snapset=0=[]:[] snapc=0=[]) -2040> 2014-11-03 11:37:55.797583 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 40 ==== MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 v2 ==== 26+0+0 (684085172 0 0) 0x9459c00 con 0x22ff3420 -2039> 2014-11-03 11:37:55.797625 7f290b031700 5 -- op tracker -- seq: 3208, time: 2014-11-03 11:37:55.797440, event: header_read, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2038> 2014-11-03 11:37:55.797637 7f290b031700 5 -- op tracker -- seq: 3208, time: 2014-11-03 11:37:55.797442, event: throttled, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2037> 2014-11-03 11:37:55.797644 7f290b031700 5 -- op tracker -- seq: 3208, time: 2014-11-03 11:37:55.797525, event: all_read, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2036> 2014-11-03 11:37:55.797653 7f290b031700 5 -- op tracker -- seq: 3208, time: 2014-11-03 11:37:55.797619, event: dispatched, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2035> 2014-11-03 11:37:55.797664 7f290b031700 5 -- op tracker -- seq: 3208, time: 2014-11-03 11:37:55.797664, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2034> 2014-11-03 11:37:55.797701 7f290b031700 5 -- op tracker -- seq: 3208, time: 2014-11-03 11:37:55.797701, event: done, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2033> 2014-11-03 11:37:55.797773 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6833/17474 -- MRecoveryReserve REQUEST pgid: 24.53e, query_epoch: 105950 v2 -- ?+0 0x25d06800 con 0x22fe0dc0 -2032> 2014-11-03 11:37:55.799541 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.107 192.168.5.8:6833/17474 36 ==== MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 v2 ==== 26+0+0 (684085172 0 0) 0x25d06800 con 0x22fe0dc0 -2031> 2014-11-03 11:37:55.799576 7f290b031700 5 -- op tracker -- seq: 3209, time: 2014-11-03 11:37:55.799392, event: header_read, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2030> 2014-11-03 11:37:55.799611 7f290b031700 5 -- op tracker -- seq: 3209, time: 2014-11-03 11:37:55.799393, event: throttled, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2029> 2014-11-03 11:37:55.799634 7f290b031700 5 -- op tracker -- seq: 3209, time: 2014-11-03 11:37:55.799443, event: all_read, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2028> 2014-11-03 11:37:55.799652 7f290b031700 5 -- op tracker -- seq: 3209, time: 2014-11-03 11:37:55.799571, event: dispatched, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2027> 2014-11-03 11:37:55.799673 7f290b031700 5 -- op tracker -- seq: 3209, time: 2014-11-03 11:37:55.799673, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2026> 2014-11-03 11:37:55.799721 7f290b031700 5 -- op tracker -- seq: 3209, time: 2014-11-03 11:37:55.799721, event: done, op: MRecoveryReserve GRANT pgid: 24.53e, query_epoch: 105950 -2025> 2014-11-03 11:37:55.799744 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 lc 105946'131027 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105946'131027 active+recovery_wait+degraded m=1] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.003759 2 0.000151 -2024> 2014-11-03 11:37:55.799786 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 lc 105946'131027 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105946'131027 active+recovery_wait+degraded m=1] enter Started/Primary/Active/Recovering -2023> 2014-11-03 11:37:55.799951 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- MOSDPGPull(24.53e 105950 [PullOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x229e7800 con 0x22ff3420 -2022> 2014-11-03 11:37:55.830354 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.7 192.168.5.1:0/3511 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.829808) v2 ==== 47+0+0 (477035752 0 0) 0x187f4200 con 0x24925000 -2021> 2014-11-03 11:37:55.830427 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.829808) v2 -- ?+0 0x29fcda00 con 0x24925000 -2020> 2014-11-03 11:37:55.830536 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.7 192.168.5.1:0/3511 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.829808) v2 ==== 47+0+0 (477035752 0 0) 0x13d27c00 con 0x23ea4c60 -2019> 2014-11-03 11:37:55.830581 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.829808) v2 -- ?+0 0x2353b400 con 0x23ea4c60 -2018> 2014-11-03 11:37:55.850780 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.39 192.168.5.3:0/23401 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.850325) v2 ==== 47+0+0 (1123042677 0 0) 0x23cbe000 con 0x2389f000 -2017> 2014-11-03 11:37:55.850833 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/23401 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.850325) v2 -- ?+0 0x13d27c00 con 0x2389f000 -2016> 2014-11-03 11:37:55.851258 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.39 192.168.5.3:0/23401 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.850325) v2 ==== 47+0+0 (1123042677 0 0) 0x22fad000 con 0x31a9000 -2015> 2014-11-03 11:37:55.851340 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/23401 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.850325) v2 -- ?+0 0x187f4200 con 0x31a9000 -2014> 2014-11-03 11:37:55.852092 7f28f0b97700 1 -- 192.168.5.2:6803/7606 <== osd.45 192.168.5.4:6821/1781 41 ==== MOSDPGPush(24.53e 105950 [PushOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, version: 105946'131028, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195218+0+0 (3596045956 0 0) 0x229e7800 con 0x22ff3420 -2013> 2014-11-03 11:37:55.852174 7f28f0b97700 5 -- op tracker -- seq: 3210, time: 2014-11-03 11:37:55.838420, event: header_read, op: MOSDPGPush(24.53e 105950 [PushOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, version: 105946'131028, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2012> 2014-11-03 11:37:55.852210 7f28f0b97700 5 -- op tracker -- seq: 3210, time: 2014-11-03 11:37:55.838423, event: throttled, op: MOSDPGPush(24.53e 105950 [PushOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, version: 105946'131028, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2011> 2014-11-03 11:37:55.852238 7f28f0b97700 5 -- op tracker -- seq: 3210, time: 2014-11-03 11:37:55.852075, event: all_read, op: MOSDPGPush(24.53e 105950 [PushOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, version: 105946'131028, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2010> 2014-11-03 11:37:55.852267 7f28f0b97700 5 -- op tracker -- seq: 3210, time: 0.000000, event: dispatched, op: MOSDPGPush(24.53e 105950 [PushOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, version: 105946'131028, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2009> 2014-11-03 11:37:55.852367 7f28fe818700 5 -- op tracker -- seq: 3210, time: 2014-11-03 11:37:55.852367, event: reached_pg, op: MOSDPGPush(24.53e 105950 [PushOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, version: 105946'131028, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2008> 2014-11-03 11:37:55.852675 7f28fe818700 5 -- op tracker -- seq: 3210, time: 2014-11-03 11:37:55.852675, event: done, op: MOSDPGPush(24.53e 105950 [PushOp(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24, version: 105946'131028, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(a209953e/rbd_data.5b871a2ae8944a.00000000000010f3/head//24@105946'131028, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -2007> 2014-11-03 11:37:55.872451 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 102 ==== osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (1160424561 0 3511520646) 0x2c337800 con 0x2308b840 -2006> 2014-11-03 11:37:55.872501 7f28ef985700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.872320, event: header_read, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -2005> 2014-11-03 11:37:55.872523 7f28ef985700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.872322, event: throttled, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -2004> 2014-11-03 11:37:55.872538 7f28ef985700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.872431, event: all_read, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -2003> 2014-11-03 11:37:55.872554 7f28ef985700 5 -- op tracker -- seq: 3211, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -2002> 2014-11-03 11:37:55.872630 7f28fe017700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.872629, event: reached_pg, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -2001> 2014-11-03 11:37:55.872670 7f28fe017700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.872670, event: started, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -2000> 2014-11-03 11:37:55.872779 7f28fe017700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.872778, event: started, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -1999> 2014-11-03 11:37:55.872816 7f28fe017700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.872816, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -1998> 2014-11-03 11:37:55.873543 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 103 ==== osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (2221361753 0 3511520646) 0x2c336000 con 0x2308b840 -1997> 2014-11-03 11:37:55.873591 7f28ef985700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.873431, event: header_read, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1996> 2014-11-03 11:37:55.873613 7f28ef985700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.873433, event: throttled, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1995> 2014-11-03 11:37:55.873628 7f28ef985700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.873527, event: all_read, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1994> 2014-11-03 11:37:55.873661 7f28ef985700 5 -- op tracker -- seq: 3212, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1993> 2014-11-03 11:37:55.873735 7f290081c700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.873735, event: reached_pg, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1992> 2014-11-03 11:37:55.873778 7f290081c700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.873778, event: started, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1991> 2014-11-03 11:37:55.873907 7f290081c700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.873907, event: started, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1990> 2014-11-03 11:37:55.873947 7f290081c700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.873947, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1989> 2014-11-03 11:37:55.874510 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.47 192.168.5.4:0/12077 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.872834) v2 ==== 47+0+0 (331498303 0 0) 0x3353400 con 0x243216e0 -1988> 2014-11-03 11:37:55.874580 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.872834) v2 -- ?+0 0x23cbe000 con 0x243216e0 -1987> 2014-11-03 11:37:55.874518 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.47 192.168.5.4:0/12077 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.872834) v2 ==== 47+0+0 (331498303 0 0) 0x2a8efa00 con 0x242926e0 -1986> 2014-11-03 11:37:55.874759 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/12077 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.872834) v2 -- ?+0 0x22fad000 con 0x242926e0 -1985> 2014-11-03 11:37:55.879206 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x23614340 sd=66 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9a8000).accept sd=66 192.168.5.2:42986/0 -1984> 2014-11-03 11:37:55.880243 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419985 192.168.5.2:0/1004437 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x27150000 con 0x1c9a8000 -1983> 2014-11-03 11:37:55.880974 7f28fb812700 0 osd.21 105950 do_command r=0 -1982> 2014-11-03 11:37:55.881013 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004437 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x1e798000 con 0x1c9a8000 -1981> 2014-11-03 11:37:55.884397 7f28f6df9700 1 -- 192.168.5.2:6803/7606 <== osd.1 192.168.5.1:6807/11054 52 ==== osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1248015649 0 3511520646) 0x2c690c00 con 0x22fbc160 -1980> 2014-11-03 11:37:55.884443 7f28f6df9700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.884242, event: header_read, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1979> 2014-11-03 11:37:55.884467 7f28f6df9700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.884244, event: throttled, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1978> 2014-11-03 11:37:55.884485 7f28f6df9700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.884382, event: all_read, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1977> 2014-11-03 11:37:55.884507 7f28f6df9700 5 -- op tracker -- seq: 3213, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1976> 2014-11-03 11:37:55.884575 7f290101d700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.884575, event: reached_pg, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1975> 2014-11-03 11:37:55.884607 7f290101d700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.884607, event: started, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1974> 2014-11-03 11:37:55.884798 7f290101d700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.884797, event: started, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1973> 2014-11-03 11:37:55.884857 7f290101d700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.884857, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1972> 2014-11-03 11:37:55.885228 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 90 ==== osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (4077378221 0 3511520646) 0x2448d000 con 0x22fef420 -1971> 2014-11-03 11:37:55.885299 7f28f41cd700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.885073, event: header_read, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1970> 2014-11-03 11:37:55.885321 7f28f41cd700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.885076, event: throttled, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1969> 2014-11-03 11:37:55.885356 7f28f41cd700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.885206, event: all_read, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1968> 2014-11-03 11:37:55.885366 7f28f41cd700 5 -- op tracker -- seq: 3214, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1967> 2014-11-03 11:37:55.886172 7f28fe818700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.886172, event: reached_pg, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1966> 2014-11-03 11:37:55.886213 7f28fe818700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.886213, event: started, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1965> 2014-11-03 11:37:55.886315 7f28fe818700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.886315, event: started, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1964> 2014-11-03 11:37:55.886363 7f28fe818700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.886363, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1963> 2014-11-03 11:37:55.886725 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 73 ==== osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (27115436 0 3511520646) 0x24663400 con 0x22fbc840 -1962> 2014-11-03 11:37:55.886761 7f28f41cd700 1 -- 192.168.5.2:6803/7606 <== osd.76 192.168.5.6:6836/22493 91 ==== osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (3767663710 0 3511520646) 0x26c42c00 con 0x22fef420 -1961> 2014-11-03 11:37:55.886814 7f28f41cd700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.886485, event: header_read, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1960> 2014-11-03 11:37:55.886850 7f28f41cd700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.886487, event: throttled, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1959> 2014-11-03 11:37:55.886793 7f28f68f4700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.886538, event: header_read, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1958> 2014-11-03 11:37:55.886876 7f28f41cd700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.886742, event: all_read, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1957> 2014-11-03 11:37:55.886882 7f28f68f4700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.886542, event: throttled, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1956> 2014-11-03 11:37:55.886897 7f28f41cd700 5 -- op tracker -- seq: 3216, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1955> 2014-11-03 11:37:55.886914 7f28f68f4700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.886694, event: all_read, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1954> 2014-11-03 11:37:55.886936 7f28f68f4700 5 -- op tracker -- seq: 3215, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1953> 2014-11-03 11:37:55.887051 7f28fe017700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.887050, event: reached_pg, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1952> 2014-11-03 11:37:55.887105 7f28fe017700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.887105, event: started, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1951> 2014-11-03 11:37:55.887275 7f28fe017700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.887275, event: started, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1950> 2014-11-03 11:37:55.887309 7f28fe017700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.887309, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1949> 2014-11-03 11:37:55.887493 7f290101d700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.887493, event: reached_pg, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1948> 2014-11-03 11:37:55.887536 7f290101d700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.887536, event: started, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1947> 2014-11-03 11:37:55.887655 7f290101d700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.887654, event: started, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1946> 2014-11-03 11:37:55.887690 7f290101d700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.887690, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1945> 2014-11-03 11:37:55.888035 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 74 ==== osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (2484733948 0 3511520646) 0x24661000 con 0x22fbc840 -1944> 2014-11-03 11:37:55.888071 7f28f68f4700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.887923, event: header_read, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1943> 2014-11-03 11:37:55.888094 7f28f68f4700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.887924, event: throttled, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1942> 2014-11-03 11:37:55.888113 7f28f68f4700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.888024, event: all_read, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1941> 2014-11-03 11:37:55.888131 7f28f68f4700 5 -- op tracker -- seq: 3217, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1940> 2014-11-03 11:37:55.888195 7f290081c700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.888195, event: reached_pg, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1939> 2014-11-03 11:37:55.888232 7f290081c700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.888232, event: started, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1938> 2014-11-03 11:37:55.888335 7f290081c700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.888335, event: started, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1937> 2014-11-03 11:37:55.888369 7f290081c700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.888369, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1936> 2014-11-03 11:37:55.889707 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 75 ==== osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (657822832 0 3511520646) 0x26dba000 con 0x22fbc840 -1935> 2014-11-03 11:37:55.889739 7f28f68f4700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.889630, event: header_read, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1934> 2014-11-03 11:37:55.889756 7f28f68f4700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.889631, event: throttled, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1933> 2014-11-03 11:37:55.889768 7f28f68f4700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.889696, event: all_read, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1932> 2014-11-03 11:37:55.889780 7f28f68f4700 5 -- op tracker -- seq: 3218, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1931> 2014-11-03 11:37:55.889855 7f28fe017700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.889855, event: reached_pg, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1930> 2014-11-03 11:37:55.889907 7f28fe017700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.889907, event: started, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1929> 2014-11-03 11:37:55.890104 7f28fe017700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.890103, event: started, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1928> 2014-11-03 11:37:55.890201 7f28fe017700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.890201, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1927> 2014-11-03 11:37:55.893971 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419985 192.168.5.2:0/1004437 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2353b400 con 0x1c9a8000 -1926> 2014-11-03 11:37:55.894152 7f28f0a96700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 95 ==== MOSDPGPush(24.74d 105950 [PushOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24, version: 105938'157337, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24@105938'157337, copy_subset: [0~4194304], clone_subset: {5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/2fc//24=[]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24, version: 105946'157340, data_included: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], data_size: 2605056, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24@105946'157340, copy_subset: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], clone_subset: {3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/130//24=[1540096~16384,1589248~49152,1654784~16384,1687552~212992,1916928~32768,1966080~114688,3014656~131072,3178496~1015808]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3178496, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 6802119+0+0 (193265423 0 0) 0x2a6e8400 con 0x230a92c0 -1925> 2014-11-03 11:37:55.894259 7f28f0a96700 5 -- op tracker -- seq: 3219, time: 2014-11-03 11:37:55.872472, event: header_read, op: MOSDPGPush(24.74d 105950 [PushOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24, version: 105938'157337, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24@105938'157337, copy_subset: [0~4194304], clone_subset: {5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/2fc//24=[]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24, version: 105946'157340, data_included: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], data_size: 2605056, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24@105946'157340, copy_subset: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], clone_subset: {3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/130//24=[1540096~16384,1589248~49152,1654784~16384,1687552~212992,1916928~32768,1966080~114688,3014656~131072,3178496~1015808]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3178496, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1924> 2014-11-03 11:37:55.894305 7f28f0a96700 5 -- op tracker -- seq: 3219, time: 2014-11-03 11:37:55.872475, event: throttled, op: MOSDPGPush(24.74d 105950 [PushOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24, version: 105938'157337, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24@105938'157337, copy_subset: [0~4194304], clone_subset: {5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/2fc//24=[]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24, version: 105946'157340, data_included: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], data_size: 2605056, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24@105946'157340, copy_subset: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], clone_subset: {3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/130//24=[1540096~16384,1589248~49152,1654784~16384,1687552~212992,1916928~32768,1966080~114688,3014656~131072,3178496~1015808]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3178496, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1923> 2014-11-03 11:37:55.894335 7f28f0a96700 5 -- op tracker -- seq: 3219, time: 2014-11-03 11:37:55.894140, event: all_read, op: MOSDPGPush(24.74d 105950 [PushOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24, version: 105938'157337, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24@105938'157337, copy_subset: [0~4194304], clone_subset: {5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/2fc//24=[]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24, version: 105946'157340, data_included: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], data_size: 2605056, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24@105946'157340, copy_subset: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], clone_subset: {3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/130//24=[1540096~16384,1589248~49152,1654784~16384,1687552~212992,1916928~32768,1966080~114688,3014656~131072,3178496~1015808]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3178496, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1922> 2014-11-03 11:37:55.894365 7f28f0a96700 5 -- op tracker -- seq: 3219, time: 0.000000, event: dispatched, op: MOSDPGPush(24.74d 105950 [PushOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24, version: 105938'157337, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24@105938'157337, copy_subset: [0~4194304], clone_subset: {5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/2fc//24=[]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24, version: 105946'157340, data_included: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], data_size: 2605056, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24@105946'157340, copy_subset: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], clone_subset: {3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/130//24=[1540096~16384,1589248~49152,1654784~16384,1687552~212992,1916928~32768,1966080~114688,3014656~131072,3178496~1015808]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3178496, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1921> 2014-11-03 11:37:55.894443 7f290001b700 5 -- op tracker -- seq: 3219, time: 2014-11-03 11:37:55.894443, event: reached_pg, op: MOSDPGPush(24.74d 105950 [PushOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24, version: 105938'157337, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24@105938'157337, copy_subset: [0~4194304], clone_subset: {5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/2fc//24=[]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24, version: 105946'157340, data_included: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], data_size: 2605056, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24@105946'157340, copy_subset: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], clone_subset: {3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/130//24=[1540096~16384,1589248~49152,1654784~16384,1687552~212992,1916928~32768,1966080~114688,3014656~131072,3178496~1015808]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3178496, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1920> 2014-11-03 11:37:55.894790 7f290001b700 5 -- op tracker -- seq: 3219, time: 2014-11-03 11:37:55.894790, event: done, op: MOSDPGPush(24.74d 105950 [PushOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24, version: 105938'157337, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24@105938'157337, copy_subset: [0~4194304], clone_subset: {5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/2fc//24=[]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24, version: 105946'157340, data_included: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24@105946'157340, copy_subset: [0~1540096,1556480~32768,1638400~16384,1671168~16384,1900544~16384,1949696~16384,2080768~933888,3145728~32768], clone_subset: {3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/130//24=[1540096~16384,1589248~49152,1654784~16384,1687552~212992,1916928~32768,1966080~114688,3014656~131072,3178496~1015808]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:3178496, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1919> 2014-11-03 11:37:55.895068 7f28fb812700 0 osd.21 105950 do_command r=0 -1918> 2014-11-03 11:37:55.895085 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004437 -- command_reply(tid 2: 0 ) v1 -- ?+14214 0x23d89700 con 0x1c9a8000 -1917> 2014-11-03 11:37:55.896152 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.7870073 192.168.1.72:0/2026476 8 ==== ping magic: 0 v1 ==== 0+0+0 (0 0 0) 0x235ef1c0 con 0x2311c2c0 -1916> 2014-11-03 11:37:55.901750 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004437 pipe(0x23614340 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9a8000).reader couldn't read tag, (0) Success -1915> 2014-11-03 11:37:55.901837 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004437 pipe(0x23614340 sd=66 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9a8000).fault (0) Success -1914> 2014-11-03 11:37:55.901933 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9a8000 session 0x7c35c80 -1913> 2014-11-03 11:37:55.902741 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 195 ==== osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2838414675 0 3511520646) 0x23d99000 con 0x22fefdc0 -1912> 2014-11-03 11:37:55.902790 7f28f36c2700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.902601, event: header_read, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1911> 2014-11-03 11:37:55.902818 7f28f36c2700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.902604, event: throttled, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1910> 2014-11-03 11:37:55.902833 7f28f36c2700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.902727, event: all_read, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1909> 2014-11-03 11:37:55.902847 7f28f36c2700 5 -- op tracker -- seq: 3220, time: 0.000000, event: dispatched, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1908> 2014-11-03 11:37:55.902963 7f290201f700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.902963, event: reached_pg, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1907> 2014-11-03 11:37:55.903012 7f290201f700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.903012, event: started, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1906> 2014-11-03 11:37:55.903174 7f290201f700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.903174, event: started, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1905> 2014-11-03 11:37:55.903214 7f290201f700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.903214, event: commit_queued_for_journal_write, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1904> 2014-11-03 11:37:55.903861 7f2914d0c700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.903860, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -1903> 2014-11-03 11:37:55.903938 7f2914d0c700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.903937, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1902> 2014-11-03 11:37:55.903982 7f2914d0c700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.903982, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1901> 2014-11-03 11:37:55.904021 7f2914d0c700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.904021, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1900> 2014-11-03 11:37:55.904066 7f2914d0c700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.904066, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1899> 2014-11-03 11:37:55.904102 7f2914d0c700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.904101, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1898> 2014-11-03 11:37:55.904149 7f2914d0c700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.904149, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1897> 2014-11-03 11:37:55.904179 7f2914d0c700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.904179, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1896> 2014-11-03 11:37:55.906101 7f2914d0c700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.906101, event: write_thread_in_journal_buffer, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1895> 2014-11-03 11:37:55.909797 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 113 ==== osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (827276230 0 3511520646) 0x25eb7c00 con 0x22fe0c60 -1894> 2014-11-03 11:37:55.909833 7f28f4eda700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.909695, event: header_read, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1893> 2014-11-03 11:37:55.909848 7f28f4eda700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.909697, event: throttled, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1892> 2014-11-03 11:37:55.909857 7f28f4eda700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.909783, event: all_read, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1891> 2014-11-03 11:37:55.909899 7f28f4eda700 5 -- op tracker -- seq: 3221, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1890> 2014-11-03 11:37:55.910006 7f290081c700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.910006, event: reached_pg, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1889> 2014-11-03 11:37:55.910078 7f290081c700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.910078, event: started, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1888> 2014-11-03 11:37:55.910238 7f290081c700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.910238, event: started, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1887> 2014-11-03 11:37:55.910279 7f290081c700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.910279, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1886> 2014-11-03 11:37:55.911066 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 114 ==== osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2614112886 0 3511520646) 0x25eb7000 con 0x22fe0c60 -1885> 2014-11-03 11:37:55.911129 7f28f4eda700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.910891, event: header_read, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1884> 2014-11-03 11:37:55.911154 7f28f4eda700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.910893, event: throttled, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1883> 2014-11-03 11:37:55.911178 7f28f4eda700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.911042, event: all_read, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1882> 2014-11-03 11:37:55.911193 7f28f4eda700 5 -- op tracker -- seq: 3222, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1881> 2014-11-03 11:37:55.911294 7f28fe017700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.911294, event: reached_pg, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1880> 2014-11-03 11:37:55.911336 7f28fe017700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.911336, event: started, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1879> 2014-11-03 11:37:55.911535 7f28fe017700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.911535, event: started, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1878> 2014-11-03 11:37:55.911588 7f28fe017700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.911588, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1877> 2014-11-03 11:37:55.913160 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 115 ==== osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1227148473 0 3511520646) 0x24da3400 con 0x22fe0c60 -1876> 2014-11-03 11:37:55.913193 7f28f4eda700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.913064, event: header_read, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1875> 2014-11-03 11:37:55.913228 7f28f4eda700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.913066, event: throttled, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1874> 2014-11-03 11:37:55.913237 7f28f4eda700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.913147, event: all_read, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1873> 2014-11-03 11:37:55.913246 7f28f4eda700 5 -- op tracker -- seq: 3223, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1872> 2014-11-03 11:37:55.913351 7f290081c700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.913351, event: reached_pg, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1871> 2014-11-03 11:37:55.913403 7f290081c700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.913403, event: started, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1870> 2014-11-03 11:37:55.913546 7f290081c700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.913546, event: started, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1869> 2014-11-03 11:37:55.913584 7f290081c700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.913584, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1868> 2014-11-03 11:37:55.914082 7f291450b700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.914082, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -1867> 2014-11-03 11:37:55.914141 7f2911d06700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.914141, event: commit_sent, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -1866> 2014-11-03 11:37:55.914184 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23e43340 con 0x2308b840 -1865> 2014-11-03 11:37:55.914203 7f291450b700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.914202, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1864> 2014-11-03 11:37:55.914235 7f291450b700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.914235, event: journaled_completion_queued, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1863> 2014-11-03 11:37:55.914235 7f2911d06700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.914235, event: commit_sent, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1862> 2014-11-03 11:37:55.914258 7f291450b700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.914258, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1861> 2014-11-03 11:37:55.914319 7f291450b700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.914319, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1860> 2014-11-03 11:37:55.914319 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23e43080 con 0x2308b840 -1859> 2014-11-03 11:37:55.914345 7f291450b700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.914345, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1858> 2014-11-03 11:37:55.914362 7f291450b700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.914362, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1857> 2014-11-03 11:37:55.914367 7f2911d06700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.914367, event: commit_sent, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1856> 2014-11-03 11:37:55.914378 7f291450b700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.914378, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1855> 2014-11-03 11:37:55.914419 7f2912507700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.914419, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -1854> 2014-11-03 11:37:55.914425 7f291450b700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.914424, event: journaled_completion_queued, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1853> 2014-11-03 11:37:55.914449 7f2912507700 5 -- op tracker -- seq: 3211, time: 2014-11-03 11:37:55.914449, event: done, op: osd_sub_op(client.7867942.0:137198170 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000606 snapset=0=[]:[] snapc=0=[]) -1852> 2014-11-03 11:37:55.914443 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6807/11054 -- osd_sub_op_reply(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650d840 con 0x22fbc160 -1851> 2014-11-03 11:37:55.914484 7f2911d06700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.914484, event: commit_sent, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1850> 2014-11-03 11:37:55.914501 7f2914d0c700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.914501, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1849> 2014-11-03 11:37:55.914516 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650db00 con 0x22fef420 -1848> 2014-11-03 11:37:55.914564 7f2914d0c700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.914564, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1847> 2014-11-03 11:37:55.914568 7f2911d06700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.914568, event: commit_sent, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1846> 2014-11-03 11:37:55.914599 7f2912507700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.914598, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1845> 2014-11-03 11:37:55.914606 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa0008c0 con 0x22fbc840 -1844> 2014-11-03 11:37:55.914627 7f2914d0c700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.914626, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1843> 2014-11-03 11:37:55.914632 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 116 ==== osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1449280787 0 3511520646) 0x24da2800 con 0x22fe0c60 -1842> 2014-11-03 11:37:55.914644 7f2912507700 5 -- op tracker -- seq: 3212, time: 2014-11-03 11:37:55.914644, event: done, op: osd_sub_op(client.7867942.0:137198171 6.508 c6d12d08/rbd_data.43c40916a10.00000000000005b3/head//6 [] v 105950'20000607 snapset=0=[]:[] snapc=0=[]) -1841> 2014-11-03 11:37:55.914663 7f28f4eda700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.914543, event: header_read, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1840> 2014-11-03 11:37:55.914683 7f28f4eda700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.914545, event: throttled, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1839> 2014-11-03 11:37:55.914662 7f2911d06700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.914662, event: commit_sent, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1838> 2014-11-03 11:37:55.914698 7f28f4eda700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.914620, event: all_read, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1837> 2014-11-03 11:37:55.914720 7f28f4eda700 5 -- op tracker -- seq: 3224, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1836> 2014-11-03 11:37:55.914744 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6836/22493 -- osd_sub_op_reply(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0xa000b80 con 0x22fef420 -1835> 2014-11-03 11:37:55.914773 7f28fe017700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.914773, event: reached_pg, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1834> 2014-11-03 11:37:55.914775 7f2911d06700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.914774, event: commit_sent, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1833> 2014-11-03 11:37:55.914775 7f2912507700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.914774, event: sub_op_applied, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1832> 2014-11-03 11:37:55.914797 7f28fe017700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.914797, event: started, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1831> 2014-11-03 11:37:55.914816 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x9fff840 con 0x22fbc840 -1830> 2014-11-03 11:37:55.914856 7f2911d06700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.914856, event: commit_sent, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1829> 2014-11-03 11:37:55.914863 7f2912507700 5 -- op tracker -- seq: 3213, time: 2014-11-03 11:37:55.914863, event: done, op: osd_sub_op(client.7867498.0:12568353 6.5ca 35d3adca/rbd_data.45e62779c99cf1.00000000000025d2/head//6 [] v 105950'9539480 snapset=0=[]:[] snapc=0=[]) -1828> 2014-11-03 11:37:55.914898 7f28fe017700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.914898, event: started, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1827> 2014-11-03 11:37:55.914935 7f28fe017700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.914935, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1826> 2014-11-03 11:37:55.914960 7f2912507700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.914960, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1825> 2014-11-03 11:37:55.914971 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23925840 con 0x22fbc840 -1824> 2014-11-03 11:37:55.915004 7f2912507700 5 -- op tracker -- seq: 3214, time: 2014-11-03 11:37:55.915004, event: done, op: osd_sub_op(client.7867942.0:137198219 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057570 snapset=0=[]:[] snapc=0=[]) -1823> 2014-11-03 11:37:55.915059 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- pg_trim(24.74d to 105946'157340 e105950) v1 -- ?+0 0xc157e00 con 0x230a92c0 -1822> 2014-11-03 11:37:55.915108 7f2911d06700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.915108, event: commit_sent, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1821> 2014-11-03 11:37:55.915127 7f2912507700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.915127, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1820> 2014-11-03 11:37:55.915142 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94d080 con 0x22fefdc0 -1819> 2014-11-03 11:37:55.915150 7f2912507700 5 -- op tracker -- seq: 3215, time: 2014-11-03 11:37:55.915150, event: done, op: osd_sub_op(client.7867942.0:137198226 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195967 snapset=0=[]:[] snapc=0=[]) -1818> 2014-11-03 11:37:55.915288 7f2912507700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.915288, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1817> 2014-11-03 11:37:55.915334 7f2912507700 5 -- op tracker -- seq: 3216, time: 2014-11-03 11:37:55.915333, event: done, op: osd_sub_op(client.7867942.0:137198220 6.755 fe9acf55/rbd_data.43c40916a10.00000000000005f9/head//6 [] v 105950'16057571 snapset=0=[]:[] snapc=0=[]) -1816> 2014-11-03 11:37:55.915443 7f2912507700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.915443, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1815> 2014-11-03 11:37:55.915468 7f2912507700 5 -- op tracker -- seq: 3217, time: 2014-11-03 11:37:55.915468, event: done, op: osd_sub_op(client.7867942.0:137198227 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195968 snapset=0=[]:[] snapc=0=[]) -1814> 2014-11-03 11:37:55.915593 7f2912507700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.915593, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1813> 2014-11-03 11:37:55.915627 7f2912507700 5 -- op tracker -- seq: 3218, time: 2014-11-03 11:37:55.915627, event: done, op: osd_sub_op(client.7867942.0:137198228 6.a8a 45f5aa8a/rbd_data.43c40916a10.000000000000070e/head//6 [] v 105950'8195969 snapset=0=[]:[] snapc=0=[]) -1812> 2014-11-03 11:37:55.915727 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 117 ==== osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2798501546 0 3511520646) 0x24da1c00 con 0x22fe0c60 -1811> 2014-11-03 11:37:55.915759 7f28f4eda700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.915616, event: header_read, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1810> 2014-11-03 11:37:55.915777 7f28f4eda700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.915617, event: throttled, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1809> 2014-11-03 11:37:55.915798 7f28f4eda700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.915717, event: all_read, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1808> 2014-11-03 11:37:55.915818 7f28f4eda700 5 -- op tracker -- seq: 3225, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1807> 2014-11-03 11:37:55.915898 7f290081c700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.915898, event: reached_pg, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1806> 2014-11-03 11:37:55.915979 7f290081c700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.915979, event: started, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1805> 2014-11-03 11:37:55.916155 7f290081c700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.916154, event: started, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1804> 2014-11-03 11:37:55.916194 7f290081c700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.916194, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1803> 2014-11-03 11:37:55.916986 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 118 ==== osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3354169126 0 3511520646) 0x24da1000 con 0x22fe0c60 -1802> 2014-11-03 11:37:55.917019 7f28f4eda700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.916906, event: header_read, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1801> 2014-11-03 11:37:55.917036 7f28f4eda700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.916907, event: throttled, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1800> 2014-11-03 11:37:55.917045 7f28f4eda700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.916969, event: all_read, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1799> 2014-11-03 11:37:55.917058 7f28f4eda700 5 -- op tracker -- seq: 3226, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1798> 2014-11-03 11:37:55.917099 7f28fe017700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.917099, event: reached_pg, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1797> 2014-11-03 11:37:55.917126 7f28fe017700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.917126, event: started, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1796> 2014-11-03 11:37:55.917205 7f28fe017700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.917204, event: started, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1795> 2014-11-03 11:37:55.917232 7f28fe017700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.917232, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1794> 2014-11-03 11:37:55.949637 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- MRecoveryReserve RELEASE pgid: 24.53e, query_epoch: 105950 v2 -- ?+0 0x25d06c00 con 0x22ff3420 -1793> 2014-11-03 11:37:55.949682 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6833/17474 -- MRecoveryReserve RELEASE pgid: 24.53e, query_epoch: 105950 v2 -- ?+0 0x2cf94e00 con 0x22fe0dc0 -1792> 2014-11-03 11:37:55.949708 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105950'131095 active+degraded] exit Started/Primary/Active/Recovering 0.149922 1 0.000100 -1791> 2014-11-03 11:37:55.949711 7f2912507700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.949711, event: sub_op_applied, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1790> 2014-11-03 11:37:55.949777 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105950'131095 active+degraded] enter Started/Primary/Active/Recovered -1789> 2014-11-03 11:37:55.949818 7f2912507700 5 -- op tracker -- seq: 3220, time: 2014-11-03 11:37:55.949818, event: done, op: osd_sub_op(client.12404072.0:246045 6.799 73a35799/rbd_data.972f5364f2ae11.0000000000000305/head//6 [] v 105950'15033688 snapset=0=[]:[] snapc=0=[]) -1788> 2014-11-03 11:37:55.949828 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105950'131095 active] exit Started/Primary/Active/Recovered 0.000052 0 0.000000 -1787> 2014-11-03 11:37:55.949853 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.53e( v 105950'131096 (105586'123283,105950'131096] local-les=105948 n=172 ec=25000 les/c 105948/105943 105947/105947/105947) [21,45,107] r=0 lpr=105947 pi=105927-105946/6 crt=105946'131084 mlcod 105950'131095 active] enter Started/Primary/Active/Clean -1786> 2014-11-03 11:37:55.949890 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.93 192.168.5.7:0/7095 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.948390) v2 ==== 47+0+0 (6174327 0 0) 0x229e7800 con 0x23ea4dc0 -1785> 2014-11-03 11:37:55.949901 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.93 192.168.5.7:0/7095 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:55.948390) v2 ==== 47+0+0 (6174327 0 0) 0x2353be00 con 0x24925160 -1784> 2014-11-03 11:37:55.949932 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/7095 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.948390) v2 -- ?+0 0x3353400 con 0x23ea4dc0 -1783> 2014-11-03 11:37:55.949936 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6821/1781 -- pg_info(1 pgs e105950:24.53e) v4 -- ?+0 0x27969600 con 0x22ff3420 -1782> 2014-11-03 11:37:55.949884 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 lc 105945'7949807 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105945'7949807 active+recovery_wait+degraded m=12 snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] exit Started/Primary/Active/WaitLocalRecoveryReserved 27.889257 117 0.000116 -1781> 2014-11-03 11:37:55.949993 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/7095 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:55.948390) v2 -- ?+0 0x2a8efa00 con 0x24925160 -1780> 2014-11-03 11:37:55.949986 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 lc 105945'7949807 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105945'7949807 active+recovery_wait+degraded m=12 snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] enter Started/Primary/Active/WaitRemoteRecoveryReserved -1779> 2014-11-03 11:37:55.950037 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- MRecoveryReserve REQUEST pgid: 6.9d8, query_epoch: 105950 v2 -- ?+0 0x247cd600 con 0x22fb7160 -1778> 2014-11-03 11:37:55.950195 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6833/17474 -- pg_info(1 pgs e105950:24.53e) v4 -- ?+0 0xa053600 con 0x22fe0dc0 -1777> 2014-11-03 11:37:55.951437 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.49 192.168.5.4:6841/21769 52 ==== MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 v2 ==== 26+0+0 (3226086829 0 0) 0x247cd600 con 0x22fb7160 -1776> 2014-11-03 11:37:55.951478 7f290b031700 5 -- op tracker -- seq: 3227, time: 2014-11-03 11:37:55.951336, event: header_read, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1775> 2014-11-03 11:37:55.951497 7f290b031700 5 -- op tracker -- seq: 3227, time: 2014-11-03 11:37:55.951338, event: throttled, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1774> 2014-11-03 11:37:55.951506 7f290b031700 5 -- op tracker -- seq: 3227, time: 2014-11-03 11:37:55.951396, event: all_read, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1773> 2014-11-03 11:37:55.951514 7f290b031700 5 -- op tracker -- seq: 3227, time: 2014-11-03 11:37:55.951459, event: dispatched, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1772> 2014-11-03 11:37:55.951527 7f290b031700 5 -- op tracker -- seq: 3227, time: 2014-11-03 11:37:55.951527, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1771> 2014-11-03 11:37:55.951569 7f290b031700 5 -- op tracker -- seq: 3227, time: 2014-11-03 11:37:55.951568, event: done, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1770> 2014-11-03 11:37:55.951619 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MRecoveryReserve REQUEST pgid: 6.9d8, query_epoch: 105950 v2 -- ?+0 0x23307e00 con 0x22fe0420 -1769> 2014-11-03 11:37:55.952895 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 40 ==== MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 v2 ==== 26+0+0 (3226086829 0 0) 0x23307e00 con 0x22fe0420 -1768> 2014-11-03 11:37:55.952916 7f290b031700 5 -- op tracker -- seq: 3228, time: 2014-11-03 11:37:55.952813, event: header_read, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1767> 2014-11-03 11:37:55.952925 7f290b031700 5 -- op tracker -- seq: 3228, time: 2014-11-03 11:37:55.952815, event: throttled, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1766> 2014-11-03 11:37:55.952930 7f290b031700 5 -- op tracker -- seq: 3228, time: 2014-11-03 11:37:55.952862, event: all_read, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1765> 2014-11-03 11:37:55.952965 7f290b031700 5 -- op tracker -- seq: 3228, time: 2014-11-03 11:37:55.952913, event: dispatched, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1764> 2014-11-03 11:37:55.952993 7f290b031700 5 -- op tracker -- seq: 3228, time: 2014-11-03 11:37:55.952993, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1763> 2014-11-03 11:37:55.953021 7f290b031700 5 -- op tracker -- seq: 3228, time: 2014-11-03 11:37:55.953021, event: done, op: MRecoveryReserve GRANT pgid: 6.9d8, query_epoch: 105950 -1762> 2014-11-03 11:37:55.953043 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 lc 105945'7949807 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105945'7949807 active+recovery_wait+degraded m=12 snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.003058 2 0.000242 -1761> 2014-11-03 11:37:55.953147 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 lc 105945'7949807 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105945'7949807 active+recovery_wait+degraded m=12 snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] enter Started/Primary/Active/Recovering -1760> 2014-11-03 11:37:55.953282 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x1adc4000 con 0x22fe0420 -1759> 2014-11-03 11:37:55.987723 7f291450b700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.987723, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1758> 2014-11-03 11:37:55.987779 7f291450b700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.987779, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1757> 2014-11-03 11:37:55.987805 7f291450b700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.987805, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1756> 2014-11-03 11:37:55.987775 7f2911d06700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.987774, event: commit_sent, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1755> 2014-11-03 11:37:55.987840 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94cdc0 con 0x22fe0c60 -1754> 2014-11-03 11:37:55.987885 7f2914d0c700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.987884, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1753> 2014-11-03 11:37:55.987901 7f2911d06700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.987901, event: commit_sent, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1752> 2014-11-03 11:37:55.987929 7f2914d0c700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.987929, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1751> 2014-11-03 11:37:55.987955 7f2914d0c700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.987955, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1750> 2014-11-03 11:37:55.987980 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94d340 con 0x22fe0c60 -1749> 2014-11-03 11:37:55.988010 7f2911d06700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.988010, event: commit_sent, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1748> 2014-11-03 11:37:55.988051 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94c840 con 0x22fe0c60 -1747> 2014-11-03 11:37:55.988579 7f291450b700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.988578, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1746> 2014-11-03 11:37:55.988609 7f291450b700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.988609, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1745> 2014-11-03 11:37:55.988628 7f291450b700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.988628, event: journaled_completion_queued, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1744> 2014-11-03 11:37:55.988673 7f2911d06700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.988673, event: commit_sent, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1743> 2014-11-03 11:37:55.988691 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94c2c0 con 0x22fe0c60 -1742> 2014-11-03 11:37:55.988711 7f2911d06700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.988711, event: commit_sent, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1741> 2014-11-03 11:37:55.988728 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94c580 con 0x22fe0c60 -1740> 2014-11-03 11:37:55.988744 7f2911d06700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.988744, event: commit_sent, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1739> 2014-11-03 11:37:55.988804 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94cb00 con 0x22fe0c60 -1738> 2014-11-03 11:37:55.990477 7f2912507700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.990477, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1737> 2014-11-03 11:37:55.990544 7f2912507700 5 -- op tracker -- seq: 3221, time: 2014-11-03 11:37:55.990544, event: done, op: osd_sub_op(client.7867900.0:14923973 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219461 snapset=0=[]:[] snapc=0=[]) -1736> 2014-11-03 11:37:55.990648 7f2912507700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.990648, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1735> 2014-11-03 11:37:55.990738 7f2912507700 5 -- op tracker -- seq: 3222, time: 2014-11-03 11:37:55.990738, event: done, op: osd_sub_op(client.7867900.0:14923974 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219462 snapset=0=[]:[] snapc=0=[]) -1734> 2014-11-03 11:37:55.991168 7f2912507700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.991168, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1733> 2014-11-03 11:37:55.991213 7f2912507700 5 -- op tracker -- seq: 3223, time: 2014-11-03 11:37:55.991213, event: done, op: osd_sub_op(client.7867900.0:14924005 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219463 snapset=0=[]:[] snapc=0=[]) -1732> 2014-11-03 11:37:55.991351 7f2912507700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.991351, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1731> 2014-11-03 11:37:55.991387 7f2912507700 5 -- op tracker -- seq: 3224, time: 2014-11-03 11:37:55.991387, event: done, op: osd_sub_op(client.7867900.0:14924006 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219464 snapset=0=[]:[] snapc=0=[]) -1730> 2014-11-03 11:37:55.991540 7f2912507700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.991540, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1729> 2014-11-03 11:37:55.991572 7f2912507700 5 -- op tracker -- seq: 3225, time: 2014-11-03 11:37:55.991572, event: done, op: osd_sub_op(client.7867900.0:14924007 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219465 snapset=0=[]:[] snapc=0=[]) -1728> 2014-11-03 11:37:55.991795 7f2912507700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.991795, event: sub_op_applied, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1727> 2014-11-03 11:37:55.991838 7f2912507700 5 -- op tracker -- seq: 3226, time: 2014-11-03 11:37:55.991838, event: done, op: osd_sub_op(client.7867900.0:14924008 6.5a8 465835a8/rbd_data.4b843b2ae8944a.0000000000000162/head//6 [] v 105950'16219466 snapset=0=[]:[] snapc=0=[]) -1726> 2014-11-03 11:37:56.026390 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 41 ==== MOSDPGPush(6.9d8 105950 [PushOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, version: 105945'7949808, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (1135943192 0 0) 0x1adc4000 con 0x22fe0420 -1725> 2014-11-03 11:37:56.026449 7f28f5be7700 5 -- op tracker -- seq: 3229, time: 2014-11-03 11:37:56.012383, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, version: 105945'7949808, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1724> 2014-11-03 11:37:56.026556 7f28f5be7700 5 -- op tracker -- seq: 3229, time: 2014-11-03 11:37:56.012385, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, version: 105945'7949808, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1723> 2014-11-03 11:37:56.026577 7f28f5be7700 5 -- op tracker -- seq: 3229, time: 2014-11-03 11:37:56.026375, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, version: 105945'7949808, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1722> 2014-11-03 11:37:56.026594 7f28f5be7700 5 -- op tracker -- seq: 3229, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, version: 105945'7949808, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1721> 2014-11-03 11:37:56.026664 7f28ff81a700 5 -- op tracker -- seq: 3229, time: 2014-11-03 11:37:56.026664, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, version: 105945'7949808, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1720> 2014-11-03 11:37:56.026877 7f28ff81a700 5 -- op tracker -- seq: 3229, time: 2014-11-03 11:37:56.026877, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6, version: 105945'7949808, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7c59c9d8/rbd_data.89cd895a911542.00000000000096b9/head//6@105945'7949808, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1719> 2014-11-03 11:37:56.038969 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x258be080 sd=563 :6802 s=0 pgs=0 cs=0 l=0 c=0x1c9a8160).accept sd=563 192.168.5.2:42990/0 -1718> 2014-11-03 11:37:56.039619 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419988 192.168.5.2:0/1004585 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0xaf2e000 con 0x1c9a8160 -1717> 2014-11-03 11:37:56.040074 7f28fb812700 0 osd.21 105950 do_command r=0 -1716> 2014-11-03 11:37:56.040095 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004585 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xccdaa80 con 0x1c9a8160 -1715> 2014-11-03 11:37:56.050168 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12419988 192.168.5.2:0/1004585 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2310a800 con 0x1c9a8160 -1714> 2014-11-03 11:37:56.050934 7f28fb812700 0 osd.21 105950 do_command r=0 -1713> 2014-11-03 11:37:56.050947 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004585 -- command_reply(tid 2: 0 ) v1 -- ?+14292 0x23e5fe00 con 0x1c9a8160 -1712> 2014-11-03 11:37:56.057025 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004585 pipe(0x258be080 sd=563 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9a8160).reader couldn't read tag, (0) Success -1711> 2014-11-03 11:37:56.057070 7f290e2d4700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004585 pipe(0x258be080 sd=563 :6802 s=2 pgs=2 cs=1 l=1 c=0x1c9a8160).fault (0) Success -1710> 2014-11-03 11:37:56.057175 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x1c9a8160 session 0x7c35c80 -1709> 2014-11-03 11:37:56.078503 7f290e2d4700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x258bd000 sd=563 :6802 s=0 pgs=0 cs=0 l=0 c=0x26906c60).accept sd=563 192.168.9.2:56807/0 -1708> 2014-11-03 11:37:56.079534 7f290e2d4700 1 -- 192.168.5.2:6802/7606 <== client.8348783 192.168.9.2:0/1007288 1 ==== osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) v4 ==== 191+0+0 (75438803 0 0) 0x2d281280 con 0x26906c60 -1707> 2014-11-03 11:37:56.079581 7f290e2d4700 5 -- op tracker -- seq: 3230, time: 2014-11-03 11:37:56.079454, event: header_read, op: osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) -1706> 2014-11-03 11:37:56.079605 7f290e2d4700 5 -- op tracker -- seq: 3230, time: 2014-11-03 11:37:56.079459, event: throttled, op: osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) -1705> 2014-11-03 11:37:56.079624 7f290e2d4700 5 -- op tracker -- seq: 3230, time: 2014-11-03 11:37:56.079518, event: all_read, op: osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) -1704> 2014-11-03 11:37:56.079642 7f290e2d4700 5 -- op tracker -- seq: 3230, time: 0.000000, event: dispatched, op: osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) -1703> 2014-11-03 11:37:56.079758 7f28fd816700 5 -- op tracker -- seq: 3230, time: 2014-11-03 11:37:56.079758, event: reached_pg, op: osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) -1702> 2014-11-03 11:37:56.092214 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x229e7e00 con 0x22fe0420 -1701> 2014-11-03 11:37:56.101710 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.63 192.168.5.5:0/21346 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.100192) v2 ==== 47+0+0 (1205600528 0 0) 0x619f800 con 0x233cb000 -1700> 2014-11-03 11:37:56.101756 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/21346 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.100192) v2 -- ?+0 0x2353be00 con 0x233cb000 -1699> 2014-11-03 11:37:56.101857 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.63 192.168.5.5:0/21346 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.100192) v2 ==== 47+0+0 (1205600528 0 0) 0x218c4a00 con 0x233cb420 -1698> 2014-11-03 11:37:56.101901 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/21346 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.100192) v2 -- ?+0 0x229e7800 con 0x233cb420 -1697> 2014-11-03 11:37:56.111602 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 42 ==== MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, version: 105945'7949815, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195278+0+0 (1720766381 0 0) 0x229e7e00 con 0x22fe0420 -1696> 2014-11-03 11:37:56.111654 7f28f5be7700 5 -- op tracker -- seq: 3231, time: 2014-11-03 11:37:56.103041, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, version: 105945'7949815, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1695> 2014-11-03 11:37:56.111677 7f28f5be7700 5 -- op tracker -- seq: 3231, time: 2014-11-03 11:37:56.103043, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, version: 105945'7949815, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1694> 2014-11-03 11:37:56.111697 7f28f5be7700 5 -- op tracker -- seq: 3231, time: 2014-11-03 11:37:56.111585, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, version: 105945'7949815, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1693> 2014-11-03 11:37:56.111713 7f28f5be7700 5 -- op tracker -- seq: 3231, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, version: 105945'7949815, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1692> 2014-11-03 11:37:56.111776 7f290201f700 5 -- op tracker -- seq: 3231, time: 2014-11-03 11:37:56.111776, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, version: 105945'7949815, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1691> 2014-11-03 11:37:56.111979 7f290201f700 5 -- op tracker -- seq: 3231, time: 2014-11-03 11:37:56.111979, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6, version: 105945'7949815, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/head//6@105945'7949815, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1690> 2014-11-03 11:37:56.113948 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.83 192.168.5.6:0/4005 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.112553) v2 ==== 47+0+0 (3398120890 0 0) 0x30db200 con 0x2445c2c0 -1689> 2014-11-03 11:37:56.113991 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.112553) v2 -- ?+0 0x619f800 con 0x2445c2c0 -1688> 2014-11-03 11:37:56.114140 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.83 192.168.5.6:0/4005 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.112553) v2 ==== 47+0+0 (3398120890 0 0) 0x26bdb600 con 0x23ea4420 -1687> 2014-11-03 11:37:56.114201 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.112553) v2 -- ?+0 0x218c4a00 con 0x23ea4420 -1686> 2014-11-03 11:37:56.124935 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.29 192.168.5.3:0/1192 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.123955) v2 ==== 47+0+0 (1361405308 0 0) 0x2350a800 con 0x23335000 -1685> 2014-11-03 11:37:56.124991 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/1192 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.123955) v2 -- ?+0 0x30db200 con 0x23335000 -1684> 2014-11-03 11:37:56.125085 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.29 192.168.5.3:0/1192 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.123955) v2 ==== 47+0+0 (1361405308 0 0) 0x26a9f200 con 0x234ed000 -1683> 2014-11-03 11:37:56.125126 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/1192 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.123955) v2 -- ?+0 0x26bdb600 con 0x234ed000 -1682> 2014-11-03 11:37:56.129227 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 45 ==== osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4925 (3280741328 0 3040076118) 0x285d8400 con 0x231ab580 -1681> 2014-11-03 11:37:56.129260 7f28ea430700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.129064, event: header_read, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1680> 2014-11-03 11:37:56.129278 7f28ea430700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.129066, event: throttled, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1679> 2014-11-03 11:37:56.129287 7f28ea430700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.129210, event: all_read, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1678> 2014-11-03 11:37:56.129295 7f28ea430700 5 -- op tracker -- seq: 3232, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1677> 2014-11-03 11:37:56.129352 7f290081c700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.129351, event: reached_pg, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1676> 2014-11-03 11:37:56.129380 7f290081c700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.129380, event: started, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1675> 2014-11-03 11:37:56.129374 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 196 ==== osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (641197059 0 3511520646) 0x2c336000 con 0x22fefdc0 -1674> 2014-11-03 11:37:56.129418 7f28f36c2700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.129205, event: header_read, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1673> 2014-11-03 11:37:56.129447 7f28f36c2700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.129211, event: throttled, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1672> 2014-11-03 11:37:56.129460 7f28f36c2700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.129360, event: all_read, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1671> 2014-11-03 11:37:56.129473 7f28f36c2700 5 -- op tracker -- seq: 3233, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1670> 2014-11-03 11:37:56.129471 7f290081c700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.129470, event: started, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1669> 2014-11-03 11:37:56.129500 7f290081c700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.129500, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1668> 2014-11-03 11:37:56.129533 7f28fe818700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.129533, event: reached_pg, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1667> 2014-11-03 11:37:56.129562 7f28fe818700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.129562, event: started, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1666> 2014-11-03 11:37:56.129650 7f28fe818700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.129650, event: started, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1665> 2014-11-03 11:37:56.129678 7f28fe818700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.129678, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1664> 2014-11-03 11:37:56.142734 7f28f018d700 1 -- 192.168.5.2:6803/7606 <== osd.34 192.168.5.3:6833/2034 49 ==== osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3288967624 0 3511520646) 0x2c690c00 con 0x230a9c60 -1663> 2014-11-03 11:37:56.142772 7f28f018d700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.142512, event: header_read, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1662> 2014-11-03 11:37:56.142790 7f28f018d700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.142515, event: throttled, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1661> 2014-11-03 11:37:56.142804 7f28f018d700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.142716, event: all_read, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1660> 2014-11-03 11:37:56.142816 7f28f018d700 5 -- op tracker -- seq: 3234, time: 0.000000, event: dispatched, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1659> 2014-11-03 11:37:56.142875 7f28ff81a700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.142874, event: reached_pg, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1658> 2014-11-03 11:37:56.142905 7f28ff81a700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.142905, event: started, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1657> 2014-11-03 11:37:56.142993 7f28ff81a700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.142993, event: started, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1656> 2014-11-03 11:37:56.143021 7f28ff81a700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.143021, event: commit_queued_for_journal_write, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1655> 2014-11-03 11:37:56.175088 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.38 192.168.5.3:0/18942 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.173510) v2 ==== 47+0+0 (192.1680151 0 0) 0x23354e00 con 0x240ad160 -1654> 2014-11-03 11:37:56.175157 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/18942 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.173510) v2 -- ?+0 0x2350a800 con 0x240ad160 -1653> 2014-11-03 11:37:56.175238 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.38 192.168.5.3:0/18942 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.173510) v2 ==== 47+0+0 (192.1680151 0 0) 0x6352600 con 0x2469e6e0 -1652> 2014-11-03 11:37:56.175283 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/18942 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.173510) v2 -- ?+0 0x26a9f200 con 0x2469e6e0 -1651> 2014-11-03 11:37:56.178497 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.107 192.168.5.8:0/17474 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.177044) v2 ==== 47+0+0 (231064362 0 0) 0x2d329e00 con 0x238b86e0 -1650> 2014-11-03 11:37:56.178505 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.107 192.168.5.8:0/17474 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.177044) v2 ==== 47+0+0 (231064362 0 0) 0x17ae8a00 con 0x239f8580 -1649> 2014-11-03 11:37:56.178563 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.177044) v2 -- ?+0 0x6352600 con 0x238b86e0 -1648> 2014-11-03 11:37:56.178650 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/17474 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.177044) v2 -- ?+0 0x23354e00 con 0x239f8580 -1647> 2014-11-03 11:37:56.197952 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.1 192.168.5.1:0/11054 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.197269) v2 ==== 47+0+0 (1175394991 0 0) 0x2512a800 con 0x2327a2c0 -1646> 2014-11-03 11:37:56.197989 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/11054 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.197269) v2 -- ?+0 0x17ae8a00 con 0x2327a2c0 -1645> 2014-11-03 11:37:56.198023 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.1 192.168.5.1:0/11054 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.197269) v2 ==== 47+0+0 (1175394991 0 0) 0x2796e800 con 0x2327a580 -1644> 2014-11-03 11:37:56.198070 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/11054 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.197269) v2 -- ?+0 0x2d329e00 con 0x2327a580 -1643> 2014-11-03 11:37:56.219594 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.53 192.168.5.4:0/5692 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.218529) v2 ==== 47+0+0 (1439446320 0 0) 0x2a77d000 con 0x23b1c9a0 -1642> 2014-11-03 11:37:56.219643 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.53 192.168.5.4:0/5692 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.218529) v2 ==== 47+0+0 (1439446320 0 0) 0x6352a00 con 0x242c0b00 -1641> 2014-11-03 11:37:56.219702 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/5692 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.218529) v2 -- ?+0 0x2512a800 con 0x23b1c9a0 -1640> 2014-11-03 11:37:56.219768 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/5692 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.218529) v2 -- ?+0 0x2796e800 con 0x242c0b00 -1639> 2014-11-03 11:37:56.221658 7f2918d14700 5 osd.21 105950 tick -1638> 2014-11-03 11:37:56.228654 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.81 192.168.5.6:0/27170 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.227205) v2 ==== 47+0+0 (2170514149 0 0) 0x2cf94e00 con 0x24229840 -1637> 2014-11-03 11:37:56.228689 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.227205) v2 -- ?+0 0x6352a00 con 0x24229840 -1636> 2014-11-03 11:37:56.228762 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.81 192.168.5.6:0/27170 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.227205) v2 ==== 47+0+0 (2170514149 0 0) 0x852f800 con 0x24925b00 -1635> 2014-11-03 11:37:56.228785 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/27170 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.227205) v2 -- ?+0 0x2a77d000 con 0x24925b00 -1634> 2014-11-03 11:37:56.230431 7f2914d0c700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.230431, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1633> 2014-11-03 11:37:56.230484 7f2914d0c700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.230484, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1632> 2014-11-03 11:37:56.230524 7f2914d0c700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.230524, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1631> 2014-11-03 11:37:56.231487 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.99 192.168.5.8:0/26930 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.229971) v2 ==== 47+0+0 (3568572222 0 0) 0x2c390a00 con 0x24205580 -1630> 2014-11-03 11:37:56.231509 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.99 192.168.5.8:0/26930 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.229971) v2 ==== 47+0+0 (3568572222 0 0) 0x2a25b800 con 0x239f8160 -1629> 2014-11-03 11:37:56.231534 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.229971) v2 -- ?+0 0x2cf94e00 con 0x24205580 -1628> 2014-11-03 11:37:56.231555 7f291450b700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.231555, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1627> 2014-11-03 11:37:56.231583 7f2911d06700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.231582, event: commit_sent, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1626> 2014-11-03 11:37:56.231606 7f291450b700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.231606, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1625> 2014-11-03 11:37:56.231578 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.229971) v2 -- ?+0 0x852f800 con 0x239f8160 -1624> 2014-11-03 11:37:56.231631 7f291450b700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.231631, event: journaled_completion_queued, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1623> 2014-11-03 11:37:56.231629 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] ondisk, result = 0) v2 -- ?+0 0x1c94d8c0 con 0x231ab580 -1622> 2014-11-03 11:37:56.231681 7f2911d06700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.231681, event: commit_sent, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1621> 2014-11-03 11:37:56.231710 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x27803b00 con 0x22fefdc0 -1620> 2014-11-03 11:37:56.231771 7f2911d06700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.231770, event: commit_sent, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1619> 2014-11-03 11:37:56.231823 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6833/2034 -- osd_sub_op_reply(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3149f580 con 0x230a9c60 -1618> 2014-11-03 11:37:56.240404 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.104 192.168.5.8:0/2576 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.238841) v2 ==== 47+0+0 (1736902012 0 0) 0x2c390800 con 0x239f8000 -1617> 2014-11-03 11:37:56.240454 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.238841) v2 -- ?+0 0x2a25b800 con 0x239f8000 -1616> 2014-11-03 11:37:56.240458 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.104 192.168.5.8:0/2576 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.238841) v2 ==== 47+0+0 (1736902012 0 0) 0xf637600 con 0x239f8dc0 -1615> 2014-11-03 11:37:56.240512 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/2576 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.238841) v2 -- ?+0 0x2c390a00 con 0x239f8dc0 -1614> 2014-11-03 11:37:56.242743 7f28fd816700 5 -- op tracker -- seq: 3230, time: 2014-11-03 11:37:56.242743, event: started, op: osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) -1613> 2014-11-03 11:37:56.261572 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.45 192.168.5.4:0/1781 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.260250) v2 ==== 47+0+0 (1812042379 0 0) 0x23414000 con 0x2445c160 -1612> 2014-11-03 11:37:56.261610 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.45 192.168.5.4:0/1781 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.260250) v2 ==== 47+0+0 (1812042379 0 0) 0x2d6ec200 con 0x23ea42c0 -1611> 2014-11-03 11:37:56.261619 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.260250) v2 -- ?+0 0x2c390800 con 0x2445c160 -1610> 2014-11-03 11:37:56.261693 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/1781 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.260250) v2 -- ?+0 0xf637600 con 0x23ea42c0 -1609> 2014-11-03 11:37:56.265938 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 197 ==== osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3319211557 0 3511520646) 0x25eb7c00 con 0x22fefdc0 -1608> 2014-11-03 11:37:56.266002 7f28f36c2700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.265770, event: header_read, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1607> 2014-11-03 11:37:56.266048 7f28f36c2700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.265773, event: throttled, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1606> 2014-11-03 11:37:56.266070 7f28f36c2700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.265915, event: all_read, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1605> 2014-11-03 11:37:56.266185 7f28f36c2700 5 -- op tracker -- seq: 3235, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1604> 2014-11-03 11:37:56.266344 7f290101d700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.266343, event: reached_pg, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1603> 2014-11-03 11:37:56.266401 7f290101d700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.266401, event: started, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1602> 2014-11-03 11:37:56.266530 7f290101d700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.266530, event: started, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1601> 2014-11-03 11:37:56.266585 7f290101d700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.266584, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1600> 2014-11-03 11:37:56.266730 7f2914d0c700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.266730, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1599> 2014-11-03 11:37:56.266940 7f291450b700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.266940, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1598> 2014-11-03 11:37:56.266975 7f2911d06700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.266975, event: commit_sent, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1597> 2014-11-03 11:37:56.267004 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x314a0340 con 0x22fefdc0 -1596> 2014-11-03 11:37:56.273735 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.79 192.168.5.6:0/12953 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.272496) v2 ==== 47+0+0 (2415342684 0 0) 0x244ab200 con 0x23b1c420 -1595> 2014-11-03 11:37:56.273787 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/12953 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.272496) v2 -- ?+0 0x23414000 con 0x23b1c420 -1594> 2014-11-03 11:37:56.273835 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.79 192.168.5.6:0/12953 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.272496) v2 ==== 47+0+0 (2415342684 0 0) 0x2796ce00 con 0x242052c0 -1593> 2014-11-03 11:37:56.273877 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/12953 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.272496) v2 -- ?+0 0x2d6ec200 con 0x242052c0 -1592> 2014-11-03 11:37:56.279207 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 198 ==== osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3348157862 0 3511520646) 0x23d99000 con 0x22fefdc0 -1591> 2014-11-03 11:37:56.279261 7f28f36c2700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279063, event: header_read, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1590> 2014-11-03 11:37:56.279284 7f28f36c2700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279065, event: throttled, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1589> 2014-11-03 11:37:56.279298 7f28f36c2700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279187, event: all_read, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1588> 2014-11-03 11:37:56.279311 7f28f36c2700 5 -- op tracker -- seq: 3236, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1587> 2014-11-03 11:37:56.279414 7f28fe818700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279414, event: reached_pg, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1586> 2014-11-03 11:37:56.279458 7f28fe818700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279457, event: started, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1585> 2014-11-03 11:37:56.279683 7f28fe818700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279682, event: started, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1584> 2014-11-03 11:37:56.279732 7f28fe818700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279731, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1583> 2014-11-03 11:37:56.279802 7f2914d0c700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.279802, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1582> 2014-11-03 11:37:56.280089 7f291450b700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.280089, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1581> 2014-11-03 11:37:56.280121 7f2911d06700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.280120, event: commit_sent, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1580> 2014-11-03 11:37:56.280151 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23613840 con 0x22fefdc0 -1579> 2014-11-03 11:37:56.287143 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.61 192.168.5.5:0/12481 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.285963) v2 ==== 47+0+0 (3362356851 0 0) 0x13d27000 con 0x2445c840 -1578> 2014-11-03 11:37:56.287239 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/12481 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.285963) v2 -- ?+0 0x244ab200 con 0x2445c840 -1577> 2014-11-03 11:37:56.287295 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.61 192.168.5.5:0/12481 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.285963) v2 ==== 47+0+0 (3362356851 0 0) 0x2796e600 con 0x23ea49a0 -1576> 2014-11-03 11:37:56.287345 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/12481 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.285963) v2 -- ?+0 0x2796ce00 con 0x23ea49a0 -1575> 2014-11-03 11:37:56.292177 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.97 192.168.5.7:0/21285 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.290111) v2 ==== 47+0+0 (3761426198 0 0) 0x2a7c4600 con 0x2484a2c0 -1574> 2014-11-03 11:37:56.292194 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.97 192.168.5.7:0/21285 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.290111) v2 ==== 47+0+0 (3761426198 0 0) 0x23a9e000 con 0x24537c60 -1573> 2014-11-03 11:37:56.292217 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/21285 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.290111) v2 -- ?+0 0x13d27000 con 0x2484a2c0 -1572> 2014-11-03 11:37:56.292317 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/21285 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.290111) v2 -- ?+0 0x2796e600 con 0x24537c60 -1571> 2014-11-03 11:37:56.293840 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 104 ==== osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (2281612592 0 3511520646) 0x26dbac00 con 0x22fbcb00 -1570> 2014-11-03 11:37:56.293870 7f28f6cf8700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.293750, event: header_read, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1569> 2014-11-03 11:37:56.293885 7f28f6cf8700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.293752, event: throttled, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1568> 2014-11-03 11:37:56.293893 7f28f6cf8700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.293829, event: all_read, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1567> 2014-11-03 11:37:56.293902 7f28f6cf8700 5 -- op tracker -- seq: 3237, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1566> 2014-11-03 11:37:56.293975 7f28fe017700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.293975, event: reached_pg, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1565> 2014-11-03 11:37:56.294011 7f28fe017700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.294011, event: started, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1564> 2014-11-03 11:37:56.294096 7f28fe017700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.294096, event: started, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1563> 2014-11-03 11:37:56.294137 7f28fe017700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.294137, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1562> 2014-11-03 11:37:56.294221 7f2914d0c700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.294221, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1561> 2014-11-03 11:37:56.294547 7f291450b700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.294546, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1560> 2014-11-03 11:37:56.294581 7f2911d06700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.294581, event: commit_sent, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1559> 2014-11-03 11:37:56.294607 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23614340 con 0x22fbcb00 -1558> 2014-11-03 11:37:56.295951 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 105 ==== osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4925 (2142618281 0 3535862671) 0xd4a5c00 con 0x22fbcb00 -1557> 2014-11-03 11:37:56.295982 7f28f6cf8700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.295845, event: header_read, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1556> 2014-11-03 11:37:56.295996 7f28f6cf8700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.295846, event: throttled, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1555> 2014-11-03 11:37:56.296005 7f28f6cf8700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.295940, event: all_read, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1554> 2014-11-03 11:37:56.296012 7f28f6cf8700 5 -- op tracker -- seq: 3238, time: 0.000000, event: dispatched, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1553> 2014-11-03 11:37:56.296065 7f290201f700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.296065, event: reached_pg, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1552> 2014-11-03 11:37:56.296096 7f290201f700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.296096, event: started, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1551> 2014-11-03 11:37:56.296205 7f290201f700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.296205, event: started, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1550> 2014-11-03 11:37:56.296236 7f290201f700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.296235, event: commit_queued_for_journal_write, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1549> 2014-11-03 11:37:56.296328 7f2914d0c700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.296327, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1548> 2014-11-03 11:37:56.296555 7f291450b700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.296554, event: journaled_completion_queued, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1547> 2014-11-03 11:37:56.296581 7f2911d06700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.296581, event: commit_sent, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1546> 2014-11-03 11:37:56.296603 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23df9dc0 con 0x22fbcb00 -1545> 2014-11-03 11:37:56.296715 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 106 ==== osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (652104354 0 3511520646) 0x2d3a9400 con 0x22fbcb00 -1544> 2014-11-03 11:37:56.296741 7f28f6cf8700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.296635, event: header_read, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1543> 2014-11-03 11:37:56.296753 7f28f6cf8700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.296637, event: throttled, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1542> 2014-11-03 11:37:56.296761 7f28f6cf8700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.296707, event: all_read, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1541> 2014-11-03 11:37:56.296769 7f28f6cf8700 5 -- op tracker -- seq: 3239, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1540> 2014-11-03 11:37:56.296817 7f290081c700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.296817, event: reached_pg, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1539> 2014-11-03 11:37:56.296837 7f290081c700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.296836, event: started, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1538> 2014-11-03 11:37:56.296925 7f290081c700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.296925, event: started, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1537> 2014-11-03 11:37:56.296956 7f290081c700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.296955, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1536> 2014-11-03 11:37:56.297041 7f2914d0c700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.297041, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1535> 2014-11-03 11:37:56.297241 7f291450b700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.297241, event: journaled_completion_queued, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1534> 2014-11-03 11:37:56.297263 7f2911d06700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.297263, event: commit_sent, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1533> 2014-11-03 11:37:56.297286 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23df9580 con 0x22fbcb00 -1532> 2014-11-03 11:37:56.306309 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.2:0/1007288 -- osd_op_reply(3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] v0'0 uv751803 ondisk = 0) v6 -- ?+0 0x2aa6c340 con 0x26906c60 -1531> 2014-11-03 11:37:56.306364 7f28fd816700 5 -- op tracker -- seq: 3230, time: 2014-11-03 11:37:56.306364, event: done, op: osd_op(client.8348783.0:3932016 rbd_data.48f8972ae8944a.0000000000001146 [read 2768896~45056] 24.139a44dc ack+read e105950) -1530> 2014-11-03 11:37:56.306984 7f28f14a0700 1 -- 192.168.5.2:6803/7606 <== osd.50 192.168.5.4:6847/26902 54 ==== osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (339884501 0 3511520646) 0x2c337800 con 0x23000840 -1529> 2014-11-03 11:37:56.307035 7f28f14a0700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.306862, event: header_read, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1528> 2014-11-03 11:37:56.307056 7f28f14a0700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.306864, event: throttled, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1527> 2014-11-03 11:37:56.307081 7f28f14a0700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.306968, event: all_read, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1526> 2014-11-03 11:37:56.307098 7f28f14a0700 5 -- op tracker -- seq: 3240, time: 0.000000, event: dispatched, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1525> 2014-11-03 11:37:56.307204 7f28ff81a700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.307204, event: reached_pg, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1524> 2014-11-03 11:37:56.307248 7f28ff81a700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.307248, event: started, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1523> 2014-11-03 11:37:56.307391 7f28ff81a700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.307391, event: started, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1522> 2014-11-03 11:37:56.307428 7f28ff81a700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.307428, event: commit_queued_for_journal_write, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1521> 2014-11-03 11:37:56.307499 7f2914d0c700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.307499, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1520> 2014-11-03 11:37:56.307713 7f291450b700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.307713, event: journaled_completion_queued, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1519> 2014-11-03 11:37:56.307754 7f2911d06700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.307753, event: commit_sent, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1518> 2014-11-03 11:37:56.307796 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6847/26902 -- osd_sub_op_reply(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23dfa080 con 0x23000840 -1517> 2014-11-03 11:37:56.327504 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 76 ==== osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2903609453 0 3511520646) 0x2714a400 con 0x22fbc840 -1516> 2014-11-03 11:37:56.327549 7f28f68f4700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327402, event: header_read, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1515> 2014-11-03 11:37:56.327563 7f28f68f4700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327405, event: throttled, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1514> 2014-11-03 11:37:56.327572 7f28f68f4700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327493, event: all_read, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1513> 2014-11-03 11:37:56.327580 7f28f68f4700 5 -- op tracker -- seq: 3241, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1512> 2014-11-03 11:37:56.327662 7f290001b700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327661, event: reached_pg, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1511> 2014-11-03 11:37:56.327695 7f290001b700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327695, event: started, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1510> 2014-11-03 11:37:56.327818 7f290001b700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327818, event: started, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1509> 2014-11-03 11:37:56.327861 7f290001b700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327861, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1508> 2014-11-03 11:37:56.327930 7f2914d0c700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.327929, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1507> 2014-11-03 11:37:56.328178 7f291450b700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.328178, event: journaled_completion_queued, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1506> 2014-11-03 11:37:56.328221 7f2911d06700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.328221, event: commit_sent, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1505> 2014-11-03 11:37:56.328275 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23dfa340 con 0x22fbc840 -1504> 2014-11-03 11:37:56.328367 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.40 192.168.5.3:0/25675 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.327523) v2 ==== 47+0+0 (414911323 0 0) 0x1106c000 con 0x23b1c580 -1503> 2014-11-03 11:37:56.328420 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/25675 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.327523) v2 -- ?+0 0x2a7c4600 con 0x23b1c580 -1502> 2014-11-03 11:37:56.328462 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.40 192.168.5.3:0/25675 8 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.327523) v2 ==== 47+0+0 (414911323 0 0) 0x24139000 con 0x242c0420 -1501> 2014-11-03 11:37:56.328506 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/25675 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.327523) v2 -- ?+0 0x23a9e000 con 0x242c0420 -1500> 2014-11-03 11:37:56.348126 7f290d52b700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x27803000 sd=231 :6802 s=0 pgs=0 cs=0 l=0 c=0x2408bdc0).accept sd=231 192.168.5.2:42996/0 -1499> 2014-11-03 11:37:56.349223 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450940 192.168.5.2:0/1004738 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2a6e8400 con 0x2408bdc0 -1498> 2014-11-03 11:37:56.350013 7f28fb812700 0 osd.21 105950 do_command r=0 -1497> 2014-11-03 11:37:56.350062 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004738 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x1026d380 con 0x2408bdc0 -1496> 2014-11-03 11:37:56.365015 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450940 192.168.5.2:0/1004738 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x252d7000 con 0x2408bdc0 -1495> 2014-11-03 11:37:56.366344 7f28fb812700 0 osd.21 105950 do_command r=0 -1494> 2014-11-03 11:37:56.366370 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004738 -- command_reply(tid 2: 0 ) v1 -- ?+14292 0x6ef01c0 con 0x2408bdc0 -1493> 2014-11-03 11:37:56.373431 7f290d52b700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004738 pipe(0x27803000 sd=231 :6802 s=2 pgs=2 cs=1 l=1 c=0x2408bdc0).reader couldn't read tag, (0) Success -1492> 2014-11-03 11:37:56.373474 7f290d52b700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004738 pipe(0x27803000 sd=231 :6802 s=2 pgs=2 cs=1 l=1 c=0x2408bdc0).fault (0) Success -1491> 2014-11-03 11:37:56.373576 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2408bdc0 session 0x27889000 -1490> 2014-11-03 11:37:56.403525 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 55 ==== osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1211382360 0 3511520646) 0x26942000 con 0x23000b00 -1489> 2014-11-03 11:37:56.403587 7f28f21ad700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.403354, event: header_read, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1488> 2014-11-03 11:37:56.403615 7f28f21ad700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.403358, event: throttled, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1487> 2014-11-03 11:37:56.403655 7f28f21ad700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.403501, event: all_read, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1486> 2014-11-03 11:37:56.403646 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 72 ==== osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3622246706 0 3511520646) 0xd55dc00 con 0x2308b2c0 -1485> 2014-11-03 11:37:56.403672 7f28f21ad700 5 -- op tracker -- seq: 3242, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1484> 2014-11-03 11:37:56.403677 7f28eed79700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.403477, event: header_read, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1483> 2014-11-03 11:37:56.403704 7f28eed79700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.403481, event: throttled, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1482> 2014-11-03 11:37:56.403714 7f28eed79700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.403618, event: all_read, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1481> 2014-11-03 11:37:56.403722 7f28eed79700 5 -- op tracker -- seq: 3243, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1480> 2014-11-03 11:37:56.403782 7f290181e700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.403782, event: reached_pg, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1479> 2014-11-03 11:37:56.403817 7f290181e700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.403817, event: started, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1478> 2014-11-03 11:37:56.403823 7f290201f700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.403823, event: reached_pg, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1477> 2014-11-03 11:37:56.403871 7f290201f700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.403871, event: started, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1476> 2014-11-03 11:37:56.403971 7f290181e700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.403971, event: started, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1475> 2014-11-03 11:37:56.404023 7f290181e700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.404023, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1474> 2014-11-03 11:37:56.404070 7f290201f700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.404069, event: started, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1473> 2014-11-03 11:37:56.404130 7f290201f700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.404130, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1472> 2014-11-03 11:37:56.404134 7f2914d0c700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.404134, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1471> 2014-11-03 11:37:56.404184 7f2914d0c700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.404183, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1470> 2014-11-03 11:37:56.404442 7f291450b700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.404442, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1469> 2014-11-03 11:37:56.404485 7f291450b700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.404484, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1468> 2014-11-03 11:37:56.404585 7f2911d06700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.404585, event: commit_sent, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1467> 2014-11-03 11:37:56.404647 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- osd_sub_op_reply(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23df9840 con 0x23000b00 -1466> 2014-11-03 11:37:56.404700 7f2911d06700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.404700, event: commit_sent, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1465> 2014-11-03 11:37:56.404676 7f290e2d4700 1 -- 192.168.5.2:6802/7606 <== client.8348783 192.168.9.2:0/1007288 2 ==== osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) v4 ==== 191+0+0 (1926458302 0 0) 0x32be000 con 0x26906c60 -1464> 2014-11-03 11:37:56.404720 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23dfa8c0 con 0x2308b2c0 -1463> 2014-11-03 11:37:56.404725 7f290e2d4700 5 -- op tracker -- seq: 3244, time: 2014-11-03 11:37:56.404551, event: header_read, op: osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) -1462> 2014-11-03 11:37:56.404745 7f290e2d4700 5 -- op tracker -- seq: 3244, time: 2014-11-03 11:37:56.404556, event: throttled, op: osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) -1461> 2014-11-03 11:37:56.404771 7f290e2d4700 5 -- op tracker -- seq: 3244, time: 2014-11-03 11:37:56.404654, event: all_read, op: osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) -1460> 2014-11-03 11:37:56.404786 7f290e2d4700 5 -- op tracker -- seq: 3244, time: 0.000000, event: dispatched, op: osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) -1459> 2014-11-03 11:37:56.404990 7f28fd816700 5 -- op tracker -- seq: 3244, time: 2014-11-03 11:37:56.404990, event: reached_pg, op: osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) -1458> 2014-11-03 11:37:56.405153 7f28fd816700 5 -- op tracker -- seq: 3244, time: 2014-11-03 11:37:56.405153, event: started, op: osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) -1457> 2014-11-03 11:37:56.405804 7f28f21ad700 1 -- 192.168.5.2:6803/7606 <== osd.97 192.168.5.7:6861/21285 56 ==== osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (974575086 0 3511520646) 0x23faf800 con 0x23000b00 -1456> 2014-11-03 11:37:56.405854 7f28f21ad700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.405669, event: header_read, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1455> 2014-11-03 11:37:56.405894 7f28f21ad700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.405671, event: throttled, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1454> 2014-11-03 11:37:56.405919 7f28f21ad700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.405789, event: all_read, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1453> 2014-11-03 11:37:56.405948 7f28f21ad700 5 -- op tracker -- seq: 3245, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1452> 2014-11-03 11:37:56.406037 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 73 ==== osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (453675012 0 3511520646) 0x3125d400 con 0x2308b2c0 -1451> 2014-11-03 11:37:56.406079 7f28eed79700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.405956, event: header_read, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1450> 2014-11-03 11:37:56.406092 7f28eed79700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.405957, event: throttled, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1449> 2014-11-03 11:37:56.406043 7f28ff019700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.406043, event: reached_pg, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1448> 2014-11-03 11:37:56.406134 7f28eed79700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.406024, event: all_read, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1447> 2014-11-03 11:37:56.406149 7f28eed79700 5 -- op tracker -- seq: 3246, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1446> 2014-11-03 11:37:56.406151 7f28ff019700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.406151, event: started, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1445> 2014-11-03 11:37:56.406221 7f28ff81a700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.406221, event: reached_pg, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1444> 2014-11-03 11:37:56.406252 7f28ff019700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.406251, event: started, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1443> 2014-11-03 11:37:56.406257 7f28ff81a700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.406257, event: started, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1442> 2014-11-03 11:37:56.406273 7f28ff019700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.406273, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1441> 2014-11-03 11:37:56.406374 7f2914d0c700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.406373, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1440> 2014-11-03 11:37:56.406388 7f28ff81a700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.406388, event: started, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1439> 2014-11-03 11:37:56.406437 7f28ff81a700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.406436, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1438> 2014-11-03 11:37:56.406642 7f291450b700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.406641, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1437> 2014-11-03 11:37:56.406748 7f2914d0c700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.406748, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1436> 2014-11-03 11:37:56.406776 7f2911d06700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.406776, event: commit_sent, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1435> 2014-11-03 11:37:56.406805 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6861/21285 -- osd_sub_op_reply(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b41f8c0 con 0x23000b00 -1434> 2014-11-03 11:37:56.406964 7f291450b700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.406963, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1433> 2014-11-03 11:37:56.407060 7f2911d06700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.407059, event: commit_sent, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1432> 2014-11-03 11:37:56.407102 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b41e840 con 0x2308b2c0 -1431> 2014-11-03 11:37:56.408177 7f28eed79700 1 -- 192.168.5.2:6803/7606 <== osd.13 192.168.5.1:6839/22281 74 ==== osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1089632652 0 3511520646) 0x3348c00 con 0x2308b2c0 -1430> 2014-11-03 11:37:56.408204 7f28eed79700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408062, event: header_read, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1429> 2014-11-03 11:37:56.408215 7f28eed79700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408063, event: throttled, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1428> 2014-11-03 11:37:56.408223 7f28eed79700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408168, event: all_read, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1427> 2014-11-03 11:37:56.408232 7f28eed79700 5 -- op tracker -- seq: 3247, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1426> 2014-11-03 11:37:56.408279 7f290201f700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408279, event: reached_pg, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1425> 2014-11-03 11:37:56.408309 7f290201f700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408309, event: started, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1424> 2014-11-03 11:37:56.408416 7f290201f700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408416, event: started, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1423> 2014-11-03 11:37:56.408455 7f290201f700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408455, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1422> 2014-11-03 11:37:56.408588 7f2914d0c700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408588, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1421> 2014-11-03 11:37:56.408783 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.90 192.168.5.7:0/27918 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.406229) v2 ==== 47+0+0 (361226434 0 0) 0x8714c00 con 0x23ea4840 -1420> 2014-11-03 11:37:56.408825 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.90 192.168.5.7:0/27918 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.406229) v2 ==== 47+0+0 (361226434 0 0) 0x26bdb800 con 0x2445c6e0 -1419> 2014-11-03 11:37:56.408805 7f291450b700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408805, event: journaled_completion_queued, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1418> 2014-11-03 11:37:56.408866 7f2911d06700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.408866, event: commit_sent, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1417> 2014-11-03 11:37:56.408863 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/27918 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.406229) v2 -- ?+0 0x1106c000 con 0x2445c6e0 -1416> 2014-11-03 11:37:56.408892 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6839/22281 -- osd_sub_op_reply(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b2e0b00 con 0x2308b2c0 -1415> 2014-11-03 11:37:56.409029 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/27918 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.406229) v2 -- ?+0 0x24139000 con 0x23ea4840 -1414> 2014-11-03 11:37:56.416257 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x24dcf400 con 0x22fe0420 -1413> 2014-11-03 11:37:56.416350 7f2912507700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.416350, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1412> 2014-11-03 11:37:56.416406 7f2912507700 5 -- op tracker -- seq: 3232, time: 2014-11-03 11:37:56.416406, event: done, op: osd_sub_op(client.8703713.0:32254501 24.a67 e7f33a67/rbd_data.7653db76142022.0000000000001001/head//24 [] v 105950'97979 snapset=0=[]:[] snapc=0=[]) -1411> 2014-11-03 11:37:56.416607 7f2912507700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.416607, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1410> 2014-11-03 11:37:56.416637 7f2912507700 5 -- op tracker -- seq: 3233, time: 2014-11-03 11:37:56.416637, event: done, op: osd_sub_op(client.8703713.0:32254499 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953158 snapset=0=[]:[] snapc=0=[]) -1409> 2014-11-03 11:37:56.416771 7f2912507700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.416770, event: sub_op_applied, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1408> 2014-11-03 11:37:56.416807 7f2912507700 5 -- op tracker -- seq: 3234, time: 2014-11-03 11:37:56.416807, event: done, op: osd_sub_op(client.8036651.0:1068111 6.6fe 53f6f6fe/rbd_data.7515b62ae8944a.0000000000001781/head//6 [] v 105950'9124378 snapset=0=[]:[] snapc=0=[]) -1407> 2014-11-03 11:37:56.416926 7f2912507700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.416926, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1406> 2014-11-03 11:37:56.416970 7f2912507700 5 -- op tracker -- seq: 3235, time: 2014-11-03 11:37:56.416970, event: done, op: osd_sub_op(client.8703713.0:32254510 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953159 snapset=0=[]:[] snapc=0=[]) -1405> 2014-11-03 11:37:56.417100 7f2912507700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.417100, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1404> 2014-11-03 11:37:56.417139 7f2912507700 5 -- op tracker -- seq: 3236, time: 2014-11-03 11:37:56.417139, event: done, op: osd_sub_op(client.8703713.0:32254515 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953160 snapset=0=[]:[] snapc=0=[]) -1403> 2014-11-03 11:37:56.417440 7f2912507700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.417439, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1402> 2014-11-03 11:37:56.417496 7f2912507700 5 -- op tracker -- seq: 3237, time: 2014-11-03 11:37:56.417496, event: done, op: osd_sub_op(client.7869068.0:21687453 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134377 snapset=0=[]:[] snapc=0=[]) -1401> 2014-11-03 11:37:56.417816 7f2912507700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.417816, event: sub_op_applied, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1400> 2014-11-03 11:37:56.417855 7f2912507700 5 -- op tracker -- seq: 3238, time: 2014-11-03 11:37:56.417855, event: done, op: osd_sub_op(client.7871531.0:413992 24.311 19426b11/rbd_data.7097872ae8944a.00000000000000bf/head//24 [] v 105950'114069 snapset=0=[]:[] snapc=0=[]) -1399> 2014-11-03 11:37:56.418035 7f2912507700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.418035, event: sub_op_applied, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1398> 2014-11-03 11:37:56.418067 7f2912507700 5 -- op tracker -- seq: 3239, time: 2014-11-03 11:37:56.418067, event: done, op: osd_sub_op(client.7869068.0:21687454 6.323 91cb7b23/rbd_data.4fd2461209313.00000000000000b3/head//6 [] v 105950'14134378 snapset=0=[]:[] snapc=0=[]) -1397> 2014-11-03 11:37:56.418386 7f2912507700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.418386, event: sub_op_applied, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1396> 2014-11-03 11:37:56.418450 7f2912507700 5 -- op tracker -- seq: 3240, time: 2014-11-03 11:37:56.418450, event: done, op: osd_sub_op(client.8421911.0:15704566 6.a23 d6c7da23/rbd_data.805e322ae8944a.0000000000002f9e/head//6 [] v 105950'7931203 snapset=0=[]:[] snapc=0=[]) -1395> 2014-11-03 11:37:56.418596 7f2912507700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.418595, event: sub_op_applied, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1394> 2014-11-03 11:37:56.418637 7f2912507700 5 -- op tracker -- seq: 3241, time: 2014-11-03 11:37:56.418637, event: done, op: osd_sub_op(client.7868473.0:865863 6.57c 5874e57c/rbd_data.71d8f67710f9ba.0000000000000082/head//6 [] v 105950'15047359 snapset=0=[]:[] snapc=0=[]) -1393> 2014-11-03 11:37:56.418749 7f2912507700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.418749, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1392> 2014-11-03 11:37:56.418802 7f2912507700 5 -- op tracker -- seq: 3242, time: 2014-11-03 11:37:56.418802, event: done, op: osd_sub_op(client.7868828.0:25297360 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703425 snapset=0=[]:[] snapc=0=[]) -1391> 2014-11-03 11:37:56.418947 7f2912507700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.418947, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1390> 2014-11-03 11:37:56.418987 7f2912507700 5 -- op tracker -- seq: 3243, time: 2014-11-03 11:37:56.418986, event: done, op: osd_sub_op(client.7868828.0:25297362 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870436 snapset=0=[]:[] snapc=0=[]) -1389> 2014-11-03 11:37:56.419120 7f2912507700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.419120, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1388> 2014-11-03 11:37:56.419178 7f2912507700 5 -- op tracker -- seq: 3245, time: 2014-11-03 11:37:56.419178, event: done, op: osd_sub_op(client.7868828.0:25297361 6.a42 db030a42/rbd_data.4d48f961268364.00000000000000b5/head//6 [] v 105950'6703426 snapset=0=[]:[] snapc=0=[]) -1387> 2014-11-03 11:37:56.419326 7f2912507700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.419326, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1386> 2014-11-03 11:37:56.419380 7f2912507700 5 -- op tracker -- seq: 3246, time: 2014-11-03 11:37:56.419380, event: done, op: osd_sub_op(client.7868828.0:25297363 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870437 snapset=0=[]:[] snapc=0=[]) -1385> 2014-11-03 11:37:56.419492 7f2912507700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.419492, event: sub_op_applied, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1384> 2014-11-03 11:37:56.419531 7f2912507700 5 -- op tracker -- seq: 3247, time: 2014-11-03 11:37:56.419531, event: done, op: osd_sub_op(client.7868828.0:25297364 6.456 19fa5c56/rbd_data.4d48f961268364.00000000000000dd/head//6 [] v 105950'31870438 snapset=0=[]:[] snapc=0=[]) -1383> 2014-11-03 11:37:56.426716 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.2:0/1007288 -- osd_op_reply(3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] v0'0 uv751803 ondisk = 0) v6 -- ?+0 0x2d6ea000 con 0x26906c60 -1382> 2014-11-03 11:37:56.426767 7f28fd816700 5 -- op tracker -- seq: 3244, time: 2014-11-03 11:37:56.426767, event: done, op: osd_op(client.8348783.0:3932021 rbd_data.48f8972ae8944a.0000000000001146 [read 3444736~40960] 24.139a44dc ack+read e105950) -1381> 2014-11-03 11:37:56.452984 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 199 ==== osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (801602843 0 3511520646) 0x24da3400 con 0x22fefdc0 -1380> 2014-11-03 11:37:56.453050 7f28f36c2700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.452840, event: header_read, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1379> 2014-11-03 11:37:56.453076 7f28f36c2700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.452842, event: throttled, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1378> 2014-11-03 11:37:56.453095 7f28f36c2700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.452950, event: all_read, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1377> 2014-11-03 11:37:56.453137 7f28f36c2700 5 -- op tracker -- seq: 3248, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1376> 2014-11-03 11:37:56.453243 7f28ff81a700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.453243, event: reached_pg, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1375> 2014-11-03 11:37:56.453289 7f28ff81a700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.453289, event: started, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1374> 2014-11-03 11:37:56.453389 7f28ff81a700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.453389, event: started, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1373> 2014-11-03 11:37:56.453430 7f28ff81a700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.453430, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1372> 2014-11-03 11:37:56.453519 7f2914d0c700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.453519, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1371> 2014-11-03 11:37:56.479191 7f291450b700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.479191, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1370> 2014-11-03 11:37:56.479298 7f2911d06700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.479297, event: commit_sent, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1369> 2014-11-03 11:37:56.479338 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7d600 con 0x22fefdc0 -1368> 2014-11-03 11:37:56.479604 7f2912507700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.479604, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1367> 2014-11-03 11:37:56.479667 7f2912507700 5 -- op tracker -- seq: 3248, time: 2014-11-03 11:37:56.479667, event: done, op: osd_sub_op(client.7914641.0:33713992 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033689 snapset=0=[]:[] snapc=0=[]) -1366> 2014-11-03 11:37:56.480510 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 43 ==== MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, version: 105945'7949809, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195157+0+0 (3373645431 0 0) 0x24dcf400 con 0x22fe0420 -1365> 2014-11-03 11:37:56.480549 7f28f5be7700 5 -- op tracker -- seq: 3249, time: 2014-11-03 11:37:56.471488, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, version: 105945'7949809, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1364> 2014-11-03 11:37:56.480573 7f28f5be7700 5 -- op tracker -- seq: 3249, time: 2014-11-03 11:37:56.471491, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, version: 105945'7949809, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1363> 2014-11-03 11:37:56.480589 7f28f5be7700 5 -- op tracker -- seq: 3249, time: 2014-11-03 11:37:56.480491, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, version: 105945'7949809, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1362> 2014-11-03 11:37:56.480609 7f28f5be7700 5 -- op tracker -- seq: 3249, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, version: 105945'7949809, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1361> 2014-11-03 11:37:56.480671 7f290201f700 5 -- op tracker -- seq: 3249, time: 2014-11-03 11:37:56.480671, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, version: 105945'7949809, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1360> 2014-11-03 11:37:56.481194 7f290201f700 5 -- op tracker -- seq: 3249, time: 2014-11-03 11:37:56.481194, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6, version: 105945'7949809, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 1, recovery_info: ObjectRecoveryInfo(4d8609d8/rbd_data.972f5364f2ae11.000000000000124a/1a08//6@105945'7949809, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1359> 2014-11-03 11:37:56.488847 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- MOSDPGPushReply(24.74d 105950 [PushReplyOp(5c0eff4d/rbd_data.5dd32f2ae8944a.0000000000001884/head//24),PushReplyOp(3df6074d/rbd_data.44dfad2ae8944a.0000000000000aba/head//24)]) v2 -- ?+0 0x2743ac00 con 0x230a92c0 -1358> 2014-11-03 11:37:56.490455 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 96 ==== MRecoveryReserve RELEASE pgid: 24.74d, query_epoch: 105950 v2 ==== 26+0+0 (4001693973 0 0) 0x2743ac00 con 0x230a92c0 -1357> 2014-11-03 11:37:56.490517 7f290b031700 5 -- op tracker -- seq: 3250, time: 2014-11-03 11:37:56.490257, event: header_read, op: MRecoveryReserve RELEASE pgid: 24.74d, query_epoch: 105950 -1356> 2014-11-03 11:37:56.490538 7f290b031700 5 -- op tracker -- seq: 3250, time: 2014-11-03 11:37:56.490259, event: throttled, op: MRecoveryReserve RELEASE pgid: 24.74d, query_epoch: 105950 -1355> 2014-11-03 11:37:56.490549 7f290b031700 5 -- op tracker -- seq: 3250, time: 2014-11-03 11:37:56.490366, event: all_read, op: MRecoveryReserve RELEASE pgid: 24.74d, query_epoch: 105950 -1354> 2014-11-03 11:37:56.490559 7f290b031700 5 -- op tracker -- seq: 3250, time: 2014-11-03 11:37:56.490510, event: dispatched, op: MRecoveryReserve RELEASE pgid: 24.74d, query_epoch: 105950 -1353> 2014-11-03 11:37:56.490571 7f290b031700 5 -- op tracker -- seq: 3250, time: 2014-11-03 11:37:56.490571, event: waiting_for_osdmap, op: MRecoveryReserve RELEASE pgid: 24.74d, query_epoch: 105950 -1352> 2014-11-03 11:37:56.490613 7f290b031700 5 -- op tracker -- seq: 3250, time: 2014-11-03 11:37:56.490613, event: done, op: MRecoveryReserve RELEASE pgid: 24.74d, query_epoch: 105950 -1351> 2014-11-03 11:37:56.490649 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.74d( v 105946'157340 (103730'153529,105946'157340] local-les=105948 n=165 ec=25000 les/c 105948/105943 105947/105947/105344) [114,21,85] r=1 lpr=105947 pi=94556-105946/311 luod=0'0 crt=105946'157340 active] exit Started/ReplicaActive/RepRecovering 0.699132 1 0.000106 -1350> 2014-11-03 11:37:56.490752 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.74d( v 105946'157340 (103730'153529,105946'157340] local-les=105948 n=165 ec=25000 les/c 105948/105943 105947/105947/105344) [114,21,85] r=1 lpr=105947 pi=94556-105946/311 luod=0'0 crt=105946'157340 active] enter Started/ReplicaActive/RepNotRecovering -1349> 2014-11-03 11:37:56.490857 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 97 ==== pg_info(1 pgs e105950:24.74d) v4 ==== 1295+0+0 (1771555161 0 0) 0xc157e00 con 0x230a92c0 -1348> 2014-11-03 11:37:56.490905 7f290b031700 5 -- op tracker -- seq: 3251, time: 2014-11-03 11:37:56.490608, event: header_read, op: pg_info(1 pgs e105950:24.74d) -1347> 2014-11-03 11:37:56.490920 7f290b031700 5 -- op tracker -- seq: 3251, time: 2014-11-03 11:37:56.490610, event: throttled, op: pg_info(1 pgs e105950:24.74d) -1346> 2014-11-03 11:37:56.490939 7f290b031700 5 -- op tracker -- seq: 3251, time: 2014-11-03 11:37:56.490768, event: all_read, op: pg_info(1 pgs e105950:24.74d) -1345> 2014-11-03 11:37:56.490954 7f290b031700 5 -- op tracker -- seq: 3251, time: 2014-11-03 11:37:56.490901, event: dispatched, op: pg_info(1 pgs e105950:24.74d) -1344> 2014-11-03 11:37:56.490907 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6815/21567 -- MRecoveryReserve GRANT pgid: 26.159, query_epoch: 105950 v2 -- ?+0 0x6526800 con 0x2308bb00 -1343> 2014-11-03 11:37:56.490970 7f290b031700 5 -- op tracker -- seq: 3251, time: 2014-11-03 11:37:56.490970, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:24.74d) -1342> 2014-11-03 11:37:56.491047 7f290b031700 5 -- op tracker -- seq: 3251, time: 2014-11-03 11:37:56.491047, event: started, op: pg_info(1 pgs e105950:24.74d) -1341> 2014-11-03 11:37:56.491027 7f2902820700 5 osd.21 pg_epoch: 105950 pg[26.159( v 105946'6181 lc 105913'6180 (66376'3136,105946'6181] local-les=105948 n=18 ec=25085 les/c 105948/105944 105947/105947/105312) [108,21,63] r=1 lpr=105947 pi=99888-105946/320 luod=0'0 crt=105946'6181 lcod 0'0 active m=1] exit Started/ReplicaActive/RepWaitRecoveryReserved 29.794037 5 0.000177 -1340> 2014-11-03 11:37:56.491111 7f2902820700 5 osd.21 pg_epoch: 105950 pg[26.159( v 105946'6181 lc 105913'6180 (66376'3136,105946'6181] local-les=105948 n=18 ec=25085 les/c 105948/105944 105947/105947/105312) [108,21,63] r=1 lpr=105947 pi=99888-105946/320 luod=0'0 crt=105946'6181 lcod 0'0 active m=1] enter Started/ReplicaActive/RepRecovering -1339> 2014-11-03 11:37:56.491130 7f290b031700 5 -- op tracker -- seq: 3251, time: 2014-11-03 11:37:56.491130, event: done, op: pg_info(1 pgs e105950:24.74d) -1338> 2014-11-03 11:37:56.494748 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x29884800 con 0x22fe0420 -1337> 2014-11-03 11:37:56.511443 7f28f008c700 1 -- 192.168.5.2:6803/7606 <== osd.108 192.168.5.8:6815/21567 27 ==== MOSDPGPush(26.159 105950 [PushOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26, version: 105946'6181, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26@105946'6181, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (2495020196 0 0) 0x6526800 con 0x2308bb00 -1336> 2014-11-03 11:37:56.511521 7f28f008c700 5 -- op tracker -- seq: 3252, time: 2014-11-03 11:37:56.499377, event: header_read, op: MOSDPGPush(26.159 105950 [PushOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26, version: 105946'6181, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26@105946'6181, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1335> 2014-11-03 11:37:56.511556 7f28f008c700 5 -- op tracker -- seq: 3252, time: 2014-11-03 11:37:56.499383, event: throttled, op: MOSDPGPush(26.159 105950 [PushOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26, version: 105946'6181, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26@105946'6181, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1334> 2014-11-03 11:37:56.511581 7f28f008c700 5 -- op tracker -- seq: 3252, time: 2014-11-03 11:37:56.511420, event: all_read, op: MOSDPGPush(26.159 105950 [PushOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26, version: 105946'6181, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26@105946'6181, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1333> 2014-11-03 11:37:56.511627 7f28f008c700 5 -- op tracker -- seq: 3252, time: 0.000000, event: dispatched, op: MOSDPGPush(26.159 105950 [PushOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26, version: 105946'6181, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26@105946'6181, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1332> 2014-11-03 11:37:56.511783 7f28ff81a700 5 -- op tracker -- seq: 3252, time: 2014-11-03 11:37:56.511782, event: reached_pg, op: MOSDPGPush(26.159 105950 [PushOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26, version: 105946'6181, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26@105946'6181, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1331> 2014-11-03 11:37:56.512044 7f28ff81a700 5 -- op tracker -- seq: 3252, time: 2014-11-03 11:37:56.512044, event: done, op: MOSDPGPush(26.159 105950 [PushOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26, version: 105946'6181, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26@105946'6181, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1330> 2014-11-03 11:37:56.516566 7f290d52b700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x9fff000 sd=33 :6802 s=0 pgs=0 cs=0 l=0 c=0x23409160).accept sd=33 192.168.5.2:43001/0 -1329> 2014-11-03 11:37:56.517597 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450830 192.168.5.2:0/1004964 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x22fd4400 con 0x23409160 -1328> 2014-11-03 11:37:56.517940 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6815/21567 -- pg_trim(26.159 to 105946'6181 e105950) v1 -- ?+0 0x26a9f600 con 0x2308bb00 -1327> 2014-11-03 11:37:56.518384 7f28fb812700 0 osd.21 105950 do_command r=0 -1326> 2014-11-03 11:37:56.518424 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004964 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0xccda380 con 0x23409160 -1325> 2014-11-03 11:37:56.526558 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6815/21567 -- MOSDPGPushReply(26.159 105950 [PushReplyOp(c80ac159/rbd_data.4626792ae8944a.00000000000005eb/head//26)]) v2 -- ?+0 0x1adc4000 con 0x2308bb00 -1324> 2014-11-03 11:37:56.527977 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.108 192.168.5.8:6815/21567 28 ==== MRecoveryReserve RELEASE pgid: 26.159, query_epoch: 105950 v2 ==== 26+0+0 (57882753 0 0) 0x1adc4000 con 0x2308bb00 -1323> 2014-11-03 11:37:56.528012 7f290b031700 5 -- op tracker -- seq: 3253, time: 2014-11-03 11:37:56.527853, event: header_read, op: MRecoveryReserve RELEASE pgid: 26.159, query_epoch: 105950 -1322> 2014-11-03 11:37:56.528025 7f290b031700 5 -- op tracker -- seq: 3253, time: 2014-11-03 11:37:56.527856, event: throttled, op: MRecoveryReserve RELEASE pgid: 26.159, query_epoch: 105950 -1321> 2014-11-03 11:37:56.528031 7f290b031700 5 -- op tracker -- seq: 3253, time: 2014-11-03 11:37:56.527929, event: all_read, op: MRecoveryReserve RELEASE pgid: 26.159, query_epoch: 105950 -1320> 2014-11-03 11:37:56.528037 7f290b031700 5 -- op tracker -- seq: 3253, time: 2014-11-03 11:37:56.528006, event: dispatched, op: MRecoveryReserve RELEASE pgid: 26.159, query_epoch: 105950 -1319> 2014-11-03 11:37:56.528074 7f290b031700 5 -- op tracker -- seq: 3253, time: 2014-11-03 11:37:56.528074, event: waiting_for_osdmap, op: MRecoveryReserve RELEASE pgid: 26.159, query_epoch: 105950 -1318> 2014-11-03 11:37:56.528117 7f290b031700 5 -- op tracker -- seq: 3253, time: 2014-11-03 11:37:56.528117, event: done, op: MRecoveryReserve RELEASE pgid: 26.159, query_epoch: 105950 -1317> 2014-11-03 11:37:56.528208 7f2902820700 5 osd.21 pg_epoch: 105950 pg[26.159( v 105946'6181 (66376'3136,105946'6181] local-les=105948 n=18 ec=25085 les/c 105948/105944 105947/105947/105312) [108,21,63] r=1 lpr=105947 pi=99888-105946/320 luod=0'0 crt=105946'6181 active] exit Started/ReplicaActive/RepRecovering 0.037098 1 0.000264 -1316> 2014-11-03 11:37:56.528360 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.108 192.168.5.8:6815/21567 29 ==== pg_info(1 pgs e105950:26.159) v4 ==== 767+0+0 (1279367650 0 0) 0x26a9f600 con 0x2308bb00 -1315> 2014-11-03 11:37:56.528382 7f290b031700 5 -- op tracker -- seq: 3254, time: 2014-11-03 11:37:56.528146, event: header_read, op: pg_info(1 pgs e105950:26.159) -1314> 2014-11-03 11:37:56.528395 7f290b031700 5 -- op tracker -- seq: 3254, time: 2014-11-03 11:37:56.528149, event: throttled, op: pg_info(1 pgs e105950:26.159) -1313> 2014-11-03 11:37:56.528401 7f290b031700 5 -- op tracker -- seq: 3254, time: 2014-11-03 11:37:56.528303, event: all_read, op: pg_info(1 pgs e105950:26.159) -1312> 2014-11-03 11:37:56.528388 7f2902820700 5 osd.21 pg_epoch: 105950 pg[26.159( v 105946'6181 (66376'3136,105946'6181] local-les=105948 n=18 ec=25085 les/c 105948/105944 105947/105947/105312) [108,21,63] r=1 lpr=105947 pi=99888-105946/320 luod=0'0 crt=105946'6181 active] enter Started/ReplicaActive/RepNotRecovering -1311> 2014-11-03 11:37:56.528412 7f290b031700 5 -- op tracker -- seq: 3254, time: 2014-11-03 11:37:56.528379, event: dispatched, op: pg_info(1 pgs e105950:26.159) -1310> 2014-11-03 11:37:56.528423 7f290b031700 5 -- op tracker -- seq: 3254, time: 2014-11-03 11:37:56.528423, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:26.159) -1309> 2014-11-03 11:37:56.528432 7f290b031700 5 -- op tracker -- seq: 3254, time: 2014-11-03 11:37:56.528432, event: started, op: pg_info(1 pgs e105950:26.159) -1308> 2014-11-03 11:37:56.528462 7f290b031700 5 -- op tracker -- seq: 3254, time: 2014-11-03 11:37:56.528462, event: done, op: pg_info(1 pgs e105950:26.159) -1307> 2014-11-03 11:37:56.528509 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- MRecoveryReserve GRANT pgid: 24.1c1, query_epoch: 105950 v2 -- ?+0 0x3a32e00 con 0x22fe0000 -1306> 2014-11-03 11:37:56.528554 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.1c1( v 105946'139902 lc 105945'139875 (104755'134220,105946'139902] local-les=105948 n=69 ec=25000 les/c 105948/105943 105947/105947/105111) [70,125,21] r=2 lpr=105947 pi=58133-105946/393 luod=0'0 crt=105946'139902 lcod 0'0 active m=3] exit Started/ReplicaActive/RepWaitRecoveryReserved 29.831123 5 0.000068 -1305> 2014-11-03 11:37:56.528604 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.1c1( v 105946'139902 lc 105945'139875 (104755'134220,105946'139902] local-les=105948 n=69 ec=25000 les/c 105948/105943 105947/105947/105111) [70,125,21] r=2 lpr=105947 pi=58133-105946/393 luod=0'0 crt=105946'139902 lcod 0'0 active m=3] enter Started/ReplicaActive/RepRecovering -1304> 2014-11-03 11:37:56.529639 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 119 ==== osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (4078945629 0 3511520646) 0x29e1e800 con 0x22fe0c60 -1303> 2014-11-03 11:37:56.529703 7f28f4eda700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.529385, event: header_read, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1302> 2014-11-03 11:37:56.529736 7f28f4eda700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.529388, event: throttled, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1301> 2014-11-03 11:37:56.529757 7f28f4eda700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.529619, event: all_read, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1300> 2014-11-03 11:37:56.529777 7f28f4eda700 5 -- op tracker -- seq: 3255, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1299> 2014-11-03 11:37:56.529868 7f28fe017700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.529867, event: reached_pg, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1298> 2014-11-03 11:37:56.529910 7f28fe017700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.529910, event: started, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1297> 2014-11-03 11:37:56.530051 7f28fe017700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.530051, event: started, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1296> 2014-11-03 11:37:56.530097 7f28fe017700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.530096, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1295> 2014-11-03 11:37:56.531130 7f2914d0c700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.531130, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1294> 2014-11-03 11:37:56.531572 7f291450b700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.531571, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1293> 2014-11-03 11:37:56.531641 7f2911d06700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.531640, event: commit_sent, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1292> 2014-11-03 11:37:56.531696 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] ondisk, result = 0) v2 -- ?+0 0x28a7d8c0 con 0x22fe0c60 -1291> 2014-11-03 11:37:56.532469 7f28f4eda700 1 -- 192.168.5.2:6803/7606 <== osd.6 192.168.5.1:6880/1181 120 ==== osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (1227970779 0 3511520646) 0x2398c000 con 0x22fe0c60 -1290> 2014-11-03 11:37:56.532487 7f2912507700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.532486, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1289> 2014-11-03 11:37:56.532523 7f28f4eda700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.532275, event: header_read, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1288> 2014-11-03 11:37:56.532550 7f28f4eda700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.532278, event: throttled, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1287> 2014-11-03 11:37:56.532555 7f2912507700 5 -- op tracker -- seq: 3255, time: 2014-11-03 11:37:56.532554, event: done, op: osd_sub_op(client.7867942.0:137198270 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790469 snapset=0=[]:[] snapc=0=[]) -1286> 2014-11-03 11:37:56.532564 7f28f4eda700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.532454, event: all_read, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1285> 2014-11-03 11:37:56.532586 7f28f4eda700 5 -- op tracker -- seq: 3256, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1284> 2014-11-03 11:37:56.532672 7f290081c700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.532672, event: reached_pg, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1283> 2014-11-03 11:37:56.532726 7f290081c700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.532725, event: started, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1282> 2014-11-03 11:37:56.532869 7f290081c700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.532868, event: started, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1281> 2014-11-03 11:37:56.532918 7f290081c700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.532917, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1280> 2014-11-03 11:37:56.533019 7f2914d0c700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.533019, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1279> 2014-11-03 11:37:56.533689 7f291450b700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.533689, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1278> 2014-11-03 11:37:56.533902 7f2911d06700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.533901, event: commit_sent, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1277> 2014-11-03 11:37:56.533958 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6880/1181 -- osd_sub_op_reply(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] ondisk, result = 0) v2 -- ?+0 0x24320080 con 0x22fe0c60 -1276> 2014-11-03 11:37:56.534464 7f2912507700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.534464, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1275> 2014-11-03 11:37:56.534521 7f2912507700 5 -- op tracker -- seq: 3256, time: 2014-11-03 11:37:56.534521, event: done, op: osd_sub_op(client.7867942.0:137198271 6.4d1 e32b44d1/rbd_data.43c40916a10.000000000000055b/head//6 [] v 105950'12790470 snapset=0=[]:[] snapc=0=[]) -1274> 2014-11-03 11:37:56.534647 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450830 192.168.5.2:0/1004964 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x273fdc00 con 0x23409160 -1273> 2014-11-03 11:37:56.535928 7f28fb812700 0 osd.21 105950 do_command r=0 -1272> 2014-11-03 11:37:56.535954 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1004964 -- command_reply(tid 2: 0 ) v1 -- ?+16699 0x23d89a80 con 0x23409160 -1271> 2014-11-03 11:37:56.543127 7f290d52b700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004964 pipe(0x9fff000 sd=33 :6802 s=2 pgs=2 cs=1 l=1 c=0x23409160).reader couldn't read tag, (0) Success -1270> 2014-11-03 11:37:56.543204 7f290d52b700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1004964 pipe(0x9fff000 sd=33 :6802 s=2 pgs=2 cs=1 l=1 c=0x23409160).fault (0) Success -1269> 2014-11-03 11:37:56.543373 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x23409160 session 0x2d281280 -1268> 2014-11-03 11:37:56.546508 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 104 ==== osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2586890217 0 3511520646) 0x27071800 con 0x2308b840 -1267> 2014-11-03 11:37:56.546554 7f28ef985700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.546417, event: header_read, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1266> 2014-11-03 11:37:56.546571 7f28ef985700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.546419, event: throttled, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1265> 2014-11-03 11:37:56.546584 7f28ef985700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.546497, event: all_read, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1264> 2014-11-03 11:37:56.546593 7f28ef985700 5 -- op tracker -- seq: 3257, time: 0.000000, event: dispatched, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1263> 2014-11-03 11:37:56.546689 7f28fe017700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.546689, event: reached_pg, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1262> 2014-11-03 11:37:56.546723 7f28fe017700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.546723, event: started, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1261> 2014-11-03 11:37:56.546842 7f28fe017700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.546842, event: started, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1260> 2014-11-03 11:37:56.546877 7f28fe017700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.546877, event: commit_queued_for_journal_write, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1259> 2014-11-03 11:37:56.547042 7f2914d0c700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.547042, event: write_thread_in_journal_buffer, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1258> 2014-11-03 11:37:56.547332 7f291450b700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.547331, event: journaled_completion_queued, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1257> 2014-11-03 11:37:56.547451 7f2911d06700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.547451, event: commit_sent, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1256> 2014-11-03 11:37:56.547497 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2431fdc0 con 0x2308b840 -1255> 2014-11-03 11:37:56.547886 7f2912507700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.547885, event: sub_op_applied, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1254> 2014-11-03 11:37:56.547926 7f28f57e3700 1 -- 192.168.5.2:6803/7606 <== osd.40 192.168.5.3:6868/25675 42 ==== osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (2671511521 0 3511520646) 0x29054000 con 0x22fea580 -1253> 2014-11-03 11:37:56.547957 7f28f57e3700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.547781, event: header_read, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1252> 2014-11-03 11:37:56.547974 7f2912507700 5 -- op tracker -- seq: 3257, time: 2014-11-03 11:37:56.547974, event: done, op: osd_sub_op(client.11363859.0:247731 6.508 2071508/rbd_data.78779e5eb78280.0000000000000b80/head//6 [] v 105950'20000608 snapset=0=[]:[] snapc=0=[]) -1251> 2014-11-03 11:37:56.547985 7f28f57e3700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.547784, event: throttled, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1250> 2014-11-03 11:37:56.548008 7f28f57e3700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.547900, event: all_read, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1249> 2014-11-03 11:37:56.548022 7f28f57e3700 5 -- op tracker -- seq: 3258, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1248> 2014-11-03 11:37:56.548101 7f290201f700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.548101, event: reached_pg, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1247> 2014-11-03 11:37:56.548128 7f290201f700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.548128, event: started, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1246> 2014-11-03 11:37:56.548315 7f290201f700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.548315, event: started, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1245> 2014-11-03 11:37:56.548380 7f290201f700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.548380, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1244> 2014-11-03 11:37:56.548498 7f2914d0c700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.548498, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1243> 2014-11-03 11:37:56.548736 7f291450b700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.548736, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1242> 2014-11-03 11:37:56.548822 7f2911d06700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.548821, event: commit_sent, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1241> 2014-11-03 11:37:56.548892 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6868/25675 -- osd_sub_op_reply(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] ondisk, result = 0) v2 -- ?+0 0x24320340 con 0x22fea580 -1240> 2014-11-03 11:37:56.549343 7f2912507700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.549342, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1239> 2014-11-03 11:37:56.549416 7f2912507700 5 -- op tracker -- seq: 3258, time: 2014-11-03 11:37:56.549415, event: done, op: osd_sub_op(client.7867942.0:137198330 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12182999 snapset=0=[]:[] snapc=0=[]) -1238> 2014-11-03 11:37:56.552610 7f28eff8b700 1 -- 192.168.5.2:6803/7606 <== osd.124 192.168.5.9:6864/28288 22 ==== osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (3430569979 0 3511520646) 0x275f3800 con 0x2308b9a0 -1237> 2014-11-03 11:37:56.552673 7f28eff8b700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.552411, event: header_read, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1236> 2014-11-03 11:37:56.552712 7f28eff8b700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.552445, event: throttled, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1235> 2014-11-03 11:37:56.552731 7f28eff8b700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.552592, event: all_read, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1234> 2014-11-03 11:37:56.552749 7f28eff8b700 5 -- op tracker -- seq: 3259, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1233> 2014-11-03 11:37:56.552863 7f28f57e3700 1 -- 192.168.5.2:6803/7606 <== osd.40 192.168.5.3:6868/25675 43 ==== osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (1604056809 0 3511520646) 0x26944400 con 0x22fea580 -1232> 2014-11-03 11:37:56.552887 7f290101d700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.552886, event: reached_pg, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1231> 2014-11-03 11:37:56.552916 7f28f57e3700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.552731, event: header_read, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1230> 2014-11-03 11:37:56.552965 7f290101d700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.552965, event: started, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1229> 2014-11-03 11:37:56.552997 7f28f57e3700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.552734, event: throttled, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1228> 2014-11-03 11:37:56.553048 7f28f57e3700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.552841, event: all_read, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1227> 2014-11-03 11:37:56.553070 7f28f57e3700 5 -- op tracker -- seq: 3260, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1226> 2014-11-03 11:37:56.553148 7f28ff81a700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.553148, event: reached_pg, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1225> 2014-11-03 11:37:56.553177 7f290101d700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.553176, event: started, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1224> 2014-11-03 11:37:56.553194 7f28ff81a700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.553194, event: started, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1223> 2014-11-03 11:37:56.553230 7f290101d700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.553230, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1222> 2014-11-03 11:37:56.553354 7f2914d0c700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.553354, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1221> 2014-11-03 11:37:56.553411 7f28ff81a700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.553410, event: started, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1220> 2014-11-03 11:37:56.553456 7f28ff81a700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.553455, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1219> 2014-11-03 11:37:56.553585 7f291450b700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.553585, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1218> 2014-11-03 11:37:56.553665 7f2911d06700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.553664, event: commit_sent, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1217> 2014-11-03 11:37:56.553682 7f2914d0c700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.553682, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1216> 2014-11-03 11:37:56.553732 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6864/28288 -- osd_sub_op_reply(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2431f580 con 0x2308b9a0 -1215> 2014-11-03 11:37:56.553915 7f291450b700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.553915, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1214> 2014-11-03 11:37:56.553953 7f2911d06700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.553953, event: commit_sent, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1213> 2014-11-03 11:37:56.553994 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6868/25675 -- osd_sub_op_reply(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2431fb00 con 0x22fea580 -1212> 2014-11-03 11:37:56.554135 7f2912507700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.554135, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1211> 2014-11-03 11:37:56.554188 7f2912507700 5 -- op tracker -- seq: 3259, time: 2014-11-03 11:37:56.554188, event: done, op: osd_sub_op(client.7867942.0:137198358 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181708 snapset=0=[]:[] snapc=0=[]) -1210> 2014-11-03 11:37:56.554493 7f2912507700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.554492, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1209> 2014-11-03 11:37:56.554532 7f2912507700 5 -- op tracker -- seq: 3260, time: 2014-11-03 11:37:56.554532, event: done, op: osd_sub_op(client.7867942.0:137198331 6.5e1 b6fdb5e1/rbd_data.43c40916a10.0000000000000419/head//6 [] v 105950'12183000 snapset=0=[]:[] snapc=0=[]) -1208> 2014-11-03 11:37:56.554903 7f28eff8b700 1 -- 192.168.5.2:6803/7606 <== osd.124 192.168.5.9:6864/28288 23 ==== osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (1043117483 0 3511520646) 0x26613400 con 0x2308b9a0 -1207> 2014-11-03 11:37:56.554963 7f28eff8b700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.554731, event: header_read, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1206> 2014-11-03 11:37:56.554988 7f28eff8b700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.554734, event: throttled, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1205> 2014-11-03 11:37:56.555006 7f28eff8b700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.554889, event: all_read, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1204> 2014-11-03 11:37:56.555024 7f28eff8b700 5 -- op tracker -- seq: 3261, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1203> 2014-11-03 11:37:56.555003 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 53 ==== osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (1539805042 0 3511520646) 0xd4a6800 con 0x230a9b00 -1202> 2014-11-03 11:37:56.555058 7f28f038f700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.554820, event: header_read, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1201> 2014-11-03 11:37:56.555098 7f28f038f700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.554823, event: throttled, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1200> 2014-11-03 11:37:56.555100 7f28fe818700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.555100, event: reached_pg, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1199> 2014-11-03 11:37:56.555127 7f28f038f700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.554987, event: all_read, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1198> 2014-11-03 11:37:56.555142 7f28fe818700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.555142, event: started, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1197> 2014-11-03 11:37:56.555153 7f28f038f700 5 -- op tracker -- seq: 3262, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1196> 2014-11-03 11:37:56.555279 7f290101d700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.555279, event: reached_pg, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1195> 2014-11-03 11:37:56.555304 7f28fe818700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.555304, event: started, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1194> 2014-11-03 11:37:56.555348 7f290101d700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.555348, event: started, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1193> 2014-11-03 11:37:56.555384 7f28fe818700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.555384, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1192> 2014-11-03 11:37:56.555506 7f2914d0c700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.555505, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1191> 2014-11-03 11:37:56.555533 7f290101d700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.555533, event: started, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1190> 2014-11-03 11:37:56.555569 7f290101d700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.555569, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1189> 2014-11-03 11:37:56.555888 7f291450b700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.555888, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1188> 2014-11-03 11:37:56.555948 7f2911d06700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.555947, event: commit_sent, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1187> 2014-11-03 11:37:56.555997 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6864/28288 -- osd_sub_op_reply(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2431f840 con 0x2308b9a0 -1186> 2014-11-03 11:37:56.556018 7f2914d0c700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.556018, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1185> 2014-11-03 11:37:56.556279 7f2912507700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.556279, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1184> 2014-11-03 11:37:56.556339 7f2912507700 5 -- op tracker -- seq: 3261, time: 2014-11-03 11:37:56.556338, event: done, op: osd_sub_op(client.7867942.0:137198359 6.881 c13a4881/rbd_data.43c40916a10.0000000000000272/head//6 [] v 105950'8181709 snapset=0=[]:[] snapc=0=[]) -1183> 2014-11-03 11:37:56.556385 7f291450b700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.556384, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1182> 2014-11-03 11:37:56.556439 7f2911d06700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.556438, event: commit_sent, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1181> 2014-11-03 11:37:56.556492 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2431f000 con 0x230a9b00 -1180> 2014-11-03 11:37:56.556827 7f2912507700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.556827, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1179> 2014-11-03 11:37:56.556880 7f2912507700 5 -- op tracker -- seq: 3262, time: 2014-11-03 11:37:56.556880, event: done, op: osd_sub_op(client.7867942.0:137198364 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850156 snapset=0=[]:[] snapc=0=[]) -1178> 2014-11-03 11:37:56.558393 7f28f0a96700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 98 ==== osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (3248592671 0 3511520646) 0x26612800 con 0x230a92c0 -1177> 2014-11-03 11:37:56.558438 7f28f0a96700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.558244, event: header_read, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1176> 2014-11-03 11:37:56.558460 7f28f0a96700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.558246, event: throttled, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1175> 2014-11-03 11:37:56.558484 7f28f0a96700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.558374, event: all_read, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1174> 2014-11-03 11:37:56.558502 7f28f0a96700 5 -- op tracker -- seq: 3263, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1173> 2014-11-03 11:37:56.558590 7f28fe818700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.558589, event: reached_pg, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1172> 2014-11-03 11:37:56.558647 7f28fe818700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.558646, event: started, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1171> 2014-11-03 11:37:56.558874 7f28fe818700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.558873, event: started, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1170> 2014-11-03 11:37:56.558944 7f28fe818700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.558943, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1169> 2014-11-03 11:37:56.559132 7f2914d0c700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.559132, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1168> 2014-11-03 11:37:56.559211 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 77 ==== osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (1725169651 0 3511520646) 0x27149800 con 0x22fbc840 -1167> 2014-11-03 11:37:56.559265 7f28f68f4700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559075, event: header_read, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1166> 2014-11-03 11:37:56.559293 7f28f68f4700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559078, event: throttled, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1165> 2014-11-03 11:37:56.559313 7f28f68f4700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559196, event: all_read, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1164> 2014-11-03 11:37:56.559332 7f28f68f4700 5 -- op tracker -- seq: 3264, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1163> 2014-11-03 11:37:56.559367 7f291450b700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.559367, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1162> 2014-11-03 11:37:56.559427 7f2911d06700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.559427, event: commit_sent, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1161> 2014-11-03 11:37:56.559454 7f290001b700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559454, event: reached_pg, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1160> 2014-11-03 11:37:56.559472 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- osd_sub_op_reply(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2431f2c0 con 0x230a92c0 -1159> 2014-11-03 11:37:56.559524 7f290001b700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559524, event: started, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1158> 2014-11-03 11:37:56.559662 7f290001b700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559662, event: started, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1157> 2014-11-03 11:37:56.559709 7f290001b700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559709, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1156> 2014-11-03 11:37:56.559813 7f2914d0c700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559813, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1155> 2014-11-03 11:37:56.559855 7f2912507700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.559855, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1154> 2014-11-03 11:37:56.559934 7f2912507700 5 -- op tracker -- seq: 3263, time: 2014-11-03 11:37:56.559934, event: done, op: osd_sub_op(client.7867942.0:137198370 6.75f b3bb5f5f/rbd_data.43c40916a10.00000000000004f1/head//6 [] v 105950'15108021 snapset=0=[]:[] snapc=0=[]) -1153> 2014-11-03 11:37:56.559999 7f291450b700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.559999, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1152> 2014-11-03 11:37:56.560089 7f2911d06700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.560088, event: commit_sent, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1151> 2014-11-03 11:37:56.560149 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7580 con 0x22fbc840 -1150> 2014-11-03 11:37:56.560478 7f2912507700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.560478, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1149> 2014-11-03 11:37:56.560575 7f2912507700 5 -- op tracker -- seq: 3264, time: 2014-11-03 11:37:56.560574, event: done, op: osd_sub_op(client.7867942.0:137198366 6.57c 9d157d7c/rbd_data.43c40916a10.00000000000007bb/head//6 [] v 105950'15047360 snapset=0=[]:[] snapc=0=[]) -1148> 2014-11-03 11:37:56.561681 7f28f038f700 1 -- 192.168.5.2:6803/7606 <== osd.99 192.168.5.8:6829/26930 54 ==== osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) v11 ==== 1394+0+39 (1488103959 0 3511520646) 0x27071800 con 0x230a9b00 -1147> 2014-11-03 11:37:56.561734 7f28f038f700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.561526, event: header_read, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1146> 2014-11-03 11:37:56.561768 7f28f038f700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.561528, event: throttled, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1145> 2014-11-03 11:37:56.561798 7f28f038f700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.561665, event: all_read, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1144> 2014-11-03 11:37:56.561830 7f28f038f700 5 -- op tracker -- seq: 3265, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1143> 2014-11-03 11:37:56.561933 7f290101d700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.561932, event: reached_pg, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1142> 2014-11-03 11:37:56.561981 7f290101d700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.561981, event: started, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1141> 2014-11-03 11:37:56.562139 7f290101d700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.562139, event: started, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1140> 2014-11-03 11:37:56.562216 7f290101d700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.562216, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1139> 2014-11-03 11:37:56.562305 7f2914d0c700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.562305, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1138> 2014-11-03 11:37:56.562586 7f291450b700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.562586, event: journaled_completion_queued, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1137> 2014-11-03 11:37:56.562670 7f2911d06700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.562670, event: commit_sent, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1136> 2014-11-03 11:37:56.562732 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6829/26930 -- osd_sub_op_reply(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b8600 con 0x230a9b00 -1135> 2014-11-03 11:37:56.563051 7f2912507700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.563051, event: sub_op_applied, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1134> 2014-11-03 11:37:56.563120 7f2912507700 5 -- op tracker -- seq: 3265, time: 2014-11-03 11:37:56.563120, event: done, op: osd_sub_op(client.7867942.0:137198365 6.458 6c1b3c58/rbd_data.43c40916a10.00000000000007b1/head//6 [] v 105950'17850157 snapset=0=[]:[] snapc=0=[]) -1133> 2014-11-03 11:37:56.592485 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 44 ==== MOSDPGPush(6.9d8 105950 [PushOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, version: 105945'7949840, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (4053654175 0 0) 0x29884800 con 0x22fe0420 -1132> 2014-11-03 11:37:56.592544 7f28f5be7700 5 -- op tracker -- seq: 3266, time: 2014-11-03 11:37:56.575163, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, version: 105945'7949840, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1131> 2014-11-03 11:37:56.592567 7f28f5be7700 5 -- op tracker -- seq: 3266, time: 2014-11-03 11:37:56.575165, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, version: 105945'7949840, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1130> 2014-11-03 11:37:56.592592 7f28f5be7700 5 -- op tracker -- seq: 3266, time: 2014-11-03 11:37:56.592467, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, version: 105945'7949840, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1129> 2014-11-03 11:37:56.592609 7f28f5be7700 5 -- op tracker -- seq: 3266, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, version: 105945'7949840, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1128> 2014-11-03 11:37:56.592695 7f290201f700 5 -- op tracker -- seq: 3266, time: 2014-11-03 11:37:56.592694, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, version: 105945'7949840, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1127> 2014-11-03 11:37:56.593030 7f290201f700 5 -- op tracker -- seq: 3266, time: 2014-11-03 11:37:56.593030, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6, version: 105945'7949840, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(947399d8/rbd_data.89cd895a911542.000000000000a054/head//6@105945'7949840, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1126> 2014-11-03 11:37:56.613939 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.35 192.168.5.3:0/6455 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.613567) v2 ==== 47+0+0 (6244705 0 0) 0x229e7c00 con 0x2484a6e0 -1125> 2014-11-03 11:37:56.613999 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.613567) v2 -- ?+0 0x26bdb800 con 0x2484a6e0 -1124> 2014-11-03 11:37:56.614333 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.7868162 192.168.1.71:0/2009911 8 ==== ping magic: 0 v1 ==== 0+0+0 (0 0 0) 0x32f7a80 con 0x2311c840 -1123> 2014-11-03 11:37:56.615056 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.35 192.168.5.3:0/6455 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.613567) v2 ==== 47+0+0 (6244705 0 0) 0x2be31a00 con 0x24321160 -1122> 2014-11-03 11:37:56.615125 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/6455 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.613567) v2 -- ?+0 0x8714c00 con 0x24321160 -1121> 2014-11-03 11:37:56.616458 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.9 192.168.5.1:0/17062 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.615795) v2 ==== 47+0+0 (722481065 0 0) 0x5f22400 con 0x24292580 -1120> 2014-11-03 11:37:56.616509 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/17062 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.615795) v2 -- ?+0 0x229e7c00 con 0x24292580 -1119> 2014-11-03 11:37:56.616532 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.9 192.168.5.1:0/17062 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.615795) v2 ==== 47+0+0 (722481065 0 0) 0x13def800 con 0x24321580 -1118> 2014-11-03 11:37:56.616601 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/17062 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.615795) v2 -- ?+0 0x2be31a00 con 0x24321580 -1117> 2014-11-03 11:37:56.619896 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.83 192.168.5.6:0/4005 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.619085) v2 ==== 47+0+0 (2713299798 0 0) 0xddd0600 con 0x2445c2c0 -1116> 2014-11-03 11:37:56.619957 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.619085) v2 -- ?+0 0x5f22400 con 0x2445c2c0 -1115> 2014-11-03 11:37:56.619986 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.83 192.168.5.6:0/4005 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.619085) v2 ==== 47+0+0 (2713299798 0 0) 0x5d49800 con 0x23ea4420 -1114> 2014-11-03 11:37:56.620020 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/4005 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.619085) v2 -- ?+0 0x13def800 con 0x23ea4420 -1113> 2014-11-03 11:37:56.633350 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x17ae8e00 con 0x22fe0420 -1112> 2014-11-03 11:37:56.637664 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 101 ==== osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3621320512 0 3511520646) 0x2799b400 con 0x22fbc420 -1111> 2014-11-03 11:37:56.637738 7f28f72fe700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.637437, event: header_read, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1110> 2014-11-03 11:37:56.637785 7f28f72fe700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.637441, event: throttled, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1109> 2014-11-03 11:37:56.637819 7f28f72fe700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.637626, event: all_read, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1108> 2014-11-03 11:37:56.637850 7f28f72fe700 5 -- op tracker -- seq: 3267, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1107> 2014-11-03 11:37:56.637960 7f28ff81a700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.637960, event: reached_pg, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1106> 2014-11-03 11:37:56.638011 7f28ff81a700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.638011, event: started, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1105> 2014-11-03 11:37:56.638198 7f28ff81a700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.638198, event: started, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1104> 2014-11-03 11:37:56.638262 7f28ff81a700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.638262, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1103> 2014-11-03 11:37:56.638379 7f2914d0c700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.638379, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1102> 2014-11-03 11:37:56.641462 7f291450b700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.641462, event: journaled_completion_queued, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1101> 2014-11-03 11:37:56.641570 7f2911d06700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.641570, event: commit_sent, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1100> 2014-11-03 11:37:56.641615 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7b00 con 0x22fbc420 -1099> 2014-11-03 11:37:56.641846 7f2912507700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.641846, event: sub_op_applied, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1098> 2014-11-03 11:37:56.641889 7f2912507700 5 -- op tracker -- seq: 3267, time: 2014-11-03 11:37:56.641889, event: done, op: osd_sub_op(client.7912580.0:17283155 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944779 snapset=0=[]:[] snapc=0=[]) -1097> 2014-11-03 11:37:56.642815 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 102 ==== osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1139569476 0 3511520646) 0x2e169400 con 0x22fbc420 -1096> 2014-11-03 11:37:56.642852 7f28f72fe700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.642713, event: header_read, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1095> 2014-11-03 11:37:56.642873 7f28f72fe700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.642714, event: throttled, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1094> 2014-11-03 11:37:56.642895 7f28f72fe700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.642803, event: all_read, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1093> 2014-11-03 11:37:56.642915 7f28f72fe700 5 -- op tracker -- seq: 3268, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1092> 2014-11-03 11:37:56.642994 7f290201f700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.642994, event: reached_pg, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1091> 2014-11-03 11:37:56.643027 7f290201f700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.643027, event: started, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1090> 2014-11-03 11:37:56.643157 7f290201f700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.643157, event: started, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1089> 2014-11-03 11:37:56.643188 7f290201f700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.643188, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1088> 2014-11-03 11:37:56.643310 7f2914d0c700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.643310, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1087> 2014-11-03 11:37:56.643648 7f291450b700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.643636, event: journaled_completion_queued, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1086> 2014-11-03 11:37:56.643731 7f2911d06700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.643731, event: commit_sent, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1085> 2014-11-03 11:37:56.643767 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b7840 con 0x22fbc420 -1084> 2014-11-03 11:37:56.644008 7f2912507700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.644008, event: sub_op_applied, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1083> 2014-11-03 11:37:56.644057 7f2912507700 5 -- op tracker -- seq: 3268, time: 2014-11-03 11:37:56.644057, event: done, op: osd_sub_op(client.7912580.0:17283156 6.4c4 6bcfdcc4/rbd_data.74ae9c3be03aff.0000000000000fd9/head//6 [] v 105950'10944780 snapset=0=[]:[] snapc=0=[]) -1082> 2014-11-03 11:37:56.663074 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 46 ==== osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1916246665 0 3511520646) 0x191b6800 con 0x231ab580 -1081> 2014-11-03 11:37:56.663136 7f28ea430700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.662894, event: header_read, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1080> 2014-11-03 11:37:56.663170 7f28ea430700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.662898, event: throttled, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1079> 2014-11-03 11:37:56.663200 7f28ea430700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.663054, event: all_read, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1078> 2014-11-03 11:37:56.663213 7f28ea430700 5 -- op tracker -- seq: 3269, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1077> 2014-11-03 11:37:56.663276 7f290181e700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.663276, event: reached_pg, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1076> 2014-11-03 11:37:56.663313 7f290181e700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.663313, event: started, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1075> 2014-11-03 11:37:56.663475 7f290181e700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.663474, event: started, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1074> 2014-11-03 11:37:56.663525 7f290181e700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.663524, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1073> 2014-11-03 11:37:56.663649 7f2914d0c700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.663649, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1072> 2014-11-03 11:37:56.663958 7f291450b700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.663957, event: journaled_completion_queued, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1071> 2014-11-03 11:37:56.664137 7f2911d06700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.664137, event: commit_sent, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1070> 2014-11-03 11:37:56.664197 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2e4b72c0 con 0x231ab580 -1069> 2014-11-03 11:37:56.664496 7f2912507700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.664495, event: sub_op_applied, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1068> 2014-11-03 11:37:56.664548 7f2912507700 5 -- op tracker -- seq: 3269, time: 2014-11-03 11:37:56.664547, event: done, op: osd_sub_op(client.7869893.0:738139 6.a79 a2a4aa79/rbd_data.4f79bc2ae8944a.0000000000000eba/head//6 [] v 105950'8629965 snapset=0=[]:[] snapc=0=[]) -1067> 2014-11-03 11:37:56.704173 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 200 ==== osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3554705957 0 3511520646) 0x285d8400 con 0x22fefdc0 -1066> 2014-11-03 11:37:56.704220 7f28f36c2700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.703900, event: header_read, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1065> 2014-11-03 11:37:56.704250 7f28f36c2700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.703904, event: throttled, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1064> 2014-11-03 11:37:56.704266 7f28f36c2700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.704154, event: all_read, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1063> 2014-11-03 11:37:56.704288 7f28f36c2700 5 -- op tracker -- seq: 3270, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1062> 2014-11-03 11:37:56.704390 7f28fe818700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.704390, event: reached_pg, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1061> 2014-11-03 11:37:56.704468 7f28fe818700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.704468, event: started, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1060> 2014-11-03 11:37:56.704634 7f28fe818700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.704634, event: started, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1059> 2014-11-03 11:37:56.704700 7f28fe818700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.704700, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1058> 2014-11-03 11:37:56.704856 7f2914d0c700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.704855, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1057> 2014-11-03 11:37:56.705128 7f291450b700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.705128, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1056> 2014-11-03 11:37:56.705224 7f2911d06700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.705224, event: commit_sent, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1055> 2014-11-03 11:37:56.705274 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x1c94d080 con 0x22fefdc0 -1054> 2014-11-03 11:37:56.705591 7f2912507700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.705591, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1053> 2014-11-03 11:37:56.705682 7f2912507700 5 -- op tracker -- seq: 3270, time: 2014-11-03 11:37:56.705682, event: done, op: osd_sub_op(client.8703713.0:32254528 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953161 snapset=0=[]:[] snapc=0=[]) -1052> 2014-11-03 11:37:56.718438 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 45 ==== MOSDPGPush(6.9d8 105950 [PushOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, version: 105945'7949866, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195218+0+0 (2574690715 0 0) 0x17ae8e00 con 0x22fe0420 -1051> 2014-11-03 11:37:56.718495 7f28f5be7700 5 -- op tracker -- seq: 3271, time: 2014-11-03 11:37:56.710914, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, version: 105945'7949866, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1050> 2014-11-03 11:37:56.718515 7f28f5be7700 5 -- op tracker -- seq: 3271, time: 2014-11-03 11:37:56.710922, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, version: 105945'7949866, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1049> 2014-11-03 11:37:56.718553 7f28f5be7700 5 -- op tracker -- seq: 3271, time: 2014-11-03 11:37:56.718417, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, version: 105945'7949866, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1048> 2014-11-03 11:37:56.718587 7f28f5be7700 5 -- op tracker -- seq: 3271, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, version: 105945'7949866, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1047> 2014-11-03 11:37:56.718705 7f28ff81a700 5 -- op tracker -- seq: 3271, time: 2014-11-03 11:37:56.718705, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, version: 105945'7949866, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1046> 2014-11-03 11:37:56.718967 7f28ff81a700 5 -- op tracker -- seq: 3271, time: 2014-11-03 11:37:56.718967, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6, version: 105945'7949866, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e83879d8/rbd_data.4402482ae8944a.00000000000016ea/head//6@105945'7949866, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -1045> 2014-11-03 11:37:56.729668 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- MOSDPGPull(6.9d8 105950 [PullOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x54a3c00 con 0x22fb7160 -1044> 2014-11-03 11:37:56.737384 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.99 192.168.5.8:0/26930 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.736284) v2 ==== 47+0+0 (2103520095 0 0) 0x23cbea00 con 0x24205580 -1043> 2014-11-03 11:37:56.737435 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.736284) v2 -- ?+0 0x5d49800 con 0x24205580 -1042> 2014-11-03 11:37:56.738203 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.99 192.168.5.8:0/26930 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.736284) v2 ==== 47+0+0 (2103520095 0 0) 0x9459200 con 0x239f8160 -1041> 2014-11-03 11:37:56.738264 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/26930 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.736284) v2 -- ?+0 0xddd0600 con 0x239f8160 -1040> 2014-11-03 11:37:56.741654 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 105 ==== osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2310171748 0 3511520646) 0x26dbb800 con 0x2308b840 -1039> 2014-11-03 11:37:56.741698 7f28ef985700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.741492, event: header_read, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1038> 2014-11-03 11:37:56.741718 7f28ef985700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.741493, event: throttled, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1037> 2014-11-03 11:37:56.741731 7f28ef985700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.741638, event: all_read, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1036> 2014-11-03 11:37:56.741743 7f28ef985700 5 -- op tracker -- seq: 3272, time: 0.000000, event: dispatched, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1035> 2014-11-03 11:37:56.741806 7f290201f700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.741806, event: reached_pg, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1034> 2014-11-03 11:37:56.741838 7f290201f700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.741838, event: started, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1033> 2014-11-03 11:37:56.741852 7f290d52b700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31524000 sd=201 :6802 s=0 pgs=0 cs=0 l=0 c=0xee04b00).accept sd=201 192.168.1.84:56640/0 -1032> 2014-11-03 11:37:56.741929 7f290201f700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.741929, event: started, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1031> 2014-11-03 11:37:56.741966 7f290201f700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.741966, event: commit_queued_for_journal_write, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1030> 2014-11-03 11:37:56.742029 7f2914d0c700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.742029, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1029> 2014-11-03 11:37:56.742325 7f291450b700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.742324, event: journaled_completion_queued, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1028> 2014-11-03 11:37:56.742474 7f2911d06700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.742474, event: commit_sent, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1027> 2014-11-03 11:37:56.742516 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94c840 con 0x2308b840 -1026> 2014-11-03 11:37:56.742773 7f290d52b700 1 -- 192.168.5.2:6802/7606 <== client.7869968 192.168.1.84:0/1003447 1 ==== osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) v4 ==== 229+0+4096 (2310153476 0 576820508) 0x32be000 con 0xee04b00 -1025> 2014-11-03 11:37:56.742808 7f290d52b700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.742691, event: header_read, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1024> 2014-11-03 11:37:56.742828 7f290d52b700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.742696, event: throttled, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1023> 2014-11-03 11:37:56.742842 7f290d52b700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.742760, event: all_read, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1022> 2014-11-03 11:37:56.742852 7f290d52b700 5 -- op tracker -- seq: 3273, time: 0.000000, event: dispatched, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1021> 2014-11-03 11:37:56.742937 7f2912507700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.742937, event: sub_op_applied, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1020> 2014-11-03 11:37:56.743007 7f2912507700 5 -- op tracker -- seq: 3272, time: 2014-11-03 11:37:56.743007, event: done, op: osd_sub_op(client.7868671.0:3458117 6.690 27791e90/rbd_data.3fc673558a2c1e.000000000000425f/head//6 [] v 105950'28900017 snapset=0=[]:[] snapc=0=[]) -1019> 2014-11-03 11:37:56.743015 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.743014, event: reached_pg, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1018> 2014-11-03 11:37:56.743190 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.743189, event: started, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1017> 2014-11-03 11:37:56.743230 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.743230, event: started, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1016> 2014-11-03 11:37:56.743343 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.743343, event: waiting for subops from 103,114, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1015> 2014-11-03 11:37:56.743400 7f28fd816700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] v 105950'14083248 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4921 0x24450400 con 0x22fefdc0 -1014> 2014-11-03 11:37:56.743469 7f28fd816700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6816/17408 -- osd_sub_op(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] v 105950'14083248 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4921 0x3349800 con 0x230a92c0 -1013> 2014-11-03 11:37:56.743631 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.743630, event: commit_queued_for_journal_write, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1012> 2014-11-03 11:37:56.743783 7f2914d0c700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.743782, event: write_thread_in_journal_buffer, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1011> 2014-11-03 11:37:56.744151 7f291450b700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.744151, event: journaled_completion_queued, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1010> 2014-11-03 11:37:56.744234 7f2911d06700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.744234, event: op_commit, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1009> 2014-11-03 11:37:56.744924 7f2912507700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.744924, event: op_applied, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1008> 2014-11-03 11:37:56.745138 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 201 ==== osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) v2 ==== 178+0+0 (2413907871 0 0) 0x1c94d080 con 0x22fefdc0 -1007> 2014-11-03 11:37:56.745190 7f28f36c2700 5 -- op tracker -- seq: 3274, time: 2014-11-03 11:37:56.744965, event: header_read, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -1006> 2014-11-03 11:37:56.745209 7f28f36c2700 5 -- op tracker -- seq: 3274, time: 2014-11-03 11:37:56.744968, event: throttled, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -1005> 2014-11-03 11:37:56.745222 7f28f36c2700 5 -- op tracker -- seq: 3274, time: 2014-11-03 11:37:56.745123, event: all_read, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -1004> 2014-11-03 11:37:56.745238 7f28f36c2700 5 -- op tracker -- seq: 3274, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -1003> 2014-11-03 11:37:56.745312 7f290001b700 5 -- op tracker -- seq: 3274, time: 2014-11-03 11:37:56.745312, event: reached_pg, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -1002> 2014-11-03 11:37:56.745331 7f290001b700 5 -- op tracker -- seq: 3274, time: 2014-11-03 11:37:56.745331, event: started, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -1001> 2014-11-03 11:37:56.745343 7f290001b700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.745343, event: sub_op_commit_rec, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -1000> 2014-11-03 11:37:56.745359 7f290001b700 5 -- op tracker -- seq: 3274, time: 2014-11-03 11:37:56.745359, event: done, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -999> 2014-11-03 11:37:56.745391 7f28f0a96700 1 -- 192.168.5.2:6803/7606 <== osd.114 192.168.5.9:6816/17408 99 ==== osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) v2 ==== 178+0+0 (304620495 0 0) 0x23de8340 con 0x230a92c0 -998> 2014-11-03 11:37:56.745441 7f28f0a96700 5 -- op tracker -- seq: 3275, time: 2014-11-03 11:37:56.745315, event: header_read, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -997> 2014-11-03 11:37:56.745494 7f28f0a96700 5 -- op tracker -- seq: 3275, time: 2014-11-03 11:37:56.745317, event: throttled, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -996> 2014-11-03 11:37:56.745525 7f28f0a96700 5 -- op tracker -- seq: 3275, time: 2014-11-03 11:37:56.745378, event: all_read, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -995> 2014-11-03 11:37:56.745564 7f28f0a96700 5 -- op tracker -- seq: 3275, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -994> 2014-11-03 11:37:56.745678 7f28fd816700 5 -- op tracker -- seq: 3275, time: 2014-11-03 11:37:56.745678, event: reached_pg, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -993> 2014-11-03 11:37:56.745704 7f28fd816700 5 -- op tracker -- seq: 3275, time: 2014-11-03 11:37:56.745703, event: started, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -992> 2014-11-03 11:37:56.745714 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.745714, event: sub_op_commit_rec, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -991> 2014-11-03 11:37:56.745730 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.84:0/1003447 -- osd_op_reply(674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] v105950'14083248 uv14083248 ack = 0) v6 -- ?+0 0x31524dc0 con 0xee04b00 -990> 2014-11-03 11:37:56.745763 7f28fd816700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.84:0/1003447 -- osd_op_reply(674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] v105950'14083248 uv14083248 ondisk = 0) v6 -- ?+0 0x27418600 con 0xee04b00 -989> 2014-11-03 11:37:56.745773 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.745773, event: commit_sent, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -988> 2014-11-03 11:37:56.745801 7f28fd816700 5 -- op tracker -- seq: 3273, time: 2014-11-03 11:37:56.745801, event: done, op: osd_op(client.7869968.0:674633 rbd_data.3125f13abf616c.0000000000000001 [set-alloc-hint object_size 4194304 write_size 4194304,write 143360~4096] 6.7372362b ack+ondisk+write e105950) -987> 2014-11-03 11:37:56.745826 7f28fd816700 5 -- op tracker -- seq: 3275, time: 2014-11-03 11:37:56.745826, event: done, op: osd_sub_op_reply(client.7869968.0:674633 6.62b 7372362b/rbd_data.3125f13abf616c.0000000000000001/head//6 [] ondisk, result = 0) -986> 2014-11-03 11:37:56.758393 7f290d05f700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x31524580 sd=547 :6802 s=0 pgs=0 cs=0 l=0 c=0xee042c0).accept sd=547 192.168.1.73:41676/0 -985> 2014-11-03 11:37:56.759291 7f290d05f700 1 -- 192.168.5.2:6802/7606 <== client.7870167 192.168.1.73:0/2009743 1 ==== osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) v4 ==== 229+0+4096 (2581637838 0 195728429) 0x100dc500 con 0xee042c0 -984> 2014-11-03 11:37:56.759330 7f290d05f700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.759189, event: header_read, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -983> 2014-11-03 11:37:56.759370 7f290d05f700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.759194, event: throttled, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -982> 2014-11-03 11:37:56.759388 7f290d05f700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.759276, event: all_read, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -981> 2014-11-03 11:37:56.759402 7f290d05f700 5 -- op tracker -- seq: 3276, time: 0.000000, event: dispatched, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -980> 2014-11-03 11:37:56.759538 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.759538, event: reached_pg, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -979> 2014-11-03 11:37:56.759699 7f28ff019700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6868/8391 -- MOSDPGPull(6.312 105950 [PullOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x13d27400 con 0x22fea6e0 -978> 2014-11-03 11:37:56.759775 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.759775, event: waiting for missing object, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -977> 2014-11-03 11:37:56.760137 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.4 192.168.5.1:0/31636 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.759516) v2 ==== 47+0+0 (4285767005 0 0) 0x1a22ce00 con 0x31a9580 -976> 2014-11-03 11:37:56.760189 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/31636 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.759516) v2 -- ?+0 0x9459200 con 0x31a9580 -975> 2014-11-03 11:37:56.760589 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.4 192.168.5.1:0/31636 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.759516) v2 ==== 47+0+0 (4285767005 0 0) 0x405b800 con 0x2389f580 -974> 2014-11-03 11:37:56.760630 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/31636 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.759516) v2 -- ?+0 0x23cbea00 con 0x2389f580 -973> 2014-11-03 11:37:56.776469 7f28c4694700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x263aedc0 sd=566 :6802 s=0 pgs=0 cs=0 l=0 c=0x24aba9a0).accept sd=566 192.168.5.2:43007/0 -972> 2014-11-03 11:37:56.777703 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12420000 192.168.5.2:0/1005106 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x236a8400 con 0x24aba9a0 -971> 2014-11-03 11:37:56.778534 7f28fb812700 0 osd.21 105950 do_command r=0 -970> 2014-11-03 11:37:56.778561 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1005106 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x78351c0 con 0x24aba9a0 -969> 2014-11-03 11:37:56.791230 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12420000 192.168.5.2:0/1005106 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23cbea00 con 0x24aba9a0 -968> 2014-11-03 11:37:56.792558 7f28fb812700 0 osd.21 105950 do_command r=0 -967> 2014-11-03 11:37:56.792589 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1005106 -- command_reply(tid 2: 0 ) v1 -- ?+16699 0x3684e00 con 0x24aba9a0 -966> 2014-11-03 11:37:56.798006 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1005106 pipe(0x263aedc0 sd=566 :6802 s=2 pgs=2 cs=1 l=1 c=0x24aba9a0).reader couldn't read tag, (0) Success -965> 2014-11-03 11:37:56.798049 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1005106 pipe(0x263aedc0 sd=566 :6802 s=2 pgs=2 cs=1 l=1 c=0x24aba9a0).fault (0) Success -964> 2014-11-03 11:37:56.798203 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x24aba9a0 session 0x100dc780 -963> 2014-11-03 11:37:56.807164 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 202 ==== osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (796043296 0 3511520646) 0x24450400 con 0x22fefdc0 -962> 2014-11-03 11:37:56.807246 7f28f36c2700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.806959, event: header_read, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -961> 2014-11-03 11:37:56.807274 7f28f36c2700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.806961, event: throttled, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -960> 2014-11-03 11:37:56.807291 7f28f36c2700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.807129, event: all_read, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -959> 2014-11-03 11:37:56.807304 7f28f36c2700 5 -- op tracker -- seq: 3277, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -958> 2014-11-03 11:37:56.807453 7f28ff81a700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.807453, event: reached_pg, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -957> 2014-11-03 11:37:56.807503 7f28ff81a700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.807503, event: started, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -956> 2014-11-03 11:37:56.807634 7f28ff81a700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.807634, event: started, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -955> 2014-11-03 11:37:56.807698 7f28ff81a700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.807698, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -954> 2014-11-03 11:37:56.807808 7f2914d0c700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.807807, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -953> 2014-11-03 11:37:56.808066 7f291450b700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.808065, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -952> 2014-11-03 11:37:56.808155 7f2911d06700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.808155, event: commit_sent, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -951> 2014-11-03 11:37:56.808180 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94d600 con 0x22fefdc0 -950> 2014-11-03 11:37:56.808448 7f2912507700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.808448, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -949> 2014-11-03 11:37:56.808490 7f2912507700 5 -- op tracker -- seq: 3277, time: 2014-11-03 11:37:56.808490, event: done, op: osd_sub_op(client.7914641.0:33714022 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033690 snapset=0=[]:[] snapc=0=[]) -948> 2014-11-03 11:37:56.808647 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 203 ==== osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3183060966 0 3511520646) 0x27071800 con 0x22fefdc0 -947> 2014-11-03 11:37:56.808683 7f28f36c2700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808529, event: header_read, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -946> 2014-11-03 11:37:56.808700 7f28f36c2700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808536, event: throttled, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -945> 2014-11-03 11:37:56.808709 7f28f36c2700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808636, event: all_read, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -944> 2014-11-03 11:37:56.808718 7f28f36c2700 5 -- op tracker -- seq: 3278, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -943> 2014-11-03 11:37:56.808765 7f290201f700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808765, event: reached_pg, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -942> 2014-11-03 11:37:56.808796 7f290201f700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808796, event: started, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -941> 2014-11-03 11:37:56.808883 7f290201f700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808883, event: started, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -940> 2014-11-03 11:37:56.808911 7f290201f700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808911, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -939> 2014-11-03 11:37:56.808967 7f2914d0c700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.808967, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -938> 2014-11-03 11:37:56.809119 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 204 ==== osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3376913043 0 3511520646) 0x2e169400 con 0x22fefdc0 -937> 2014-11-03 11:37:56.809162 7f28f36c2700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809021, event: header_read, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -936> 2014-11-03 11:37:56.809179 7f291450b700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.809178, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -935> 2014-11-03 11:37:56.809186 7f28f36c2700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809023, event: throttled, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -934> 2014-11-03 11:37:56.809230 7f28f36c2700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809105, event: all_read, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -933> 2014-11-03 11:37:56.809244 7f28f36c2700 5 -- op tracker -- seq: 3279, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -932> 2014-11-03 11:37:56.809238 7f2911d06700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.809238, event: commit_sent, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -931> 2014-11-03 11:37:56.809280 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94cb00 con 0x22fefdc0 -930> 2014-11-03 11:37:56.809317 7f28ff81a700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809317, event: reached_pg, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -929> 2014-11-03 11:37:56.809336 7f28ff81a700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809335, event: started, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -928> 2014-11-03 11:37:56.809470 7f28ff81a700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809470, event: started, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -927> 2014-11-03 11:37:56.809505 7f28ff81a700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809505, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -926> 2014-11-03 11:37:56.809557 7f2914d0c700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809557, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -925> 2014-11-03 11:37:56.809641 7f2912507700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.809641, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -924> 2014-11-03 11:37:56.809676 7f2912507700 5 -- op tracker -- seq: 3278, time: 2014-11-03 11:37:56.809676, event: done, op: osd_sub_op(client.7914641.0:33714023 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033691 snapset=0=[]:[] snapc=0=[]) -923> 2014-11-03 11:37:56.809721 7f291450b700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809721, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -922> 2014-11-03 11:37:56.809756 7f2911d06700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.809755, event: commit_sent, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -921> 2014-11-03 11:37:56.809792 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94c580 con 0x22fefdc0 -920> 2014-11-03 11:37:56.810018 7f2912507700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.810018, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -919> 2014-11-03 11:37:56.810051 7f2912507700 5 -- op tracker -- seq: 3279, time: 2014-11-03 11:37:56.810051, event: done, op: osd_sub_op(client.7914641.0:33714024 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033692 snapset=0=[]:[] snapc=0=[]) -918> 2014-11-03 11:37:56.810546 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 205 ==== osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2123034292 0 3511520646) 0x275f3800 con 0x22fefdc0 -917> 2014-11-03 11:37:56.810581 7f28f36c2700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810473, event: header_read, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -916> 2014-11-03 11:37:56.810597 7f28f36c2700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810474, event: throttled, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -915> 2014-11-03 11:37:56.810609 7f28f36c2700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810535, event: all_read, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -914> 2014-11-03 11:37:56.810621 7f28f36c2700 5 -- op tracker -- seq: 3280, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -913> 2014-11-03 11:37:56.810669 7f290201f700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810668, event: reached_pg, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -912> 2014-11-03 11:37:56.810695 7f290201f700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810695, event: started, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -911> 2014-11-03 11:37:56.810779 7f290201f700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810778, event: started, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -910> 2014-11-03 11:37:56.810805 7f290201f700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810805, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -909> 2014-11-03 11:37:56.810858 7f2914d0c700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.810858, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -908> 2014-11-03 11:37:56.811024 7f291450b700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.811023, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -907> 2014-11-03 11:37:56.811065 7f2911d06700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.811065, event: commit_sent, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -906> 2014-11-03 11:37:56.811103 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x1c94c2c0 con 0x22fefdc0 -905> 2014-11-03 11:37:56.811273 7f2912507700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.811273, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -904> 2014-11-03 11:37:56.811303 7f2912507700 5 -- op tracker -- seq: 3280, time: 2014-11-03 11:37:56.811303, event: done, op: osd_sub_op(client.7914641.0:33714025 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033693 snapset=0=[]:[] snapc=0=[]) -903> 2014-11-03 11:37:56.812230 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 206 ==== osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3423805615 0 3511520646) 0x23d99000 con 0x22fefdc0 -902> 2014-11-03 11:37:56.812264 7f28f36c2700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812103, event: header_read, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -901> 2014-11-03 11:37:56.812280 7f28f36c2700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812104, event: throttled, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -900> 2014-11-03 11:37:56.812292 7f28f36c2700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812219, event: all_read, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -899> 2014-11-03 11:37:56.812304 7f28f36c2700 5 -- op tracker -- seq: 3281, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -898> 2014-11-03 11:37:56.812349 7f28ff81a700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812349, event: reached_pg, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -897> 2014-11-03 11:37:56.812379 7f28ff81a700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812379, event: started, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -896> 2014-11-03 11:37:56.812489 7f28ff81a700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812489, event: started, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -895> 2014-11-03 11:37:56.812517 7f28ff81a700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812517, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -894> 2014-11-03 11:37:56.812603 7f2914d0c700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812603, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -893> 2014-11-03 11:37:56.812915 7f291450b700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812914, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -892> 2014-11-03 11:37:56.812956 7f2911d06700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.812956, event: commit_sent, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -891> 2014-11-03 11:37:56.812984 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x24d75b80 con 0x22fefdc0 -890> 2014-11-03 11:37:56.813276 7f2912507700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.813276, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -889> 2014-11-03 11:37:56.813330 7f2912507700 5 -- op tracker -- seq: 3281, time: 2014-11-03 11:37:56.813330, event: done, op: osd_sub_op(client.7914641.0:33714026 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033694 snapset=0=[]:[] snapc=0=[]) -888> 2014-11-03 11:37:56.813378 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 207 ==== osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2301676139 0 3511520646) 0x23eb7400 con 0x22fefdc0 -887> 2014-11-03 11:37:56.813410 7f28f36c2700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813272, event: header_read, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -886> 2014-11-03 11:37:56.813422 7f28f36c2700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813279, event: throttled, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -885> 2014-11-03 11:37:56.813445 7f28f36c2700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813368, event: all_read, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -884> 2014-11-03 11:37:56.813455 7f28f36c2700 5 -- op tracker -- seq: 3282, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -883> 2014-11-03 11:37:56.813501 7f290201f700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813501, event: reached_pg, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -882> 2014-11-03 11:37:56.813523 7f290201f700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813523, event: started, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -881> 2014-11-03 11:37:56.813597 7f290201f700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813597, event: started, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -880> 2014-11-03 11:37:56.813616 7f290201f700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813616, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -879> 2014-11-03 11:37:56.813679 7f2914d0c700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813678, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -878> 2014-11-03 11:37:56.813922 7f291450b700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813922, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -877> 2014-11-03 11:37:56.813957 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 208 ==== osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3423402462 0 3511520646) 0x2e169400 con 0x22fefdc0 -876> 2014-11-03 11:37:56.813971 7f2911d06700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.813971, event: commit_sent, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -875> 2014-11-03 11:37:56.813984 7f28f36c2700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.813860, event: header_read, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -874> 2014-11-03 11:37:56.814000 7f28f36c2700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.813861, event: throttled, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -873> 2014-11-03 11:37:56.814004 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x24d758c0 con 0x22fefdc0 -872> 2014-11-03 11:37:56.814009 7f28f36c2700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.813947, event: all_read, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -871> 2014-11-03 11:37:56.814034 7f28f36c2700 5 -- op tracker -- seq: 3283, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -870> 2014-11-03 11:37:56.814090 7f28ff81a700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814090, event: reached_pg, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -869> 2014-11-03 11:37:56.814137 7f28ff81a700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814136, event: started, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -868> 2014-11-03 11:37:56.814266 7f28ff81a700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814265, event: started, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -867> 2014-11-03 11:37:56.814304 7f28ff81a700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814304, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -866> 2014-11-03 11:37:56.814365 7f2912507700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.814365, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -865> 2014-11-03 11:37:56.814387 7f2914d0c700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814387, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -864> 2014-11-03 11:37:56.814419 7f2912507700 5 -- op tracker -- seq: 3282, time: 2014-11-03 11:37:56.814419, event: done, op: osd_sub_op(client.7914641.0:33714027 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033695 snapset=0=[]:[] snapc=0=[]) -863> 2014-11-03 11:37:56.814551 7f291450b700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814551, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -862> 2014-11-03 11:37:56.814592 7f2911d06700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814592, event: commit_sent, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -861> 2014-11-03 11:37:56.814628 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650e340 con 0x22fefdc0 -860> 2014-11-03 11:37:56.814902 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 209 ==== osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2308946447 0 3511520646) 0x27071800 con 0x22fefdc0 -859> 2014-11-03 11:37:56.814931 7f2912507700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814931, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -858> 2014-11-03 11:37:56.814936 7f28f36c2700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.814763, event: header_read, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -857> 2014-11-03 11:37:56.814971 7f28f36c2700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.814765, event: throttled, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -856> 2014-11-03 11:37:56.814985 7f28f36c2700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.814892, event: all_read, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -855> 2014-11-03 11:37:56.814977 7f2912507700 5 -- op tracker -- seq: 3283, time: 2014-11-03 11:37:56.814977, event: done, op: osd_sub_op(client.7914641.0:33714028 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033696 snapset=0=[]:[] snapc=0=[]) -854> 2014-11-03 11:37:56.814994 7f28f36c2700 5 -- op tracker -- seq: 3284, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -853> 2014-11-03 11:37:56.815042 7f290201f700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815041, event: reached_pg, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -852> 2014-11-03 11:37:56.815065 7f290201f700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815065, event: started, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -851> 2014-11-03 11:37:56.815147 7f290201f700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815147, event: started, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -850> 2014-11-03 11:37:56.815173 7f290201f700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815173, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -849> 2014-11-03 11:37:56.815235 7f2914d0c700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815235, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -848> 2014-11-03 11:37:56.815394 7f291450b700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815393, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -847> 2014-11-03 11:37:56.815429 7f2911d06700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815429, event: commit_sent, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -846> 2014-11-03 11:37:56.815465 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650e8c0 con 0x22fefdc0 -845> 2014-11-03 11:37:56.815650 7f2912507700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815650, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -844> 2014-11-03 11:37:56.815683 7f2912507700 5 -- op tracker -- seq: 3284, time: 2014-11-03 11:37:56.815683, event: done, op: osd_sub_op(client.7914641.0:33714029 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033697 snapset=0=[]:[] snapc=0=[]) -843> 2014-11-03 11:37:56.815949 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 210 ==== osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (450995552 0 3511520646) 0x24da3400 con 0x22fefdc0 -842> 2014-11-03 11:37:56.815977 7f28f36c2700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.815776, event: header_read, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -841> 2014-11-03 11:37:56.816000 7f28f36c2700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.815777, event: throttled, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -840> 2014-11-03 11:37:56.816012 7f28f36c2700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.815939, event: all_read, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -839> 2014-11-03 11:37:56.816022 7f28f36c2700 5 -- op tracker -- seq: 3285, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -838> 2014-11-03 11:37:56.816063 7f28ff81a700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816062, event: reached_pg, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -837> 2014-11-03 11:37:56.816088 7f28ff81a700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816088, event: started, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -836> 2014-11-03 11:37:56.816164 7f28ff81a700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816164, event: started, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -835> 2014-11-03 11:37:56.816190 7f28ff81a700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816190, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -834> 2014-11-03 11:37:56.816248 7f2914d0c700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816248, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -833> 2014-11-03 11:37:56.816290 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 211 ==== osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1313621322 0 3511520646) 0x24450400 con 0x22fefdc0 -832> 2014-11-03 11:37:56.816315 7f28f36c2700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816219, event: header_read, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -831> 2014-11-03 11:37:56.816330 7f28f36c2700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816220, event: throttled, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -830> 2014-11-03 11:37:56.816342 7f28f36c2700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816282, event: all_read, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -829> 2014-11-03 11:37:56.816353 7f28f36c2700 5 -- op tracker -- seq: 3286, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -828> 2014-11-03 11:37:56.816402 7f290201f700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816402, event: reached_pg, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -827> 2014-11-03 11:37:56.816408 7f291450b700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816407, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -826> 2014-11-03 11:37:56.816430 7f290201f700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816430, event: started, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -825> 2014-11-03 11:37:56.816511 7f290201f700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816511, event: started, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -824> 2014-11-03 11:37:56.816539 7f290201f700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816539, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -823> 2014-11-03 11:37:56.816581 7f2911d06700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816581, event: commit_sent, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -822> 2014-11-03 11:37:56.816610 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2650d000 con 0x22fefdc0 -821> 2014-11-03 11:37:56.816606 7f2914d0c700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816605, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -820> 2014-11-03 11:37:56.816652 7f2912507700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816651, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -819> 2014-11-03 11:37:56.816682 7f2912507700 5 -- op tracker -- seq: 3285, time: 2014-11-03 11:37:56.816682, event: done, op: osd_sub_op(client.7914641.0:33714030 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033698 snapset=0=[]:[] snapc=0=[]) -818> 2014-11-03 11:37:56.816927 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 212 ==== osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1461725045 0 3511520646) 0x191b6800 con 0x22fefdc0 -817> 2014-11-03 11:37:56.816945 7f291450b700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816944, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -816> 2014-11-03 11:37:56.816953 7f28f36c2700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.816848, event: header_read, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -815> 2014-11-03 11:37:56.816997 7f2911d06700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.816997, event: commit_sent, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -814> 2014-11-03 11:37:56.817020 7f28f36c2700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.816850, event: throttled, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -813> 2014-11-03 11:37:56.817031 7f28f36c2700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.816919, event: all_read, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -812> 2014-11-03 11:37:56.817039 7f28f36c2700 5 -- op tracker -- seq: 3287, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -811> 2014-11-03 11:37:56.817032 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2fc9ab00 con 0x22fefdc0 -810> 2014-11-03 11:37:56.817096 7f28ff81a700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817096, event: reached_pg, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -809> 2014-11-03 11:37:56.817126 7f28ff81a700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817126, event: started, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -808> 2014-11-03 11:37:56.817265 7f28ff81a700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817265, event: started, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -807> 2014-11-03 11:37:56.817318 7f28ff81a700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817318, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -806> 2014-11-03 11:37:56.817394 7f2914d0c700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817393, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -805> 2014-11-03 11:37:56.817398 7f2912507700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.817398, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -804> 2014-11-03 11:37:56.817460 7f2912507700 5 -- op tracker -- seq: 3286, time: 2014-11-03 11:37:56.817460, event: done, op: osd_sub_op(client.7914641.0:33714031 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033699 snapset=0=[]:[] snapc=0=[]) -803> 2014-11-03 11:37:56.817582 7f291450b700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817581, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -802> 2014-11-03 11:37:56.817617 7f2911d06700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817616, event: commit_sent, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -801> 2014-11-03 11:37:56.817646 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23eaa580 con 0x22fefdc0 -800> 2014-11-03 11:37:56.817916 7f2912507700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817916, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -799> 2014-11-03 11:37:56.817975 7f2912507700 5 -- op tracker -- seq: 3287, time: 2014-11-03 11:37:56.817975, event: done, op: osd_sub_op(client.7914641.0:33714032 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033700 snapset=0=[]:[] snapc=0=[]) -798> 2014-11-03 11:37:56.818488 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 213 ==== osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (1099804731 0 3511520646) 0x27071800 con 0x22fefdc0 -797> 2014-11-03 11:37:56.818516 7f28f36c2700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818379, event: header_read, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -796> 2014-11-03 11:37:56.818548 7f28f36c2700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818380, event: throttled, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -795> 2014-11-03 11:37:56.818561 7f28f36c2700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818458, event: all_read, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -794> 2014-11-03 11:37:56.818574 7f28f36c2700 5 -- op tracker -- seq: 3288, time: 0.000000, event: dispatched, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -793> 2014-11-03 11:37:56.818633 7f290201f700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818633, event: reached_pg, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -792> 2014-11-03 11:37:56.818660 7f290201f700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818660, event: started, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -791> 2014-11-03 11:37:56.818788 7f290201f700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818787, event: started, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -790> 2014-11-03 11:37:56.818819 7f290201f700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818819, event: commit_queued_for_journal_write, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -789> 2014-11-03 11:37:56.818883 7f2914d0c700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.818883, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -788> 2014-11-03 11:37:56.819068 7f291450b700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.819068, event: journaled_completion_queued, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -787> 2014-11-03 11:37:56.819121 7f2911d06700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.819121, event: commit_sent, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -786> 2014-11-03 11:37:56.819150 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] ondisk, result = 0) v2 -- ?+0 0x27804080 con 0x22fefdc0 -785> 2014-11-03 11:37:56.819483 7f2912507700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.819483, event: sub_op_applied, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -784> 2014-11-03 11:37:56.819530 7f2912507700 5 -- op tracker -- seq: 3288, time: 2014-11-03 11:37:56.819530, event: done, op: osd_sub_op(client.7914641.0:33714033 6.799 99fa1f99/rbd_data.49e9f52ae8944a.0000000000000cda/head//6 [] v 105950'15033701 snapset=0=[]:[] snapc=0=[]) -783> 2014-11-03 11:37:56.826608 7f28f66f2700 1 -- 192.168.5.2:6803/7606 <== osd.54 192.168.5.4:6868/8391 19 ==== MOSDPGPush(6.312 105950 [PushOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, version: 105946'7697917, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (2167333465 0 0) 0x13d27400 con 0x22fea6e0 -782> 2014-11-03 11:37:56.826654 7f28f66f2700 5 -- op tracker -- seq: 3289, time: 2014-11-03 11:37:56.810860, event: header_read, op: MOSDPGPush(6.312 105950 [PushOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, version: 105946'7697917, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -781> 2014-11-03 11:37:56.826678 7f28f66f2700 5 -- op tracker -- seq: 3289, time: 2014-11-03 11:37:56.810862, event: throttled, op: MOSDPGPush(6.312 105950 [PushOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, version: 105946'7697917, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -780> 2014-11-03 11:37:56.826713 7f28f66f2700 5 -- op tracker -- seq: 3289, time: 2014-11-03 11:37:56.826590, event: all_read, op: MOSDPGPush(6.312 105950 [PushOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, version: 105946'7697917, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -779> 2014-11-03 11:37:56.826735 7f28f66f2700 5 -- op tracker -- seq: 3289, time: 0.000000, event: dispatched, op: MOSDPGPush(6.312 105950 [PushOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, version: 105946'7697917, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -778> 2014-11-03 11:37:56.826836 7f290181e700 5 -- op tracker -- seq: 3289, time: 2014-11-03 11:37:56.826835, event: reached_pg, op: MOSDPGPush(6.312 105950 [PushOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, version: 105946'7697917, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -777> 2014-11-03 11:37:56.827251 7f290181e700 5 -- op tracker -- seq: 3289, time: 2014-11-03 11:37:56.827251, event: done, op: MOSDPGPush(6.312 105950 [PushOp(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6, version: 105946'7697917, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6@105946'7697917, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -776> 2014-11-03 11:37:56.827255 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.827255, event: reached_pg, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -775> 2014-11-03 11:37:56.827345 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.827345, event: started, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -774> 2014-11-03 11:37:56.827371 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.827371, event: started, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -773> 2014-11-03 11:37:56.827449 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.827449, event: waiting for subops from 54,82, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -772> 2014-11-03 11:37:56.827491 7f28ff019700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6868/8391 -- osd_sub_op(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] v 105950'7697950 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4921 0x23eb6800 con 0x22fea6e0 -771> 2014-11-03 11:37:56.827537 7f28ff019700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6866/31491 -- osd_sub_op(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] v 105950'7697950 snapset=0=[]:[] snapc=0=[]) v11 -- ?+4921 0x23eb5c00 con 0x22fe09a0 -770> 2014-11-03 11:37:56.829481 7f28f66f2700 1 -- 192.168.5.2:6803/7606 <== osd.54 192.168.5.4:6868/8391 20 ==== osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) v2 ==== 178+0+0 (1602108752 0 0) 0x24d758c0 con 0x22fea6e0 -769> 2014-11-03 11:37:56.829520 7f28f66f2700 5 -- op tracker -- seq: 3290, time: 2014-11-03 11:37:56.829396, event: header_read, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -768> 2014-11-03 11:37:56.829536 7f28f66f2700 5 -- op tracker -- seq: 3290, time: 2014-11-03 11:37:56.829397, event: throttled, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -767> 2014-11-03 11:37:56.829546 7f28f66f2700 5 -- op tracker -- seq: 3290, time: 2014-11-03 11:37:56.829467, event: all_read, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -766> 2014-11-03 11:37:56.829558 7f28f66f2700 5 -- op tracker -- seq: 3290, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -765> 2014-11-03 11:37:56.829810 7f28f4dd9700 1 -- 192.168.5.2:6803/7606 <== osd.82 192.168.5.6:6866/31491 20 ==== osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) v2 ==== 178+0+0 (1719355201 0 0) 0x29ce3600 con 0x22fe09a0 -764> 2014-11-03 11:37:56.829842 7f28f4dd9700 5 -- op tracker -- seq: 3291, time: 2014-11-03 11:37:56.829669, event: header_read, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -763> 2014-11-03 11:37:56.829898 7f28f4dd9700 5 -- op tracker -- seq: 3291, time: 2014-11-03 11:37:56.829670, event: throttled, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -762> 2014-11-03 11:37:56.829909 7f28f4dd9700 5 -- op tracker -- seq: 3291, time: 2014-11-03 11:37:56.829756, event: all_read, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -761> 2014-11-03 11:37:56.829923 7f28f4dd9700 5 -- op tracker -- seq: 3291, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -760> 2014-11-03 11:37:56.830131 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.830131, event: commit_queued_for_journal_write, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -759> 2014-11-03 11:37:56.830245 7f28ff019700 5 -- op tracker -- seq: 3290, time: 2014-11-03 11:37:56.830245, event: reached_pg, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -758> 2014-11-03 11:37:56.830266 7f28ff019700 5 -- op tracker -- seq: 3290, time: 2014-11-03 11:37:56.830266, event: started, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -757> 2014-11-03 11:37:56.830277 7f28ff019700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.830277, event: sub_op_commit_rec, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -756> 2014-11-03 11:37:56.830337 7f290181e700 5 -- op tracker -- seq: 3291, time: 2014-11-03 11:37:56.830337, event: reached_pg, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -755> 2014-11-03 11:37:56.830368 7f290181e700 5 -- op tracker -- seq: 3291, time: 2014-11-03 11:37:56.830368, event: started, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -754> 2014-11-03 11:37:56.830384 7f290181e700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.830384, event: sub_op_commit_rec, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -753> 2014-11-03 11:37:56.830406 7f290181e700 5 -- op tracker -- seq: 3291, time: 2014-11-03 11:37:56.830406, event: done, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -752> 2014-11-03 11:37:56.830427 7f290181e700 5 -- op tracker -- seq: 3290, time: 2014-11-03 11:37:56.830427, event: done, op: osd_sub_op_reply(client.7870167.0:988412 6.312 c6bfd312/rbd_data.580683128f4eda.0000000000000177/head//6 [] ondisk, result = 0) -751> 2014-11-03 11:37:56.833220 7f2914d0c700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.833219, event: write_thread_in_journal_buffer, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -750> 2014-11-03 11:37:56.833742 7f291450b700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.833741, event: journaled_completion_queued, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -749> 2014-11-03 11:37:56.833777 7f2911d06700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.833777, event: op_commit, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -748> 2014-11-03 11:37:56.833844 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.1.73:0/2009743 -- osd_op_reply(988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] v105950'7697950 uv7697950 ondisk = 0) v6 -- ?+0 0x23dfab80 con 0xee042c0 -747> 2014-11-03 11:37:56.833872 7f2911d06700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.833872, event: commit_sent, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -746> 2014-11-03 11:37:56.857125 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 214 ==== osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1834123098 0 3511520646) 0x2d3a9400 con 0x22fefdc0 -745> 2014-11-03 11:37:56.857187 7f28f36c2700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.856857, event: header_read, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -744> 2014-11-03 11:37:56.857206 7f28f36c2700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.856859, event: throttled, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -743> 2014-11-03 11:37:56.857237 7f28f36c2700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.857102, event: all_read, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -742> 2014-11-03 11:37:56.857258 7f28f36c2700 5 -- op tracker -- seq: 3292, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -741> 2014-11-03 11:37:56.857340 7f290101d700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.857339, event: reached_pg, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -740> 2014-11-03 11:37:56.857371 7f290101d700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.857371, event: started, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -739> 2014-11-03 11:37:56.857479 7f290101d700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.857478, event: started, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -738> 2014-11-03 11:37:56.857512 7f290101d700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.857512, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -737> 2014-11-03 11:37:56.857599 7f2914d0c700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.857599, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -736> 2014-11-03 11:37:56.857880 7f291450b700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.857880, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -735> 2014-11-03 11:37:56.858064 7f2911d06700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.858063, event: commit_sent, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -734> 2014-11-03 11:37:56.858162 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x27803dc0 con 0x22fefdc0 -733> 2014-11-03 11:37:56.864061 7f28ed05c700 1 -- 192.168.5.2:6803/7606 <== osd.49 192.168.5.4:6841/21769 53 ==== MOSDPGPush(6.9d8 105950 [PushOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, version: 105946'7949949, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (3716712066 0 0) 0x54a3c00 con 0x22fb7160 -732> 2014-11-03 11:37:56.864106 7f28ed05c700 5 -- op tracker -- seq: 3293, time: 2014-11-03 11:37:56.852607, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, version: 105946'7949949, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -731> 2014-11-03 11:37:56.864166 7f28ed05c700 5 -- op tracker -- seq: 3293, time: 2014-11-03 11:37:56.852609, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, version: 105946'7949949, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -730> 2014-11-03 11:37:56.864185 7f28ed05c700 5 -- op tracker -- seq: 3293, time: 2014-11-03 11:37:56.864047, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, version: 105946'7949949, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -729> 2014-11-03 11:37:56.864203 7f28ed05c700 5 -- op tracker -- seq: 3293, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, version: 105946'7949949, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -728> 2014-11-03 11:37:56.864281 7f28ff81a700 5 -- op tracker -- seq: 3293, time: 2014-11-03 11:37:56.864281, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, version: 105946'7949949, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -727> 2014-11-03 11:37:56.864513 7f28ff81a700 5 -- op tracker -- seq: 3293, time: 2014-11-03 11:37:56.864513, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6, version: 105946'7949949, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(71aa19d8/rbd_data.89cd895a911542.000000000000627c/head//6@105946'7949949, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -726> 2014-11-03 11:37:56.874950 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 53 ==== osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2631791911 0 3511520646) 0x2de2a000 con 0x2301fb00 -725> 2014-11-03 11:37:56.874987 7f28f31bd700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.874797, event: header_read, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -724> 2014-11-03 11:37:56.875001 7f28f31bd700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.874800, event: throttled, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -723> 2014-11-03 11:37:56.875010 7f28f31bd700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.874921, event: all_read, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -722> 2014-11-03 11:37:56.875018 7f28f31bd700 5 -- op tracker -- seq: 3294, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -721> 2014-11-03 11:37:56.875077 7f290201f700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.875077, event: reached_pg, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -720> 2014-11-03 11:37:56.875113 7f290201f700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.875113, event: started, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -719> 2014-11-03 11:37:56.875207 7f290201f700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.875207, event: started, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -718> 2014-11-03 11:37:56.875238 7f290201f700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.875238, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -717> 2014-11-03 11:37:56.876854 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 54 ==== osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1264974142 0 3511520646) 0x24661c00 con 0x2301fb00 -716> 2014-11-03 11:37:56.876919 7f28f31bd700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.876756, event: header_read, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -715> 2014-11-03 11:37:56.876935 7f28f31bd700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.876758, event: throttled, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -714> 2014-11-03 11:37:56.876944 7f28f31bd700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.876843, event: all_read, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -713> 2014-11-03 11:37:56.876953 7f28f31bd700 5 -- op tracker -- seq: 3295, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -712> 2014-11-03 11:37:56.877054 7f28ff81a700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.877054, event: reached_pg, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -711> 2014-11-03 11:37:56.877088 7f28ff81a700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.877088, event: started, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -710> 2014-11-03 11:37:56.877185 7f28ff81a700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.877185, event: started, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -709> 2014-11-03 11:37:56.877221 7f28ff81a700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.877221, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -708> 2014-11-03 11:37:56.878717 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 107 ==== osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3246794416 0 3511520646) 0x2d3a8800 con 0x22fbcb00 -707> 2014-11-03 11:37:56.878770 7f28f6cf8700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.878562, event: header_read, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -706> 2014-11-03 11:37:56.878793 7f28f6cf8700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.878565, event: throttled, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -705> 2014-11-03 11:37:56.878807 7f28f6cf8700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.878693, event: all_read, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -704> 2014-11-03 11:37:56.878821 7f28f6cf8700 5 -- op tracker -- seq: 3296, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -703> 2014-11-03 11:37:56.878926 7f290081c700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.878925, event: reached_pg, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -702> 2014-11-03 11:37:56.878953 7f290081c700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.878953, event: started, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -701> 2014-11-03 11:37:56.879042 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 55 ==== osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3906726637 0 3511520646) 0xd55dc00 con 0x2301fb00 -700> 2014-11-03 11:37:56.879065 7f290081c700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.879064, event: started, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -699> 2014-11-03 11:37:56.879076 7f28f31bd700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.878902, event: header_read, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -698> 2014-11-03 11:37:56.879094 7f28f31bd700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.878904, event: throttled, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -697> 2014-11-03 11:37:56.879103 7f290081c700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.879103, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -696> 2014-11-03 11:37:56.879111 7f28f31bd700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.879029, event: all_read, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -695> 2014-11-03 11:37:56.879147 7f28f31bd700 5 -- op tracker -- seq: 3297, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -694> 2014-11-03 11:37:56.879203 7f290201f700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.879203, event: reached_pg, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -693> 2014-11-03 11:37:56.879231 7f290201f700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.879231, event: started, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -692> 2014-11-03 11:37:56.879313 7f290201f700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.879313, event: started, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -691> 2014-11-03 11:37:56.879338 7f290201f700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.879338, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -690> 2014-11-03 11:37:56.879953 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 108 ==== osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1606948684 0 3511520646) 0x23fb0400 con 0x22fbcb00 -689> 2014-11-03 11:37:56.880003 7f28f6cf8700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.879819, event: header_read, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -688> 2014-11-03 11:37:56.880024 7f28f6cf8700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.879820, event: throttled, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -687> 2014-11-03 11:37:56.880042 7f28f6cf8700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.879940, event: all_read, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -686> 2014-11-03 11:37:56.880060 7f28f6cf8700 5 -- op tracker -- seq: 3298, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -685> 2014-11-03 11:37:56.880131 7f28fe017700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.880131, event: reached_pg, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -684> 2014-11-03 11:37:56.880155 7f28fe017700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.880155, event: started, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -683> 2014-11-03 11:37:56.880177 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.111 192.168.5.8:0/32649 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.879025) v2 ==== 47+0+0 (4135031716 0 0) 0xde23800 con 0x23b1c000 -682> 2014-11-03 11:37:56.880194 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.111 192.168.5.8:0/32649 13 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.879025) v2 ==== 47+0+0 (4135031716 0 0) 0x30dba00 con 0x24229c60 -681> 2014-11-03 11:37:56.880219 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.879025) v2 -- ?+0 0x1a22ce00 con 0x23b1c000 -680> 2014-11-03 11:37:56.880255 7f28fe017700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.880255, event: started, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -679> 2014-11-03 11:37:56.880269 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/32649 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.879025) v2 -- ?+0 0x405b800 con 0x24229c60 -678> 2014-11-03 11:37:56.880299 7f28fe017700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.880298, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -677> 2014-11-03 11:37:56.881525 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 109 ==== osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2564107693 0 3511520646) 0x23fae000 con 0x22fbcb00 -676> 2014-11-03 11:37:56.881563 7f28f6cf8700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.881415, event: header_read, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -675> 2014-11-03 11:37:56.881580 7f28f6cf8700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.881417, event: throttled, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -674> 2014-11-03 11:37:56.881592 7f28f6cf8700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.881511, event: all_read, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -673> 2014-11-03 11:37:56.881604 7f28f6cf8700 5 -- op tracker -- seq: 3299, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -672> 2014-11-03 11:37:56.881656 7f290081c700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.881656, event: reached_pg, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -671> 2014-11-03 11:37:56.881680 7f290081c700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.881680, event: started, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -670> 2014-11-03 11:37:56.881766 7f290081c700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.881766, event: started, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -669> 2014-11-03 11:37:56.881794 7f290081c700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.881794, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -668> 2014-11-03 11:37:56.894584 7f28c4694700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x23de8b80 sd=566 :6802 s=0 pgs=0 cs=0 l=0 c=0x2439c420).accept sd=566 192.168.5.201:58714/0 -667> 2014-11-03 11:37:56.895452 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12420006 192.168.5.201:0/1030288 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x2a8d7e00 con 0x2439c420 -666> 2014-11-03 11:37:56.895885 7f2914d0c700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.895885, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -665> 2014-11-03 11:37:56.895925 7f2914d0c700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.895925, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -664> 2014-11-03 11:37:56.895951 7f28fb812700 0 osd.21 105950 do_command r=0 -663> 2014-11-03 11:37:56.895967 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1030288 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x234168c0 con 0x2439c420 -662> 2014-11-03 11:37:56.895967 7f2914d0c700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.895967, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -661> 2014-11-03 11:37:56.896008 7f2914d0c700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.896008, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -660> 2014-11-03 11:37:56.896038 7f2914d0c700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.896038, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -659> 2014-11-03 11:37:56.896063 7f2914d0c700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.896063, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -658> 2014-11-03 11:37:56.896370 7f291450b700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.896370, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -657> 2014-11-03 11:37:56.896405 7f291450b700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.896405, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -656> 2014-11-03 11:37:56.896409 7f2911d06700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.896409, event: commit_sent, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -655> 2014-11-03 11:37:56.896423 7f291450b700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.896423, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -654> 2014-11-03 11:37:56.896438 7f291450b700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.896438, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -653> 2014-11-03 11:37:56.896445 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] ondisk, result = 0) v2 -- ?+0 0x23bd8000 con 0x2301fb00 -652> 2014-11-03 11:37:56.896455 7f291450b700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.896455, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -651> 2014-11-03 11:37:56.896469 7f291450b700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.896469, event: journaled_completion_queued, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -650> 2014-11-03 11:37:56.896489 7f2911d06700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.896489, event: commit_sent, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -649> 2014-11-03 11:37:56.896566 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294ae340 con 0x2301fb00 -648> 2014-11-03 11:37:56.896595 7f2911d06700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.896595, event: commit_sent, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -647> 2014-11-03 11:37:56.896617 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294adb00 con 0x22fbcb00 -646> 2014-11-03 11:37:56.896644 7f2911d06700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.896644, event: commit_sent, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -645> 2014-11-03 11:37:56.896661 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] ondisk, result = 0) v2 -- ?+0 0x294aeb80 con 0x2301fb00 -644> 2014-11-03 11:37:56.896680 7f2911d06700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.896680, event: commit_sent, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -643> 2014-11-03 11:37:56.896757 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x25591840 con 0x22fbcb00 -642> 2014-11-03 11:37:56.896786 7f2911d06700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.896786, event: commit_sent, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -641> 2014-11-03 11:37:56.896806 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x25592080 con 0x22fbcb00 -640> 2014-11-03 11:37:56.900713 7f2912507700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.900713, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -639> 2014-11-03 11:37:56.900795 7f2912507700 5 -- op tracker -- seq: 3292, time: 2014-11-03 11:37:56.900795, event: done, op: osd_sub_op(client.8703713.0:32254535 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953162 snapset=0=[]:[] snapc=0=[]) -638> 2014-11-03 11:37:56.900893 7f2912507700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.900893, event: op_applied, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -637> 2014-11-03 11:37:56.900966 7f2912507700 5 -- op tracker -- seq: 3276, time: 2014-11-03 11:37:56.900966, event: done, op: osd_op(client.7870167.0:988412 rbd_data.580683128f4eda.0000000000000177 [set-alloc-hint object_size 4194304 write_size 4194304,write 1114112~4096] 6.c6bfd312 ack+ondisk+write e105950) -636> 2014-11-03 11:37:56.901108 7f2912507700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.901107, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -635> 2014-11-03 11:37:56.901170 7f2912507700 5 -- op tracker -- seq: 3294, time: 2014-11-03 11:37:56.901170, event: done, op: osd_sub_op(client.7869116.0:21661223 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029772 snapset=0=[]:[] snapc=0=[]) -634> 2014-11-03 11:37:56.901341 7f2912507700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.901341, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -633> 2014-11-03 11:37:56.901390 7f2912507700 5 -- op tracker -- seq: 3295, time: 2014-11-03 11:37:56.901390, event: done, op: osd_sub_op(client.7869116.0:21661224 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029773 snapset=0=[]:[] snapc=0=[]) -632> 2014-11-03 11:37:56.901551 7f2912507700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.901551, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -631> 2014-11-03 11:37:56.901596 7f2912507700 5 -- op tracker -- seq: 3296, time: 2014-11-03 11:37:56.901596, event: done, op: osd_sub_op(client.7869116.0:21661240 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262142 snapset=0=[]:[] snapc=0=[]) -630> 2014-11-03 11:37:56.901698 7f2912507700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.901697, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -629> 2014-11-03 11:37:56.901742 7f2912507700 5 -- op tracker -- seq: 3297, time: 2014-11-03 11:37:56.901742, event: done, op: osd_sub_op(client.7869116.0:21661225 6.99c 3887199c/rbd_data.4fd1a75007e195.0000000000000085/head//6 [] v 105950'8029774 snapset=0=[]:[] snapc=0=[]) -628> 2014-11-03 11:37:56.901851 7f2912507700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.901851, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -627> 2014-11-03 11:37:56.901897 7f2912507700 5 -- op tracker -- seq: 3298, time: 2014-11-03 11:37:56.901896, event: done, op: osd_sub_op(client.7869116.0:21661241 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262143 snapset=0=[]:[] snapc=0=[]) -626> 2014-11-03 11:37:56.901993 7f2912507700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.901993, event: sub_op_applied, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -625> 2014-11-03 11:37:56.902043 7f2912507700 5 -- op tracker -- seq: 3299, time: 2014-11-03 11:37:56.902043, event: done, op: osd_sub_op(client.7869116.0:21661242 6.576 b7260576/rbd_data.4fd1a75007e195.00000000000000d6/head//6 [] v 105950'17262144 snapset=0=[]:[] snapc=0=[]) -624> 2014-11-03 11:37:56.906976 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x17ae8400 con 0x22fe0420 -623> 2014-11-03 11:37:56.908986 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12420006 192.168.5.201:0/1030288 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x23a9e600 con 0x2439c420 -622> 2014-11-03 11:37:56.909810 7f28fb812700 0 osd.21 105950 do_command r=0 -621> 2014-11-03 11:37:56.909827 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1030288 -- command_reply(tid 2: 0 ) v1 -- ?+16697 0x2dcb0700 con 0x2439c420 -620> 2014-11-03 11:37:56.916124 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.68 192.168.5.5:0/8133 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.914282) v2 ==== 47+0+0 (4071977106 0 0) 0x23a92200 con 0x24292420 -619> 2014-11-03 11:37:56.916166 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.68 192.168.5.5:0/8133 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:56.914282) v2 ==== 47+0+0 (4071977106 0 0) 0x236a8000 con 0x24321420 -618> 2014-11-03 11:37:56.916181 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/8133 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.914282) v2 -- ?+0 0xde23800 con 0x24292420 -617> 2014-11-03 11:37:56.916291 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/8133 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:56.914282) v2 -- ?+0 0x30dba00 con 0x24321420 -616> 2014-11-03 11:37:56.916696 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1030288 pipe(0x23de8b80 sd=566 :6802 s=2 pgs=2 cs=1 l=1 c=0x2439c420).reader couldn't read tag, (0) Success -615> 2014-11-03 11:37:56.916735 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1030288 pipe(0x23de8b80 sd=566 :6802 s=2 pgs=2 cs=1 l=1 c=0x2439c420).fault (0) Success -614> 2014-11-03 11:37:56.916799 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2439c420 session 0x32fbc80 -613> 2014-11-03 11:37:56.920942 7f28c4694700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x28229600 sd=539 :6802 s=0 pgs=0 cs=0 l=0 c=0x2439cb00).accept sd=539 192.168.5.2:43012/0 -612> 2014-11-03 11:37:56.921643 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450946 192.168.5.2:0/1005382 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x6526800 con 0x2439cb00 -611> 2014-11-03 11:37:56.922082 7f28fb812700 0 osd.21 105950 do_command r=0 -610> 2014-11-03 11:37:56.922100 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1005382 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x9958380 con 0x2439cb00 -609> 2014-11-03 11:37:56.932147 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450946 192.168.5.2:0/1005382 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x243ff200 con 0x2439cb00 -608> 2014-11-03 11:37:56.933294 7f28fb812700 0 osd.21 105950 do_command r=0 -607> 2014-11-03 11:37:56.933358 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1005382 -- command_reply(tid 2: 0 ) v1 -- ?+16697 0x9958000 con 0x2439cb00 -606> 2014-11-03 11:37:56.938722 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1005382 pipe(0x28229600 sd=539 :6802 s=2 pgs=2 cs=1 l=1 c=0x2439cb00).reader couldn't read tag, (0) Success -605> 2014-11-03 11:37:56.938775 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1005382 pipe(0x28229600 sd=539 :6802 s=2 pgs=2 cs=1 l=1 c=0x2439cb00).fault (0) Success -604> 2014-11-03 11:37:56.938905 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2439cb00 session 0x32fbc80 -603> 2014-11-03 11:37:56.961388 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 46 ==== MOSDPGPush(6.9d8 105950 [PushOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, version: 105946'7949971, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195207+0+0 (4195947811 0 0) 0x17ae8400 con 0x22fe0420 -602> 2014-11-03 11:37:56.961447 7f28f5be7700 5 -- op tracker -- seq: 3300, time: 2014-11-03 11:37:56.951109, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, version: 105946'7949971, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -601> 2014-11-03 11:37:56.961474 7f28f5be7700 5 -- op tracker -- seq: 3300, time: 2014-11-03 11:37:56.951113, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, version: 105946'7949971, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -600> 2014-11-03 11:37:56.961493 7f28f5be7700 5 -- op tracker -- seq: 3300, time: 2014-11-03 11:37:56.961373, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, version: 105946'7949971, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -599> 2014-11-03 11:37:56.961510 7f28f5be7700 5 -- op tracker -- seq: 3300, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, version: 105946'7949971, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -598> 2014-11-03 11:37:56.961573 7f28ff81a700 5 -- op tracker -- seq: 3300, time: 2014-11-03 11:37:56.961573, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, version: 105946'7949971, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -597> 2014-11-03 11:37:56.961792 7f28ff81a700 5 -- op tracker -- seq: 3300, time: 2014-11-03 11:37:56.961791, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6, version: 105946'7949971, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4fcb09d8/rbd_data.885435b2bbeeb.0000000000003ed1/head//6@105946'7949971, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -596> 2014-11-03 11:37:56.974712 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- MOSDPGPull(6.9d8 105950 [PullOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x252d7200 con 0x22fb7160 -595> 2014-11-03 11:37:56.991084 7f28ed05c700 1 -- 192.168.5.2:6803/7606 <== osd.49 192.168.5.4:6841/21769 54 ==== MOSDPGPush(6.9d8 105950 [PushOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, version: 105946'7950003, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (626173063 0 0) 0x252d7200 con 0x22fb7160 -594> 2014-11-03 11:37:56.991129 7f28ed05c700 5 -- op tracker -- seq: 3301, time: 2014-11-03 11:37:56.979556, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, version: 105946'7950003, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -593> 2014-11-03 11:37:56.991149 7f28ed05c700 5 -- op tracker -- seq: 3301, time: 2014-11-03 11:37:56.979559, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, version: 105946'7950003, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -592> 2014-11-03 11:37:56.991162 7f28ed05c700 5 -- op tracker -- seq: 3301, time: 2014-11-03 11:37:56.991071, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, version: 105946'7950003, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -591> 2014-11-03 11:37:56.991175 7f28ed05c700 5 -- op tracker -- seq: 3301, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, version: 105946'7950003, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -590> 2014-11-03 11:37:56.991290 7f290201f700 5 -- op tracker -- seq: 3301, time: 2014-11-03 11:37:56.991290, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, version: 105946'7950003, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -589> 2014-11-03 11:37:56.991500 7f290201f700 5 -- op tracker -- seq: 3301, time: 2014-11-03 11:37:56.991500, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6, version: 105946'7950003, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(23daf9d8/rbd_data.7986317d92ecf2.000000000000575e/head//6@105946'7950003, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -588> 2014-11-03 11:37:57.001568 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x252d7400 con 0x22fe0420 -587> 2014-11-03 11:37:57.002957 7f28c4694700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x28228b00 sd=35 :6802 s=0 pgs=0 cs=0 l=0 c=0x2439cc60).accept sd=35 192.168.5.201:58717/0 -586> 2014-11-03 11:37:57.003812 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450848 192.168.5.201:0/1030333 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "24.7d7"}) v1 ==== 80+0+0 (3756934197 0 0) 0x17cac800 con 0x2439cc60 -585> 2014-11-03 11:37:57.004371 7f28fb812700 0 osd.21 105950 do_command r=0 -584> 2014-11-03 11:37:57.004387 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1030333 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x234168c0 con 0x2439cc60 -583> 2014-11-03 11:37:57.013640 7f28c2fc7700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2ea7ab00 sd=566 :6802 s=0 pgs=0 cs=0 l=0 c=0xa2aa2c0).accept sd=566 192.168.5.201:58719/0 -582> 2014-11-03 11:37:57.014556 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450851 192.168.5.201:0/1030338 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.58e"}) v1 ==== 79+0+0 (3567090102 0 0) 0x8714000 con 0xa2aa2c0 -581> 2014-11-03 11:37:57.015040 7f28fb812700 0 osd.21 105950 do_command r=0 -580> 2014-11-03 11:37:57.015064 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1030338 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x3243540 con 0xa2aa2c0 -579> 2014-11-03 11:37:57.015348 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450848 192.168.5.201:0/1030333 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "24.7d7"}) v1 ==== 74+0+0 (774687807 0 0) 0x30c1400 con 0x2439cc60 -578> 2014-11-03 11:37:57.016021 7f28fb812700 0 osd.21 105950 do_command r=0 -577> 2014-11-03 11:37:57.016042 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1030333 -- command_reply(tid 2: 0 ) v1 -- ?+13413 0x968f000 con 0x2439cc60 -576> 2014-11-03 11:37:57.021046 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1030333 pipe(0x28228b00 sd=35 :6802 s=2 pgs=2 cs=1 l=1 c=0x2439cc60).reader couldn't read tag, (0) Success -575> 2014-11-03 11:37:57.021113 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1030333 pipe(0x28228b00 sd=35 :6802 s=2 pgs=2 cs=1 l=1 c=0x2439cc60).fault (0) Success -574> 2014-11-03 11:37:57.021211 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x2439cc60 session 0x32fbc80 -573> 2014-11-03 11:37:57.021500 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 53 ==== MOSDPGPush(24.1c1 105950 [PushOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24, version: 105945'139876, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24@105945'139876, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24, version: 105946'139897, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24@105946'139897, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (3752614615 0 0) 0x3a32e00 con 0x22fe0000 -572> 2014-11-03 11:37:57.021555 7f28f4fdb700 5 -- op tracker -- seq: 3302, time: 2014-11-03 11:37:56.980814, event: header_read, op: MOSDPGPush(24.1c1 105950 [PushOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24, version: 105945'139876, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24@105945'139876, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24, version: 105946'139897, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24@105946'139897, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -571> 2014-11-03 11:37:57.021578 7f28f4fdb700 5 -- op tracker -- seq: 3302, time: 2014-11-03 11:37:56.980817, event: throttled, op: MOSDPGPush(24.1c1 105950 [PushOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24, version: 105945'139876, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24@105945'139876, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24, version: 105946'139897, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24@105946'139897, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -570> 2014-11-03 11:37:57.021613 7f28f4fdb700 5 -- op tracker -- seq: 3302, time: 2014-11-03 11:37:57.021488, event: all_read, op: MOSDPGPush(24.1c1 105950 [PushOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24, version: 105945'139876, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24@105945'139876, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24, version: 105946'139897, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24@105946'139897, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -569> 2014-11-03 11:37:57.021635 7f28f4fdb700 5 -- op tracker -- seq: 3302, time: 0.000000, event: dispatched, op: MOSDPGPush(24.1c1 105950 [PushOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24, version: 105945'139876, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24@105945'139876, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24, version: 105946'139897, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24@105946'139897, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -568> 2014-11-03 11:37:57.021706 7f290001b700 5 -- op tracker -- seq: 3302, time: 2014-11-03 11:37:57.021705, event: reached_pg, op: MOSDPGPush(24.1c1 105950 [PushOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24, version: 105945'139876, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24@105945'139876, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24, version: 105946'139897, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24@105946'139897, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -567> 2014-11-03 11:37:57.022006 7f290001b700 5 -- op tracker -- seq: 3302, time: 2014-11-03 11:37:57.022005, event: done, op: MOSDPGPush(24.1c1 105950 [PushOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24, version: 105945'139876, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24@105945'139876, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24, version: 105946'139897, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24@105946'139897, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -566> 2014-11-03 11:37:57.025652 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450851 192.168.5.201:0/1030338 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.58e"}) v1 ==== 73+0+0 (125580214 0 0) 0x23fdf600 con 0xa2aa2c0 -565> 2014-11-03 11:37:57.026596 7f28fb812700 0 osd.21 105950 do_command r=0 -564> 2014-11-03 11:37:57.026621 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.201:0/1030338 -- command_reply(tid 2: 0 ) v1 -- ?+14243 0x2dcb0700 con 0xa2aa2c0 -563> 2014-11-03 11:37:57.032463 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1030338 pipe(0x2ea7ab00 sd=566 :6802 s=2 pgs=2 cs=1 l=1 c=0xa2aa2c0).reader couldn't read tag, (0) Success -562> 2014-11-03 11:37:57.032530 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.201:0/1030338 pipe(0x2ea7ab00 sd=566 :6802 s=2 pgs=2 cs=1 l=1 c=0xa2aa2c0).fault (0) Success -561> 2014-11-03 11:37:57.032611 7f290c834700 1 osd.21 105950 ms_handle_reset con 0xa2aa2c0 session 0x2348c000 -560> 2014-11-03 11:37:57.039360 7f28f4fdb700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 54 ==== MOSDPGPush(24.1c1 105950 [PushOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24, version: 105946'139902, data_included: [1048576~3145728], data_size: 3145728, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24@105946'139902, copy_subset: [1048576~3145728], clone_subset: {70a501c1/rbd_data.5348922ae8944a.0000000000000989/301//24=[0~1048576]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 3146872+0+0 (314273069 0 0) 0x29fcd000 con 0x22fe0000 -559> 2014-11-03 11:37:57.039409 7f28f4fdb700 5 -- op tracker -- seq: 3303, time: 2014-11-03 11:37:57.021698, event: header_read, op: MOSDPGPush(24.1c1 105950 [PushOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24, version: 105946'139902, data_included: [1048576~3145728], data_size: 3145728, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24@105946'139902, copy_subset: [1048576~3145728], clone_subset: {70a501c1/rbd_data.5348922ae8944a.0000000000000989/301//24=[0~1048576]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -558> 2014-11-03 11:37:57.039438 7f28f4fdb700 5 -- op tracker -- seq: 3303, time: 2014-11-03 11:37:57.021699, event: throttled, op: MOSDPGPush(24.1c1 105950 [PushOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24, version: 105946'139902, data_included: [1048576~3145728], data_size: 3145728, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24@105946'139902, copy_subset: [1048576~3145728], clone_subset: {70a501c1/rbd_data.5348922ae8944a.0000000000000989/301//24=[0~1048576]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -557> 2014-11-03 11:37:57.039463 7f28f4fdb700 5 -- op tracker -- seq: 3303, time: 2014-11-03 11:37:57.039349, event: all_read, op: MOSDPGPush(24.1c1 105950 [PushOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24, version: 105946'139902, data_included: [1048576~3145728], data_size: 3145728, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24@105946'139902, copy_subset: [1048576~3145728], clone_subset: {70a501c1/rbd_data.5348922ae8944a.0000000000000989/301//24=[0~1048576]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -556> 2014-11-03 11:37:57.039477 7f28f4fdb700 5 -- op tracker -- seq: 3303, time: 0.000000, event: dispatched, op: MOSDPGPush(24.1c1 105950 [PushOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24, version: 105946'139902, data_included: [1048576~3145728], data_size: 3145728, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24@105946'139902, copy_subset: [1048576~3145728], clone_subset: {70a501c1/rbd_data.5348922ae8944a.0000000000000989/301//24=[0~1048576]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -555> 2014-11-03 11:37:57.039556 7f28fd816700 5 -- op tracker -- seq: 3303, time: 2014-11-03 11:37:57.039556, event: reached_pg, op: MOSDPGPush(24.1c1 105950 [PushOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24, version: 105946'139902, data_included: [1048576~3145728], data_size: 3145728, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24@105946'139902, copy_subset: [1048576~3145728], clone_subset: {70a501c1/rbd_data.5348922ae8944a.0000000000000989/301//24=[0~1048576]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -554> 2014-11-03 11:37:57.039772 7f28fd816700 5 -- op tracker -- seq: 3303, time: 2014-11-03 11:37:57.039772, event: done, op: MOSDPGPush(24.1c1 105950 [PushOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24, version: 105946'139902, data_included: [1048576~3145728], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24@105946'139902, copy_subset: [1048576~3145728], clone_subset: {70a501c1/rbd_data.5348922ae8944a.0000000000000989/301//24=[0~1048576]}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -553> 2014-11-03 11:37:57.056172 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.108 192.168.5.8:0/21567 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.055095) v2 ==== 47+0+0 (4136295957 0 0) 0xa5d2600 con 0x2484a840 -552> 2014-11-03 11:37:57.056222 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.108 192.168.5.8:0/21567 12 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.055095) v2 ==== 47+0+0 (4136295957 0 0) 0x286bb400 con 0x243212c0 -551> 2014-11-03 11:37:57.056243 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.055095) v2 -- ?+0 0x23a92200 con 0x2484a840 -550> 2014-11-03 11:37:57.056323 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/21567 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.055095) v2 -- ?+0 0x236a8000 con 0x243212c0 -549> 2014-11-03 11:37:57.080809 7f28c2fc7700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2494b840 sd=63 :6802 s=0 pgs=0 cs=0 l=0 c=0x6364160).accept sd=63 192.168.5.2:43018/0 -548> 2014-11-03 11:37:57.081722 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450949 192.168.5.2:0/1005578 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x6425600 con 0x6364160 -547> 2014-11-03 11:37:57.082463 7f28fb812700 0 osd.21 105950 do_command r=0 -546> 2014-11-03 11:37:57.082493 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1005578 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x9958380 con 0x6364160 -545> 2014-11-03 11:37:57.099014 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450949 192.168.5.2:0/1005578 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2be31c00 con 0x6364160 -544> 2014-11-03 11:37:57.099960 7f28fb812700 0 osd.21 105950 do_command r=0 -543> 2014-11-03 11:37:57.099980 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1005578 -- command_reply(tid 2: 0 ) v1 -- ?+16699 0x234168c0 con 0x6364160 -542> 2014-11-03 11:37:57.107370 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1005578 pipe(0x2494b840 sd=63 :6802 s=2 pgs=2 cs=1 l=1 c=0x6364160).reader couldn't read tag, (0) Success -541> 2014-11-03 11:37:57.107416 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1005578 pipe(0x2494b840 sd=63 :6802 s=2 pgs=2 cs=1 l=1 c=0x6364160).fault (0) Success -540> 2014-11-03 11:37:57.107514 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x6364160 session 0x2348c000 -539> 2014-11-03 11:37:57.107537 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 35 ==== osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (131666292 0 3511520646) 0x2fe0c000 con 0x22fe0840 -538> 2014-11-03 11:37:57.107570 7f28f4cd8700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.107438, event: header_read, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -537> 2014-11-03 11:37:57.107595 7f28f4cd8700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.107441, event: throttled, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -536> 2014-11-03 11:37:57.107615 7f28f4cd8700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.107521, event: all_read, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -535> 2014-11-03 11:37:57.107632 7f28f4cd8700 5 -- op tracker -- seq: 3304, time: 0.000000, event: dispatched, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -534> 2014-11-03 11:37:57.107692 7f28fe017700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.107692, event: reached_pg, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -533> 2014-11-03 11:37:57.107721 7f28fe017700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.107721, event: started, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -532> 2014-11-03 11:37:57.107815 7f28fe017700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.107815, event: started, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -531> 2014-11-03 11:37:57.107847 7f28fe017700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.107847, event: commit_queued_for_journal_write, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -530> 2014-11-03 11:37:57.110844 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 36 ==== osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) v11 ==== 1398+0+39 (787367646 0 3511520646) 0x2fe0cc00 con 0x22fe0840 -529> 2014-11-03 11:37:57.110909 7f28f4cd8700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.110690, event: header_read, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -528> 2014-11-03 11:37:57.110940 7f28f4cd8700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.110694, event: throttled, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -527> 2014-11-03 11:37:57.110953 7f28f4cd8700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.110826, event: all_read, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -526> 2014-11-03 11:37:57.110968 7f28f4cd8700 5 -- op tracker -- seq: 3305, time: 0.000000, event: dispatched, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -525> 2014-11-03 11:37:57.111056 7f290081c700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.111056, event: reached_pg, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -524> 2014-11-03 11:37:57.111094 7f290081c700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.111094, event: started, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -523> 2014-11-03 11:37:57.111206 7f290081c700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.111206, event: started, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -522> 2014-11-03 11:37:57.111241 7f290081c700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.111241, event: commit_queued_for_journal_write, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -521> 2014-11-03 11:37:57.112592 7f2914d0c700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.112591, event: write_thread_in_journal_buffer, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -520> 2014-11-03 11:37:57.112621 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- pg_trim(24.1c1 to 105946'139902 e105950) v1 -- ?+0 0x26a9fe00 con 0x22fe0000 -519> 2014-11-03 11:37:57.112680 7f2914d0c700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.112679, event: write_thread_in_journal_buffer, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -518> 2014-11-03 11:37:57.123334 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 47 ==== MOSDPGPush(6.9d8 105950 [PushOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, version: 105946'7950012, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (297279715 0 0) 0x252d7400 con 0x22fe0420 -517> 2014-11-03 11:37:57.123401 7f28f5be7700 5 -- op tracker -- seq: 3306, time: 2014-11-03 11:37:57.113145, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, version: 105946'7950012, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -516> 2014-11-03 11:37:57.123434 7f28f5be7700 5 -- op tracker -- seq: 3306, time: 2014-11-03 11:37:57.113148, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, version: 105946'7950012, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -515> 2014-11-03 11:37:57.123461 7f28f5be7700 5 -- op tracker -- seq: 3306, time: 2014-11-03 11:37:57.123307, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, version: 105946'7950012, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -514> 2014-11-03 11:37:57.123478 7f28f5be7700 5 -- op tracker -- seq: 3306, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, version: 105946'7950012, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -513> 2014-11-03 11:37:57.123590 7f28ff81a700 5 -- op tracker -- seq: 3306, time: 2014-11-03 11:37:57.123590, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, version: 105946'7950012, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -512> 2014-11-03 11:37:57.123802 7f28ff81a700 5 -- op tracker -- seq: 3306, time: 2014-11-03 11:37:57.123802, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6, version: 105946'7950012, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1d2f89d8/rbd_data.30d52d421a870a.0000000000002e08/head//6@105946'7950012, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -511> 2014-11-03 11:37:57.141830 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.65 192.168.5.5:0/30042 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.139953) v2 ==== 47+0+0 (970199406 0 0) 0x27150800 con 0x24229420 -510> 2014-11-03 11:37:57.141922 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/30042 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.139953) v2 -- ?+0 0x286bb400 con 0x24229420 -509> 2014-11-03 11:37:57.175349 7f291450b700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.175348, event: journaled_completion_queued, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -508> 2014-11-03 11:37:57.175399 7f291450b700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.175399, event: journaled_completion_queued, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -507> 2014-11-03 11:37:57.175430 7f2911d06700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.175430, event: commit_sent, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -506> 2014-11-03 11:37:57.175494 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x25591dc0 con 0x22fe0840 -505> 2014-11-03 11:37:57.175588 7f2911d06700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.175588, event: commit_sent, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -504> 2014-11-03 11:37:57.175596 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- MOSDPGPushReply(24.1c1 105950 [PushReplyOp(f309d1c1/rbd_data.5b64062ae8944a.0000000000000088/head//24),PushReplyOp(b1d131c1/rbd_data.534b342ae8944a.00000000000003c1/head//24)]) v2 -- ?+0 0x259f3600 con 0x22fe0000 -503> 2014-11-03 11:37:57.175720 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op_reply(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] ondisk, result = 0) v2 -- ?+0 0x25592600 con 0x22fe0840 -502> 2014-11-03 11:37:57.176318 7f2912507700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.176318, event: sub_op_applied, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -501> 2014-11-03 11:37:57.176402 7f2912507700 5 -- op tracker -- seq: 3304, time: 2014-11-03 11:37:57.176402, event: done, op: osd_sub_op(client.10192289.0:1727273 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960085 snapset=0=[]:[] snapc=0=[]) -500> 2014-11-03 11:37:57.176673 7f2912507700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.176672, event: sub_op_applied, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -499> 2014-11-03 11:37:57.176740 7f2912507700 5 -- op tracker -- seq: 3305, time: 2014-11-03 11:37:57.176740, event: done, op: osd_sub_op(client.10192289.0:1727275 6.35a 9d74db5a/rbd_data.4d69cfb1c8798.0000000000000f6c/head//6 [] v 105950'14960086 snapset=0=[]:[] snapc=0=[]) -498> 2014-11-03 11:37:57.183988 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 78 ==== osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2768435914 0 3511520646) 0x27148c00 con 0x22fbc840 -497> 2014-11-03 11:37:57.184026 7f28f68f4700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.183889, event: header_read, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -496> 2014-11-03 11:37:57.184040 7f28f68f4700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.183891, event: throttled, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -495> 2014-11-03 11:37:57.184049 7f28f68f4700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.183976, event: all_read, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -494> 2014-11-03 11:37:57.184057 7f28f68f4700 5 -- op tracker -- seq: 3307, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -493> 2014-11-03 11:37:57.184119 7f28fe017700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.184119, event: reached_pg, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -492> 2014-11-03 11:37:57.184155 7f28fe017700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.184155, event: started, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -491> 2014-11-03 11:37:57.184265 7f28fe017700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.184265, event: started, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -490> 2014-11-03 11:37:57.184299 7f28fe017700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.184299, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -489> 2014-11-03 11:37:57.185743 7f28f68f4700 1 -- 192.168.5.2:6803/7606 <== osd.75 192.168.5.6:6828/12612 79 ==== osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (351738236 0 3511520646) 0x27148000 con 0x22fbc840 -488> 2014-11-03 11:37:57.185780 7f28f68f4700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.185628, event: header_read, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -487> 2014-11-03 11:37:57.185802 7f28f68f4700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.185630, event: throttled, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -486> 2014-11-03 11:37:57.185820 7f28f68f4700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.185731, event: all_read, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -485> 2014-11-03 11:37:57.185837 7f28f68f4700 5 -- op tracker -- seq: 3308, time: 0.000000, event: dispatched, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -484> 2014-11-03 11:37:57.185904 7f290081c700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.185904, event: reached_pg, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -483> 2014-11-03 11:37:57.185952 7f290081c700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.185952, event: started, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -482> 2014-11-03 11:37:57.186053 7f290081c700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.186053, event: started, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -481> 2014-11-03 11:37:57.186088 7f290081c700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.186088, event: commit_queued_for_journal_write, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -480> 2014-11-03 11:37:57.221827 7f2918d14700 5 osd.21 105950 tick -479> 2014-11-03 11:37:57.232762 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 110 ==== osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3511413365 0 3511520646) 0x26942000 con 0x22fbcb00 -478> 2014-11-03 11:37:57.232814 7f28f6cf8700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.232571, event: header_read, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -477> 2014-11-03 11:37:57.232841 7f28f6cf8700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.232574, event: throttled, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -476> 2014-11-03 11:37:57.232855 7f28f6cf8700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.232743, event: all_read, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -475> 2014-11-03 11:37:57.232867 7f28f6cf8700 5 -- op tracker -- seq: 3309, time: 0.000000, event: dispatched, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -474> 2014-11-03 11:37:57.232942 7f28fe017700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.232942, event: reached_pg, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -473> 2014-11-03 11:37:57.232972 7f28fe017700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.232972, event: started, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -472> 2014-11-03 11:37:57.233074 7f28fe017700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.233073, event: started, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -471> 2014-11-03 11:37:57.233105 7f28fe017700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.233105, event: commit_queued_for_journal_write, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -470> 2014-11-03 11:37:57.242681 7f28c2fc7700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x2b2e0dc0 sd=95 :6802 s=0 pgs=0 cs=0 l=0 c=0x63649a0).accept sd=95 192.168.5.2:43024/0 -469> 2014-11-03 11:37:57.243047 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.80 192.168.5.6:0/21329 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.241664) v2 ==== 47+0+0 (1070730492 0 0) 0x1a327200 con 0x249252c0 -468> 2014-11-03 11:37:57.243081 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.80 192.168.5.6:0/21329 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.241664) v2 ==== 47+0+0 (1070730492 0 0) 0x91ae400 con 0x24229000 -467> 2014-11-03 11:37:57.243097 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/21329 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.241664) v2 -- ?+0 0xa5d2600 con 0x249252c0 -466> 2014-11-03 11:37:57.243183 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/21329 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.241664) v2 -- ?+0 0x27150800 con 0x24229000 -465> 2014-11-03 11:37:57.243707 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450952 192.168.5.2:0/1006072 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x1a327a00 con 0x63649a0 -464> 2014-11-03 11:37:57.244229 7f28fb812700 0 osd.21 105950 do_command r=0 -463> 2014-11-03 11:37:57.244246 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006072 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x2e01be00 con 0x63649a0 -462> 2014-11-03 11:37:57.248420 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.109 192.168.5.8:0/25883 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.247294) v2 ==== 47+0+0 (361014571 0 0) 0x2b1ff800 con 0x2327a160 -461> 2014-11-03 11:37:57.248441 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.109 192.168.5.8:0/25883 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.247294) v2 ==== 47+0+0 (361014571 0 0) 0x27256600 con 0x2327a000 -460> 2014-11-03 11:37:57.248469 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/25883 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.247294) v2 -- ?+0 0x91ae400 con 0x2327a160 -459> 2014-11-03 11:37:57.248581 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/25883 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.247294) v2 -- ?+0 0x1a327200 con 0x2327a000 -458> 2014-11-03 11:37:57.250052 7f2914d0c700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.250052, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -457> 2014-11-03 11:37:57.250147 7f2914d0c700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.250147, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -456> 2014-11-03 11:37:57.250191 7f2914d0c700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.250191, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -455> 2014-11-03 11:37:57.250904 7f291450b700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.250904, event: journaled_completion_queued, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -454> 2014-11-03 11:37:57.250945 7f291450b700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.250945, event: journaled_completion_queued, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -453> 2014-11-03 11:37:57.250948 7f2911d06700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.250948, event: commit_sent, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -452> 2014-11-03 11:37:57.250956 7f291450b700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.250956, event: journaled_completion_queued, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -451> 2014-11-03 11:37:57.251015 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x314a0600 con 0x22fbc840 -450> 2014-11-03 11:37:57.251048 7f2911d06700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.251047, event: commit_sent, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -449> 2014-11-03 11:37:57.251149 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6828/12612 -- osd_sub_op_reply(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] ondisk, result = 0) v2 -- ?+0 0x314a0080 con 0x22fbc840 -448> 2014-11-03 11:37:57.251189 7f2911d06700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.251189, event: commit_sent, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -447> 2014-11-03 11:37:57.251225 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3149fdc0 con 0x22fbcb00 -446> 2014-11-03 11:37:57.256791 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.31 192.168.5.3:0/20618 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.256248) v2 ==== 47+0+0 (3957337044 0 0) 0x2470a600 con 0x24925840 -445> 2014-11-03 11:37:57.256844 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.256248) v2 -- ?+0 0x27256600 con 0x24925840 -444> 2014-11-03 11:37:57.256830 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.31 192.168.5.3:0/20618 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.256248) v2 ==== 47+0+0 (3957337044 0 0) 0x2a9d3000 con 0x24229580 -443> 2014-11-03 11:37:57.256903 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/20618 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.256248) v2 -- ?+0 0x2b1ff800 con 0x24229580 -442> 2014-11-03 11:37:57.257597 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450952 192.168.5.2:0/1006072 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2a9d3800 con 0x63649a0 -441> 2014-11-03 11:37:57.258871 7f28fb812700 0 osd.21 105950 do_command r=0 -440> 2014-11-03 11:37:57.258904 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006072 -- command_reply(tid 2: 0 ) v1 -- ?+14290 0xccda540 con 0x63649a0 -439> 2014-11-03 11:37:57.267879 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006072 pipe(0x2b2e0dc0 sd=95 :6802 s=2 pgs=2 cs=1 l=1 c=0x63649a0).reader couldn't read tag, (0) Success -438> 2014-11-03 11:37:57.267965 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006072 pipe(0x2b2e0dc0 sd=95 :6802 s=2 pgs=2 cs=1 l=1 c=0x63649a0).fault (0) Success -437> 2014-11-03 11:37:57.268146 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x63649a0 session 0x2348c000 -436> 2014-11-03 11:37:57.293311 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6808/3622 -- MOSDPGPushReply(24.1c1 105950 [PushReplyOp(70a501c1/rbd_data.5348922ae8944a.0000000000000989/head//24)]) v2 -- ?+0 0x2796c600 con 0x22fe0000 -435> 2014-11-03 11:37:57.293516 7f2912507700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.293516, event: sub_op_applied, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -434> 2014-11-03 11:37:57.293558 7f2912507700 5 -- op tracker -- seq: 3307, time: 2014-11-03 11:37:57.293558, event: done, op: osd_sub_op(client.7867945.0:15039402 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195970 snapset=0=[]:[] snapc=0=[]) -433> 2014-11-03 11:37:57.293703 7f2912507700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.293703, event: sub_op_applied, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -432> 2014-11-03 11:37:57.293739 7f2912507700 5 -- op tracker -- seq: 3308, time: 2014-11-03 11:37:57.293738, event: done, op: osd_sub_op(client.7867945.0:15039403 6.a8a 69d8a8a/rbd_data.538e2715e66900.000000000000017a/head//6 [] v 105950'8195971 snapset=0=[]:[] snapc=0=[]) -431> 2014-11-03 11:37:57.293868 7f2912507700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.293867, event: sub_op_applied, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -430> 2014-11-03 11:37:57.293899 7f2912507700 5 -- op tracker -- seq: 3309, time: 2014-11-03 11:37:57.293899, event: done, op: osd_sub_op(client.7937666.0:741815 6.576 bec10d76/rbd_data.76815e5f1adf89.00000000000001a6/head//6 [] v 105950'17262145 snapset=0=[]:[] snapc=0=[]) -429> 2014-11-03 11:37:57.294821 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 55 ==== MRecoveryReserve RELEASE pgid: 24.1c1, query_epoch: 105950 v2 ==== 26+0+0 (1056099995 0 0) 0x2796c600 con 0x22fe0000 -428> 2014-11-03 11:37:57.294882 7f290b031700 5 -- op tracker -- seq: 3310, time: 2014-11-03 11:37:57.294653, event: header_read, op: MRecoveryReserve RELEASE pgid: 24.1c1, query_epoch: 105950 -427> 2014-11-03 11:37:57.294904 7f290b031700 5 -- op tracker -- seq: 3310, time: 2014-11-03 11:37:57.294657, event: throttled, op: MRecoveryReserve RELEASE pgid: 24.1c1, query_epoch: 105950 -426> 2014-11-03 11:37:57.294910 7f290b031700 5 -- op tracker -- seq: 3310, time: 2014-11-03 11:37:57.294740, event: all_read, op: MRecoveryReserve RELEASE pgid: 24.1c1, query_epoch: 105950 -425> 2014-11-03 11:37:57.294916 7f290b031700 5 -- op tracker -- seq: 3310, time: 2014-11-03 11:37:57.294875, event: dispatched, op: MRecoveryReserve RELEASE pgid: 24.1c1, query_epoch: 105950 -424> 2014-11-03 11:37:57.294922 7f290b031700 5 -- op tracker -- seq: 3310, time: 2014-11-03 11:37:57.294922, event: waiting_for_osdmap, op: MRecoveryReserve RELEASE pgid: 24.1c1, query_epoch: 105950 -423> 2014-11-03 11:37:57.294963 7f290b031700 5 -- op tracker -- seq: 3310, time: 2014-11-03 11:37:57.294963, event: done, op: MRecoveryReserve RELEASE pgid: 24.1c1, query_epoch: 105950 -422> 2014-11-03 11:37:57.295008 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.1c1( v 105946'139902 (104755'134220,105946'139902] local-les=105948 n=69 ec=25000 les/c 105948/105943 105947/105947/105111) [70,125,21] r=2 lpr=105947 pi=58133-105946/393 luod=0'0 crt=105946'139902 active] exit Started/ReplicaActive/RepRecovering 0.766404 1 0.000137 -421> 2014-11-03 11:37:57.295102 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.70 192.168.5.6:6808/3622 56 ==== pg_info(1 pgs e105950:24.1c1) v4 ==== 1295+0+0 (2865116763 0 0) 0x259f3600 con 0x22fe0000 -420> 2014-11-03 11:37:57.295115 7f2903021700 5 osd.21 pg_epoch: 105950 pg[24.1c1( v 105946'139902 (104755'134220,105946'139902] local-les=105948 n=69 ec=25000 les/c 105948/105943 105947/105947/105111) [70,125,21] r=2 lpr=105947 pi=58133-105946/393 luod=0'0 crt=105946'139902 active] enter Started/ReplicaActive/RepNotRecovering -419> 2014-11-03 11:37:57.295138 7f290b031700 5 -- op tracker -- seq: 3311, time: 2014-11-03 11:37:57.294965, event: header_read, op: pg_info(1 pgs e105950:24.1c1) -418> 2014-11-03 11:37:57.295153 7f290b031700 5 -- op tracker -- seq: 3311, time: 2014-11-03 11:37:57.294967, event: throttled, op: pg_info(1 pgs e105950:24.1c1) -417> 2014-11-03 11:37:57.295159 7f290b031700 5 -- op tracker -- seq: 3311, time: 2014-11-03 11:37:57.295055, event: all_read, op: pg_info(1 pgs e105950:24.1c1) -416> 2014-11-03 11:37:57.295164 7f290b031700 5 -- op tracker -- seq: 3311, time: 2014-11-03 11:37:57.295134, event: dispatched, op: pg_info(1 pgs e105950:24.1c1) -415> 2014-11-03 11:37:57.295169 7f290b031700 5 -- op tracker -- seq: 3311, time: 2014-11-03 11:37:57.295169, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:24.1c1) -414> 2014-11-03 11:37:57.295178 7f290b031700 5 -- op tracker -- seq: 3311, time: 2014-11-03 11:37:57.295178, event: started, op: pg_info(1 pgs e105950:24.1c1) -413> 2014-11-03 11:37:57.295188 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- MRecoveryReserve GRANT pgid: 26.538, query_epoch: 105950 v2 -- ?+0 0x23307a00 con 0x22fef580 -412> 2014-11-03 11:37:57.295211 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.538( v 105945'8485 lc 105913'8483 (66374'5241,105945'8485] local-les=105948 n=16 ec=25085 les/c 105948/105943 105947/105947/104791) [10,21,98] r=1 lpr=105947 pi=95759-105946/296 luod=0'0 crt=105945'8485 lcod 0'0 active m=2] exit Started/ReplicaActive/RepWaitRecoveryReserved 30.597517 5 0.000128 -411> 2014-11-03 11:37:57.295247 7f290b031700 5 -- op tracker -- seq: 3311, time: 2014-11-03 11:37:57.295247, event: done, op: pg_info(1 pgs e105950:24.1c1) -410> 2014-11-03 11:37:57.295245 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.538( v 105945'8485 lc 105913'8483 (66374'5241,105945'8485] local-les=105948 n=16 ec=25085 les/c 105948/105943 105947/105947/104791) [10,21,98] r=1 lpr=105947 pi=95759-105946/296 luod=0'0 crt=105945'8485 lcod 0'0 active m=2] enter Started/ReplicaActive/RepRecovering -409> 2014-11-03 11:37:57.327191 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x2be31600 con 0x22fe0420 -408> 2014-11-03 11:37:57.358701 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.65 192.168.5.5:0/30042 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.139953) v2 ==== 47+0+0 (970199406 0 0) 0x2438f800 con 0x249256e0 -407> 2014-11-03 11:37:57.358783 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/30042 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.139953) v2 -- ?+0 0x2470a600 con 0x249256e0 -406> 2014-11-03 11:37:57.373535 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.36 192.168.5.3:0/9756 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.373050) v2 ==== 47+0+0 (1288449192 0 0) 0x23f8ca00 con 0x240ad2c0 -405> 2014-11-03 11:37:57.373563 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.36 192.168.5.3:0/9756 7 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.373050) v2 ==== 47+0+0 (1288449192 0 0) 0x286bb400 con 0x2469e840 -404> 2014-11-03 11:37:57.373620 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.3:0/9756 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.373050) v2 -- ?+0 0x2438f800 con 0x240ad2c0 -403> 2014-11-03 11:37:57.373798 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.3:0/9756 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.373050) v2 -- ?+0 0x2a9d3000 con 0x2469e840 -402> 2014-11-03 11:37:57.385663 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 124 ==== osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2676427904 0 3511520646) 0x29056400 con 0x22fb76e0 -401> 2014-11-03 11:37:57.385728 7f28edd69700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.385496, event: header_read, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -400> 2014-11-03 11:37:57.385749 7f28edd69700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.385500, event: throttled, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -399> 2014-11-03 11:37:57.385762 7f28edd69700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.385643, event: all_read, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -398> 2014-11-03 11:37:57.385805 7f28edd69700 5 -- op tracker -- seq: 3312, time: 0.000000, event: dispatched, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -397> 2014-11-03 11:37:57.385930 7f290001b700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.385930, event: reached_pg, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -396> 2014-11-03 11:37:57.385988 7f290001b700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.385988, event: started, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -395> 2014-11-03 11:37:57.386197 7f290001b700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.386197, event: started, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -394> 2014-11-03 11:37:57.386253 7f290001b700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.386253, event: commit_queued_for_journal_write, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -393> 2014-11-03 11:37:57.386393 7f2914d0c700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.386392, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -392> 2014-11-03 11:37:57.388131 7f291450b700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.388131, event: journaled_completion_queued, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -391> 2014-11-03 11:37:57.388247 7f2911d06700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.388246, event: commit_sent, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -390> 2014-11-03 11:37:57.388305 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3149f000 con 0x22fb76e0 -389> 2014-11-03 11:37:57.388681 7f2912507700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.388680, event: sub_op_applied, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -388> 2014-11-03 11:37:57.388735 7f2912507700 5 -- op tracker -- seq: 3312, time: 2014-11-03 11:37:57.388735, event: done, op: osd_sub_op(client.8036651.0:1068114 6.46e b1b6446e/rbd_data.7515b62ae8944a.00000000000002fe/head//6 [] v 105950'10288361 snapset=0=[]:[] snapc=0=[]) -387> 2014-11-03 11:37:57.404344 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 48 ==== MOSDPGPush(6.9d8 105950 [PushOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, version: 105946'7950017, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (3137700582 0 0) 0x2be31600 con 0x22fe0420 -386> 2014-11-03 11:37:57.404411 7f28f5be7700 5 -- op tracker -- seq: 3313, time: 2014-11-03 11:37:57.393926, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, version: 105946'7950017, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -385> 2014-11-03 11:37:57.404446 7f28f5be7700 5 -- op tracker -- seq: 3313, time: 2014-11-03 11:37:57.393928, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, version: 105946'7950017, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -384> 2014-11-03 11:37:57.404470 7f28f5be7700 5 -- op tracker -- seq: 3313, time: 2014-11-03 11:37:57.404326, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, version: 105946'7950017, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -383> 2014-11-03 11:37:57.404493 7f28f5be7700 5 -- op tracker -- seq: 3313, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, version: 105946'7950017, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -382> 2014-11-03 11:37:57.404583 7f290201f700 5 -- op tracker -- seq: 3313, time: 2014-11-03 11:37:57.404582, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, version: 105946'7950017, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -381> 2014-11-03 11:37:57.404821 7f290201f700 5 -- op tracker -- seq: 3313, time: 2014-11-03 11:37:57.404820, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6, version: 105946'7950017, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(1524c9d8/rbd_data.485fef4a9a0ba9.00000000000003fe/head//6@105946'7950017, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -380> 2014-11-03 11:37:57.416814 7f28c2fc7700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x255928c0 sd=231 :6802 s=0 pgs=0 cs=0 l=0 c=0x6364420).accept sd=231 192.168.5.2:43029/0 -379> 2014-11-03 11:37:57.417938 7f28fd015700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- MOSDPGPull(6.9d8 105950 [PullOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x2be31400 con 0x22fb7160 -378> 2014-11-03 11:37:57.418083 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12420027 192.168.5.2:0/1006350 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x286bb600 con 0x6364420 -377> 2014-11-03 11:37:57.418928 7f28fb812700 0 osd.21 105950 do_command r=0 -376> 2014-11-03 11:37:57.418957 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006350 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x24500000 con 0x6364420 -375> 2014-11-03 11:37:57.431896 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12420027 192.168.5.2:0/1006350 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x328ec00 con 0x6364420 -374> 2014-11-03 11:37:57.432773 7f28fb812700 0 osd.21 105950 do_command r=0 -373> 2014-11-03 11:37:57.432790 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006350 -- command_reply(tid 2: 0 ) v1 -- ?+16699 0x2b23e1c0 con 0x6364420 -372> 2014-11-03 11:37:57.438218 7f28f70fc700 1 -- 192.168.5.2:6803/7606 <== osd.4 192.168.5.1:6867/31636 101 ==== osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (3270627571 0 3511520646) 0x2ce3b000 con 0x22fbc580 -371> 2014-11-03 11:37:57.438272 7f28f70fc700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438057, event: header_read, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -370> 2014-11-03 11:37:57.438301 7f28f70fc700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438060, event: throttled, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -369> 2014-11-03 11:37:57.438327 7f28f70fc700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438185, event: all_read, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -368> 2014-11-03 11:37:57.438340 7f28f70fc700 5 -- op tracker -- seq: 3314, time: 0.000000, event: dispatched, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -367> 2014-11-03 11:37:57.438412 7f28fe818700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438412, event: reached_pg, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -366> 2014-11-03 11:37:57.438447 7f28fe818700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438447, event: started, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -365> 2014-11-03 11:37:57.438565 7f28fe818700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438565, event: started, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -364> 2014-11-03 11:37:57.438597 7f28fe818700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438597, event: commit_queued_for_journal_write, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -363> 2014-11-03 11:37:57.438634 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006350 pipe(0x255928c0 sd=231 :6802 s=2 pgs=2 cs=1 l=1 c=0x6364420).reader couldn't read tag, (0) Success -362> 2014-11-03 11:37:57.438674 7f2914d0c700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438674, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -361> 2014-11-03 11:37:57.438709 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006350 pipe(0x255928c0 sd=231 :6802 s=2 pgs=2 cs=1 l=1 c=0x6364420).fault (0) Success -360> 2014-11-03 11:37:57.438832 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x6364420 session 0x2d281780 -359> 2014-11-03 11:37:57.438938 7f291450b700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.438937, event: journaled_completion_queued, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -358> 2014-11-03 11:37:57.439014 7f2911d06700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.439014, event: commit_sent, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -357> 2014-11-03 11:37:57.439051 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6867/31636 -- osd_sub_op_reply(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] ondisk, result = 0) v2 -- ?+0 0x314a0340 con 0x22fbc580 -356> 2014-11-03 11:37:57.439245 7f2912507700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.439245, event: sub_op_applied, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -355> 2014-11-03 11:37:57.439293 7f2912507700 5 -- op tracker -- seq: 3314, time: 2014-11-03 11:37:57.439293, event: done, op: osd_sub_op(client.8709328.0:1681061 6.4ee ceda9cee/rbd_data.797e692ae8944a.000000000000066f/head//6 [] v 105950'48068289 snapset=0=[]:[] snapc=0=[]) -354> 2014-11-03 11:37:57.461409 7f28f45d1700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 50 ==== MOSDPGPush(26.538 105950 [PushOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26, version: 105938'8484, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26@105938'8484, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26, version: 105945'8485, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26@105945'8485, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390501+0+0 (2610668415 0 0) 0x23307a00 con 0x22fef580 -353> 2014-11-03 11:37:57.461473 7f28f45d1700 5 -- op tracker -- seq: 3315, time: 2014-11-03 11:37:57.439337, event: header_read, op: MOSDPGPush(26.538 105950 [PushOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26, version: 105938'8484, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26@105938'8484, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26, version: 105945'8485, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26@105945'8485, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -352> 2014-11-03 11:37:57.461504 7f28f45d1700 5 -- op tracker -- seq: 3315, time: 2014-11-03 11:37:57.439339, event: throttled, op: MOSDPGPush(26.538 105950 [PushOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26, version: 105938'8484, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26@105938'8484, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26, version: 105945'8485, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26@105945'8485, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -351> 2014-11-03 11:37:57.461529 7f28f45d1700 5 -- op tracker -- seq: 3315, time: 2014-11-03 11:37:57.461391, event: all_read, op: MOSDPGPush(26.538 105950 [PushOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26, version: 105938'8484, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26@105938'8484, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26, version: 105945'8485, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26@105945'8485, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -350> 2014-11-03 11:37:57.461552 7f28f45d1700 5 -- op tracker -- seq: 3315, time: 0.000000, event: dispatched, op: MOSDPGPush(26.538 105950 [PushOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26, version: 105938'8484, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26@105938'8484, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26, version: 105945'8485, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26@105945'8485, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -349> 2014-11-03 11:37:57.461680 7f28ff019700 5 -- op tracker -- seq: 3315, time: 2014-11-03 11:37:57.461680, event: reached_pg, op: MOSDPGPush(26.538 105950 [PushOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26, version: 105938'8484, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26@105938'8484, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26, version: 105945'8485, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26@105945'8485, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -348> 2014-11-03 11:37:57.461927 7f28ff019700 5 -- op tracker -- seq: 3315, time: 2014-11-03 11:37:57.461927, event: done, op: MOSDPGPush(26.538 105950 [PushOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26, version: 105938'8484, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26@105938'8484, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26, version: 105945'8485, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26@105945'8485, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -347> 2014-11-03 11:37:57.485560 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.57 192.168.5.5:0/16411 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.483902) v2 ==== 47+0+0 (3278972028 0 0) 0x896de00 con 0x2469e420 -346> 2014-11-03 11:37:57.485610 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.483902) v2 -- ?+0 0x286bb400 con 0x2469e420 -345> 2014-11-03 11:37:57.485732 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.57 192.168.5.5:0/16411 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.483902) v2 ==== 47+0+0 (3278972028 0 0) 0x243ffe00 con 0x242c0160 -344> 2014-11-03 11:37:57.485811 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/16411 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.483902) v2 -- ?+0 0x23f8ca00 con 0x242c0160 -343> 2014-11-03 11:37:57.485958 7f28f72fe700 1 -- 192.168.5.2:6803/7606 <== osd.7 192.168.5.1:6886/3511 103 ==== osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (2864939662 0 3511520646) 0x2c338400 con 0x22fbc420 -342> 2014-11-03 11:37:57.486004 7f28f72fe700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.485816, event: header_read, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -341> 2014-11-03 11:37:57.486031 7f28f72fe700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.485818, event: throttled, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -340> 2014-11-03 11:37:57.486060 7f28f72fe700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.485940, event: all_read, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -339> 2014-11-03 11:37:57.486081 7f28f72fe700 5 -- op tracker -- seq: 3316, time: 0.000000, event: dispatched, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -338> 2014-11-03 11:37:57.486180 7f28ff81a700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.486180, event: reached_pg, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -337> 2014-11-03 11:37:57.486215 7f28ff81a700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.486215, event: started, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -336> 2014-11-03 11:37:57.486429 7f28ff81a700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.486429, event: started, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -335> 2014-11-03 11:37:57.486470 7f28ff81a700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.486470, event: commit_queued_for_journal_write, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -334> 2014-11-03 11:37:57.494027 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.114 192.168.5.9:0/17408 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.492373) v2 ==== 47+0+0 (1811155153 0 0) 0x328e800 con 0x2469e580 -333> 2014-11-03 11:37:57.494075 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.9:0/17408 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.492373) v2 -- ?+0 0x896de00 con 0x2469e580 -332> 2014-11-03 11:37:57.494037 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.114 192.168.5.9:0/17408 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.492373) v2 ==== 47+0+0 (1811155153 0 0) 0x23482e00 con 0x2484a000 -331> 2014-11-03 11:37:57.494201 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.9:0/17408 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.492373) v2 -- ?+0 0x243ffe00 con 0x2484a000 -330> 2014-11-03 11:37:57.496348 7f28ed05c700 1 -- 192.168.5.2:6803/7606 <== osd.49 192.168.5.4:6841/21769 55 ==== MOSDPGPush(6.9d8 105950 [PushOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, version: 105946'7950020, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195490+0+0 (3765659146 0 0) 0x2be31400 con 0x22fb7160 -329> 2014-11-03 11:37:57.496395 7f28ed05c700 5 -- op tracker -- seq: 3317, time: 2014-11-03 11:37:57.479921, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, version: 105946'7950020, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -328> 2014-11-03 11:37:57.496431 7f28ed05c700 5 -- op tracker -- seq: 3317, time: 2014-11-03 11:37:57.479922, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, version: 105946'7950020, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -327> 2014-11-03 11:37:57.496449 7f28ed05c700 5 -- op tracker -- seq: 3317, time: 2014-11-03 11:37:57.496333, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, version: 105946'7950020, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -326> 2014-11-03 11:37:57.496463 7f28ed05c700 5 -- op tracker -- seq: 3317, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, version: 105946'7950020, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -325> 2014-11-03 11:37:57.496536 7f290201f700 5 -- op tracker -- seq: 3317, time: 2014-11-03 11:37:57.496536, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, version: 105946'7950020, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -324> 2014-11-03 11:37:57.496736 7f290201f700 5 -- op tracker -- seq: 3317, time: 2014-11-03 11:37:57.496736, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6, version: 105946'7950020, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(e13589d8/rbd_data.4b843b2ae8944a.0000000000000c00/head//6@105946'7950020, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -323> 2014-11-03 11:37:57.504412 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- pg_trim(26.538 to 105945'8485 e105950) v1 -- ?+0 0x23561a00 con 0x22fef580 -322> 2014-11-03 11:37:57.504433 7f2914d0c700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.504433, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -321> 2014-11-03 11:37:57.524442 7f28eca56700 1 -- 192.168.5.2:6803/7606 <== osd.104 192.168.5.8:6828/2576 49 ==== osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3622301001 0 3511520646) 0x96ec400 con 0x230c2dc0 -320> 2014-11-03 11:37:57.524506 7f28eca56700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.524285, event: header_read, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -319> 2014-11-03 11:37:57.524538 7f28eca56700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.524288, event: throttled, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -318> 2014-11-03 11:37:57.524557 7f28eca56700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.524421, event: all_read, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -317> 2014-11-03 11:37:57.524575 7f28eca56700 5 -- op tracker -- seq: 3318, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -316> 2014-11-03 11:37:57.524665 7f290181e700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.524665, event: reached_pg, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -315> 2014-11-03 11:37:57.524701 7f290181e700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.524701, event: started, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -314> 2014-11-03 11:37:57.524860 7f290181e700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.524860, event: started, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -313> 2014-11-03 11:37:57.524904 7f290181e700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.524903, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -312> 2014-11-03 11:37:57.534965 7f28ea430700 1 -- 192.168.5.2:6803/7606 <== osd.47 192.168.5.4:6831/12077 47 ==== osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1600386103 0 3511520646) 0x3349800 con 0x231ab580 -311> 2014-11-03 11:37:57.535017 7f28ea430700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.534737, event: header_read, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -310> 2014-11-03 11:37:57.535115 7f28ea430700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.534740, event: throttled, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -309> 2014-11-03 11:37:57.535140 7f28ea430700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.534945, event: all_read, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -308> 2014-11-03 11:37:57.535160 7f28ea430700 5 -- op tracker -- seq: 3319, time: 0.000000, event: dispatched, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -307> 2014-11-03 11:37:57.535241 7f28ff019700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.535241, event: reached_pg, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -306> 2014-11-03 11:37:57.535278 7f28ff019700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.535278, event: started, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -305> 2014-11-03 11:37:57.535391 7f28ff019700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.535391, event: started, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -304> 2014-11-03 11:37:57.535429 7f28ff019700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.535429, event: commit_queued_for_journal_write, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -303> 2014-11-03 11:37:57.535878 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.7 192.168.5.1:0/3511 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.535119) v2 ==== 47+0+0 (2659044067 0 0) 0x2a347c00 con 0x23ea4c60 -302> 2014-11-03 11:37:57.535939 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.535119) v2 -- ?+0 0x328e800 con 0x23ea4c60 -301> 2014-11-03 11:37:57.535944 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.7 192.168.5.1:0/3511 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.535119) v2 ==== 47+0+0 (2659044067 0 0) 0x29c10a00 con 0x24925000 -300> 2014-11-03 11:37:57.536012 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.1:0/3511 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.535119) v2 -- ?+0 0x23482e00 con 0x24925000 -299> 2014-11-03 11:37:57.548672 7f291450b700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.548671, event: journaled_completion_queued, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -298> 2014-11-03 11:37:57.548776 7f2911d06700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.548775, event: commit_sent, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -297> 2014-11-03 11:37:57.548801 7f2914d0c700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.548800, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -296> 2014-11-03 11:37:57.548822 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6886/3511 -- osd_sub_op_reply(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] ondisk, result = 0) v2 -- ?+0 0x3149f840 con 0x22fbc420 -295> 2014-11-03 11:37:57.548846 7f2914d0c700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.548846, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -294> 2014-11-03 11:37:57.549117 7f2912507700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.549117, event: sub_op_applied, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -293> 2014-11-03 11:37:57.549154 7f2912507700 5 -- op tracker -- seq: 3316, time: 2014-11-03 11:37:57.549154, event: done, op: osd_sub_op(client.7870178.0:905390 6.4c4 f1684c4/rbd_data.19ea631056015d.0000000000000300/head//6 [] v 105950'10944781 snapset=0=[]:[] snapc=0=[]) -292> 2014-11-03 11:37:57.566609 7f28f36c2700 1 -- 192.168.5.2:6803/7606 <== osd.103 192.168.5.8:6817/29722 215 ==== osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (2407847038 0 3511520646) 0x24450400 con 0x22fefdc0 -291> 2014-11-03 11:37:57.566650 7f28f36c2700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.566478, event: header_read, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -290> 2014-11-03 11:37:57.566670 7f28f36c2700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.566480, event: throttled, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -289> 2014-11-03 11:37:57.566690 7f28f36c2700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.566599, event: all_read, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -288> 2014-11-03 11:37:57.566702 7f28f36c2700 5 -- op tracker -- seq: 3320, time: 0.000000, event: dispatched, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -287> 2014-11-03 11:37:57.566767 7f290101d700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.566766, event: reached_pg, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -286> 2014-11-03 11:37:57.566798 7f290101d700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.566798, event: started, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -285> 2014-11-03 11:37:57.566915 7f290101d700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.566915, event: started, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -284> 2014-11-03 11:37:57.566951 7f290101d700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.566951, event: commit_queued_for_journal_write, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -283> 2014-11-03 11:37:57.572300 7f28c2fc7700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x25592b80 sd=33 :6802 s=0 pgs=0 cs=0 l=0 c=0x19180b00).accept sd=33 192.168.5.2:43032/0 -282> 2014-11-03 11:37:57.572981 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450958 192.168.5.2:0/1006509 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23386400 con 0x19180b00 -281> 2014-11-03 11:37:57.573473 7f28fb812700 0 osd.21 105950 do_command r=0 -280> 2014-11-03 11:37:57.573491 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006509 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x2b23e1c0 con 0x19180b00 -279> 2014-11-03 11:37:57.577133 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 106 ==== MOSDPGPush(24.4f6 105950 [PushOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24, version: 105946'217149, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24@105946'217149, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195273+0+0 (2865183399 0 0) 0x236a8400 con 0x2308b840 -278> 2014-11-03 11:37:57.577195 7f28ef985700 5 -- op tracker -- seq: 3321, time: 2014-11-03 11:37:57.565670, event: header_read, op: MOSDPGPush(24.4f6 105950 [PushOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24, version: 105946'217149, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24@105946'217149, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -277> 2014-11-03 11:37:57.577227 7f28ef985700 5 -- op tracker -- seq: 3321, time: 2014-11-03 11:37:57.565675, event: throttled, op: MOSDPGPush(24.4f6 105950 [PushOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24, version: 105946'217149, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24@105946'217149, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -276> 2014-11-03 11:37:57.577253 7f28ef985700 5 -- op tracker -- seq: 3321, time: 2014-11-03 11:37:57.577119, event: all_read, op: MOSDPGPush(24.4f6 105950 [PushOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24, version: 105946'217149, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24@105946'217149, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -275> 2014-11-03 11:37:57.577307 7f28ef985700 5 -- op tracker -- seq: 3321, time: 0.000000, event: dispatched, op: MOSDPGPush(24.4f6 105950 [PushOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24, version: 105946'217149, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24@105946'217149, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -274> 2014-11-03 11:37:57.577459 7f28ff81a700 5 -- op tracker -- seq: 3321, time: 2014-11-03 11:37:57.577459, event: reached_pg, op: MOSDPGPush(24.4f6 105950 [PushOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24, version: 105946'217149, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24@105946'217149, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -273> 2014-11-03 11:37:57.577604 7f28ff81a700 5 -- op tracker -- seq: 3321, time: 2014-11-03 11:37:57.577604, event: done, op: MOSDPGPush(24.4f6 105950 [PushOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24, version: 105946'217149, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24@105946'217149, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -272> 2014-11-03 11:37:57.583852 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450958 192.168.5.2:0/1006509 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2a8d7e00 con 0x19180b00 -271> 2014-11-03 11:37:57.584631 7f28fb812700 0 osd.21 105950 do_command r=0 -270> 2014-11-03 11:37:57.584661 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006509 -- command_reply(tid 2: 0 ) v1 -- ?+14290 0x3243540 con 0x19180b00 -269> 2014-11-03 11:37:57.590851 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006509 pipe(0x25592b80 sd=33 :6802 s=2 pgs=2 cs=1 l=1 c=0x19180b00).reader couldn't read tag, (0) Success -268> 2014-11-03 11:37:57.590903 7f28c2fc7700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006509 pipe(0x25592b80 sd=33 :6802 s=2 pgs=2 cs=1 l=1 c=0x19180b00).fault (0) Success -267> 2014-11-03 11:37:57.591056 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x19180b00 session 0x2d281780 -266> 2014-11-03 11:37:57.594280 7f291450b700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.594280, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -265> 2014-11-03 11:37:57.594333 7f291450b700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.594333, event: journaled_completion_queued, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -264> 2014-11-03 11:37:57.594383 7f2911d06700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.594383, event: commit_sent, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -263> 2014-11-03 11:37:57.594450 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6828/2576 -- osd_sub_op_reply(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b41e2c0 con 0x230c2dc0 -262> 2014-11-03 11:37:57.594486 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.1:6816/15344 -- MOSDPGPushReply(26.538 105950 [PushReplyOp(5b268d38/rbd_data.4626792ae8944a.0000000000001055/head//26),PushReplyOp(87bb1538/rbd_data.4626792ae8944a.00000000000017b7/head//26)]) v2 -- ?+0 0x23d2b800 con 0x22fef580 -261> 2014-11-03 11:37:57.594542 7f2911d06700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.594541, event: commit_sent, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -260> 2014-11-03 11:37:57.594595 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6831/12077 -- osd_sub_op_reply(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] ondisk, result = 0) v2 -- ?+0 0x2b41fb80 con 0x231ab580 -259> 2014-11-03 11:37:57.594458 7f2914d0c700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.594458, event: write_thread_in_journal_buffer, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -258> 2014-11-03 11:37:57.594854 7f2912507700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.594854, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -257> 2014-11-03 11:37:57.594934 7f2912507700 5 -- op tracker -- seq: 3318, time: 2014-11-03 11:37:57.594934, event: done, op: osd_sub_op(client.8236383.0:16630956 6.69b 40f0ee9b/rbd_data.7e655d2ae8944a.00000000000015bd/head//6 [] v 105950'21540858 snapset=0=[]:[] snapc=0=[]) -256> 2014-11-03 11:37:57.595443 7f2912507700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.595442, event: sub_op_applied, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -255> 2014-11-03 11:37:57.595493 7f2912507700 5 -- op tracker -- seq: 3319, time: 2014-11-03 11:37:57.595493, event: done, op: osd_sub_op(client.8236383.0:16631009 6.a79 6d18da79/rbd_data.7e655d2ae8944a.000000000000008e/head//6 [] v 105950'8629966 snapset=0=[]:[] snapc=0=[]) -254> 2014-11-03 11:37:57.595852 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 51 ==== MRecoveryReserve RELEASE pgid: 26.538, query_epoch: 105950 v2 ==== 26+0+0 (4180334409 0 0) 0x23d2b800 con 0x22fef580 -253> 2014-11-03 11:37:57.595919 7f290b031700 5 -- op tracker -- seq: 3322, time: 2014-11-03 11:37:57.595703, event: header_read, op: MRecoveryReserve RELEASE pgid: 26.538, query_epoch: 105950 -252> 2014-11-03 11:37:57.595947 7f290b031700 5 -- op tracker -- seq: 3322, time: 2014-11-03 11:37:57.595706, event: throttled, op: MRecoveryReserve RELEASE pgid: 26.538, query_epoch: 105950 -251> 2014-11-03 11:37:57.595970 7f290b031700 5 -- op tracker -- seq: 3322, time: 2014-11-03 11:37:57.595784, event: all_read, op: MRecoveryReserve RELEASE pgid: 26.538, query_epoch: 105950 -250> 2014-11-03 11:37:57.595982 7f290b031700 5 -- op tracker -- seq: 3322, time: 2014-11-03 11:37:57.595910, event: dispatched, op: MRecoveryReserve RELEASE pgid: 26.538, query_epoch: 105950 -249> 2014-11-03 11:37:57.596002 7f290b031700 5 -- op tracker -- seq: 3322, time: 2014-11-03 11:37:57.596001, event: waiting_for_osdmap, op: MRecoveryReserve RELEASE pgid: 26.538, query_epoch: 105950 -248> 2014-11-03 11:37:57.596069 7f290b031700 5 -- op tracker -- seq: 3322, time: 2014-11-03 11:37:57.596068, event: done, op: MRecoveryReserve RELEASE pgid: 26.538, query_epoch: 105950 -247> 2014-11-03 11:37:57.596127 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.10 192.168.5.1:6816/15344 52 ==== pg_info(1 pgs e105950:26.538) v4 ==== 767+0+0 (3210406615 0 0) 0x23561a00 con 0x22fef580 -246> 2014-11-03 11:37:57.596143 7f290b031700 5 -- op tracker -- seq: 3323, time: 2014-11-03 11:37:57.595964, event: header_read, op: pg_info(1 pgs e105950:26.538) -245> 2014-11-03 11:37:57.596150 7f290b031700 5 -- op tracker -- seq: 3323, time: 2014-11-03 11:37:57.595966, event: throttled, op: pg_info(1 pgs e105950:26.538) -244> 2014-11-03 11:37:57.596155 7f290b031700 5 -- op tracker -- seq: 3323, time: 2014-11-03 11:37:57.596042, event: all_read, op: pg_info(1 pgs e105950:26.538) -243> 2014-11-03 11:37:57.596168 7f290b031700 5 -- op tracker -- seq: 3323, time: 2014-11-03 11:37:57.596141, event: dispatched, op: pg_info(1 pgs e105950:26.538) -242> 2014-11-03 11:37:57.596173 7f290b031700 5 -- op tracker -- seq: 3323, time: 2014-11-03 11:37:57.596173, event: waiting_for_osdmap, op: pg_info(1 pgs e105950:26.538) -241> 2014-11-03 11:37:57.596197 7f290b031700 5 -- op tracker -- seq: 3323, time: 2014-11-03 11:37:57.596197, event: started, op: pg_info(1 pgs e105950:26.538) -240> 2014-11-03 11:37:57.596148 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.538( v 105945'8485 (66374'5241,105945'8485] local-les=105948 n=16 ec=25085 les/c 105948/105943 105947/105947/104791) [10,21,98] r=1 lpr=105947 pi=95759-105946/296 luod=0'0 crt=105945'8485 active] exit Started/ReplicaActive/RepRecovering 0.300903 1 0.000083 -239> 2014-11-03 11:37:57.596392 7f2903021700 5 osd.21 pg_epoch: 105950 pg[26.538( v 105945'8485 (66374'5241,105945'8485] local-les=105948 n=16 ec=25085 les/c 105948/105943 105947/105947/104791) [10,21,98] r=1 lpr=105947 pi=95759-105946/296 luod=0'0 crt=105945'8485 active] enter Started/ReplicaActive/RepNotRecovering -238> 2014-11-03 11:37:57.596570 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6807/16411 -- MRecoveryReserve GRANT pgid: 6.167, query_epoch: 105950 v2 -- ?+0 0x78cbe00 con 0x230c29a0 -237> 2014-11-03 11:37:57.596585 7f290b031700 5 -- op tracker -- seq: 3323, time: 2014-11-03 11:37:57.596585, event: done, op: pg_info(1 pgs e105950:26.538) -236> 2014-11-03 11:37:57.596642 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.167( v 105950'5727269 lc 105941'5726565 (105913'5723361,105950'5727269] local-les=105948 n=4496 ec=164 les/c 105948/105943 105947/105947/105036) [57,21,42] r=1 lpr=105947 pi=97459-105946/340 luod=0'0 crt=105946'5727261 active m=17] exit Started/ReplicaActive/RepWaitRecoveryReserved 30.569763 4 0.000073 -235> 2014-11-03 11:37:57.596704 7f2902820700 5 osd.21 pg_epoch: 105950 pg[6.167( v 105950'5727269 lc 105941'5726565 (105913'5723361,105950'5727269] local-les=105948 n=4496 ec=164 les/c 105948/105943 105947/105947/105036) [57,21,42] r=1 lpr=105947 pi=97459-105946/340 luod=0'0 crt=105946'5727261 active m=17] enter Started/ReplicaActive/RepRecovering -234> 2014-11-03 11:37:57.597437 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.49 192.168.5.4:0/21769 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.596114) v2 ==== 47+0+0 (3093796187 0 0) 0x2a77dc00 con 0x242296e0 -233> 2014-11-03 11:37:57.597449 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.49 192.168.5.4:0/21769 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.596114) v2 ==== 47+0+0 (3093796187 0 0) 0x2be31000 con 0x249259a0 -232> 2014-11-03 11:37:57.597482 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.4:0/21769 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.596114) v2 -- ?+0 0x2a347c00 con 0x242296e0 -231> 2014-11-03 11:37:57.597547 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.4:0/21769 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.596114) v2 -- ?+0 0x29c10a00 con 0x249259a0 -230> 2014-11-03 11:37:57.626177 7f291450b700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.626177, event: journaled_completion_queued, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -229> 2014-11-03 11:37:57.626248 7f2911d06700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.626247, event: commit_sent, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -228> 2014-11-03 11:37:57.626298 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6817/29722 -- osd_sub_op_reply(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] ondisk, result = 0) v2 -- ?+0 0x2b41e580 con 0x22fefdc0 -227> 2014-11-03 11:37:57.626563 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- pg_trim(24.4f6 to 105946'217149 e105950) v1 -- ?+0 0x22a08a00 con 0x2308b840 -226> 2014-11-03 11:37:57.626767 7f2912507700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.626767, event: sub_op_applied, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -225> 2014-11-03 11:37:57.626818 7f2912507700 5 -- op tracker -- seq: 3320, time: 2014-11-03 11:37:57.626818, event: done, op: osd_sub_op(client.8703713.0:32254539 24.787 cc968787/rbd_data.7653db76142022.0000000000000040/head//24 [] v 105950'4953163 snapset=0=[]:[] snapc=0=[]) -224> 2014-11-03 11:37:57.626899 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MOSDPGPull(6.9d8 105950 [PullOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x23a9e800 con 0x22fe0420 -223> 2014-11-03 11:37:57.634544 7f28edd69700 1 -- 192.168.5.2:6803/7606 <== osd.90 192.168.5.7:6831/27918 125 ==== osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (1526815961 0 3511520646) 0x26942c00 con 0x22fb76e0 -222> 2014-11-03 11:37:57.634580 7f28edd69700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.634450, event: header_read, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -221> 2014-11-03 11:37:57.634599 7f28edd69700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.634452, event: throttled, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -220> 2014-11-03 11:37:57.634609 7f28edd69700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.634529, event: all_read, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -219> 2014-11-03 11:37:57.634618 7f28edd69700 5 -- op tracker -- seq: 3324, time: 0.000000, event: dispatched, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -218> 2014-11-03 11:37:57.634801 7f28fd816700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.634801, event: reached_pg, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -217> 2014-11-03 11:37:57.634838 7f28fd816700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.634838, event: started, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -216> 2014-11-03 11:37:57.635015 7f28fd816700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.635014, event: started, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -215> 2014-11-03 11:37:57.635051 7f28fd816700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.635050, event: commit_queued_for_journal_write, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -214> 2014-11-03 11:37:57.635135 7f2914d0c700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.635135, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -213> 2014-11-03 11:37:57.635346 7f291450b700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.635346, event: journaled_completion_queued, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -212> 2014-11-03 11:37:57.635397 7f2911d06700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.635397, event: commit_sent, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -211> 2014-11-03 11:37:57.635424 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6831/27918 -- osd_sub_op_reply(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] ondisk, result = 0) v2 -- ?+0 0x270d2dc0 con 0x22fb76e0 -210> 2014-11-03 11:37:57.638002 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.84 192.168.5.7:0/19824 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.636212) v2 ==== 47+0+0 (611801135 0 0) 0x2743ae00 con 0x31a9420 -209> 2014-11-03 11:37:57.638027 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.84 192.168.5.7:0/19824 9 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.636212) v2 ==== 47+0+0 (611801135 0 0) 0x2470a000 con 0x2389f420 -208> 2014-11-03 11:37:57.638104 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.7:0/19824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.636212) v2 -- ?+0 0x2be31000 con 0x31a9420 -207> 2014-11-03 11:37:57.638181 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.7:0/19824 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.636212) v2 -- ?+0 0x2a77dc00 con 0x2389f420 -206> 2014-11-03 11:37:57.642449 7f28f6cf8700 1 -- 192.168.5.2:6803/7606 <== osd.85 192.168.5.7:6801/27520 111 ==== osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) v11 ==== 1400+0+39 (824465077 0 3511520646) 0x23fb0400 con 0x22fbcb00 -205> 2014-11-03 11:37:57.642494 7f28f6cf8700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642279, event: header_read, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -204> 2014-11-03 11:37:57.642518 7f28f6cf8700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642281, event: throttled, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -203> 2014-11-03 11:37:57.642537 7f28f6cf8700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642437, event: all_read, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -202> 2014-11-03 11:37:57.642562 7f28f6cf8700 5 -- op tracker -- seq: 3325, time: 0.000000, event: dispatched, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -201> 2014-11-03 11:37:57.642631 7f290081c700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642630, event: reached_pg, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -200> 2014-11-03 11:37:57.642661 7f290081c700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642661, event: started, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -199> 2014-11-03 11:37:57.642753 7f290081c700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642753, event: started, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -198> 2014-11-03 11:37:57.642781 7f290081c700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642781, event: commit_queued_for_journal_write, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -197> 2014-11-03 11:37:57.642836 7f2914d0c700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.642836, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -196> 2014-11-03 11:37:57.643027 7f291450b700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.643027, event: journaled_completion_queued, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -195> 2014-11-03 11:37:57.643069 7f2911d06700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.643069, event: commit_sent, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -194> 2014-11-03 11:37:57.643088 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6801/27520 -- osd_sub_op_reply(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] ondisk, result = 0) v2 -- ?+0 0x270d3340 con 0x22fbcb00 -193> 2014-11-03 11:37:57.658226 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.58 192.168.5.5:0/24563 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.657096) v2 ==== 47+0+0 (758809531 0 0) 0x13defe00 con 0x244cbdc0 -192> 2014-11-03 11:37:57.658272 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.5:0/24563 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.657096) v2 -- ?+0 0x2743ae00 con 0x244cbdc0 -191> 2014-11-03 11:37:57.658265 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.58 192.168.5.5:0/24563 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.657096) v2 ==== 47+0+0 (758809531 0 0) 0x24dcf200 con 0x2469eb00 -190> 2014-11-03 11:37:57.658355 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.5:0/24563 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.657096) v2 -- ?+0 0x2470a000 con 0x2469eb00 -189> 2014-11-03 11:37:57.666286 7f28f5be7700 1 -- 192.168.5.2:6803/7606 <== osd.93 192.168.5.7:6839/7095 49 ==== MOSDPGPush(6.9d8 105950 [PushOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, version: 105946'7950023, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 4195210+0+0 (4246891419 0 0) 0x23a9e800 con 0x22fe0420 -188> 2014-11-03 11:37:57.666344 7f28f5be7700 5 -- op tracker -- seq: 3326, time: 2014-11-03 11:37:57.655321, event: header_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, version: 105946'7950023, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -187> 2014-11-03 11:37:57.666394 7f28f5be7700 5 -- op tracker -- seq: 3326, time: 2014-11-03 11:37:57.655323, event: throttled, op: MOSDPGPush(6.9d8 105950 [PushOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, version: 105946'7950023, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -186> 2014-11-03 11:37:57.666425 7f28f5be7700 5 -- op tracker -- seq: 3326, time: 2014-11-03 11:37:57.666269, event: all_read, op: MOSDPGPush(6.9d8 105950 [PushOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, version: 105946'7950023, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -185> 2014-11-03 11:37:57.666463 7f28f5be7700 5 -- op tracker -- seq: 3326, time: 0.000000, event: dispatched, op: MOSDPGPush(6.9d8 105950 [PushOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, version: 105946'7950023, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -184> 2014-11-03 11:37:57.666582 7f290201f700 5 -- op tracker -- seq: 3326, time: 2014-11-03 11:37:57.666581, event: reached_pg, op: MOSDPGPush(6.9d8 105950 [PushOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, version: 105946'7950023, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -183> 2014-11-03 11:37:57.666775 7f290201f700 5 -- op tracker -- seq: 3326, time: 2014-11-03 11:37:57.666775, event: done, op: MOSDPGPush(6.9d8 105950 [PushOp(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6, version: 105946'7950023, data_included: [0~4194304], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(4b2629d8/rbd_data.491b9f2ae8944a.0000000000002344/head//6@105946'7950023, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) -182> 2014-11-03 11:37:57.706103 7f28c2fc7700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x25591840 sd=564 :6802 s=0 pgs=0 cs=0 l=0 c=0x191809a0).accept sd=564 192.168.9.3:49698/0 -181> 2014-11-03 11:37:57.706410 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 56 ==== osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3160902430 0 3511520646) 0x285d8400 con 0x2301fb00 -180> 2014-11-03 11:37:57.706465 7f28f31bd700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.706232, event: header_read, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -179> 2014-11-03 11:37:57.706511 7f28f31bd700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.706235, event: throttled, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -178> 2014-11-03 11:37:57.706521 7f28f31bd700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.706385, event: all_read, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -177> 2014-11-03 11:37:57.706533 7f28f31bd700 5 -- op tracker -- seq: 3327, time: 0.000000, event: dispatched, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -176> 2014-11-03 11:37:57.706641 7f28fe818700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.706641, event: reached_pg, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -175> 2014-11-03 11:37:57.706676 7f28fe818700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.706675, event: started, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -174> 2014-11-03 11:37:57.706817 7f28fe818700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.706817, event: started, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -173> 2014-11-03 11:37:57.706866 7f28fe818700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.706866, event: commit_queued_for_journal_write, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -172> 2014-11-03 11:37:57.707092 7f28ef783700 1 -- 192.168.5.2:6803/7606 <== osd.74 192.168.5.6:6826/9268 41 ==== osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+9021 (2919541858 0 3397851279) 0x3349800 con 0x2308bdc0 -171> 2014-11-03 11:37:57.707139 7f28ef783700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.706894, event: header_read, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -170> 2014-11-03 11:37:57.707158 7f28ef783700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.706896, event: throttled, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -169> 2014-11-03 11:37:57.707171 7f28ef783700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.707075, event: all_read, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -168> 2014-11-03 11:37:57.707183 7f28ef783700 5 -- op tracker -- seq: 3328, time: 0.000000, event: dispatched, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -167> 2014-11-03 11:37:57.707288 7f290101d700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.707287, event: reached_pg, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -166> 2014-11-03 11:37:57.707353 7f290101d700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.707353, event: started, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -165> 2014-11-03 11:37:57.707539 7f290101d700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.707539, event: started, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -164> 2014-11-03 11:37:57.707535 7f28c2fc7700 1 -- 192.168.5.2:6802/7606 <== client.7916306 192.168.9.3:0/1014472 1 ==== osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) v4 ==== 229+0+8192 (2126653450 0 2916327792) 0x100dc500 con 0x191809a0 -163> 2014-11-03 11:37:57.707576 7f290101d700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.707576, event: commit_queued_for_journal_write, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -162> 2014-11-03 11:37:57.707578 7f28c2fc7700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.707394, event: header_read, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -161> 2014-11-03 11:37:57.707637 7f28c2fc7700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.707401, event: throttled, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -160> 2014-11-03 11:37:57.707656 7f28c2fc7700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.707524, event: all_read, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -159> 2014-11-03 11:37:57.707670 7f28c2fc7700 5 -- op tracker -- seq: 3329, time: 0.000000, event: dispatched, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -158> 2014-11-03 11:37:57.707826 7f28ff81a700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.707825, event: reached_pg, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -157> 2014-11-03 11:37:57.708035 7f28ff81a700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.708035, event: started, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -156> 2014-11-03 11:37:57.708065 7f28ff81a700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.708065, event: started, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -155> 2014-11-03 11:37:57.708126 7f28ff81a700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.708126, event: waiting for subops from 61,77, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -154> 2014-11-03 11:37:57.708169 7f28ff81a700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.5:6831/12481 -- osd_sub_op(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] v 105950'303250 snapset=0=[]:[] snapc=0=[]) v11 -- ?+9021 0x26944400 con 0x22fe0840 -153> 2014-11-03 11:37:57.708217 7f28ff81a700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6841/32590 -- osd_sub_op(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] v 105950'303250 snapset=0=[]:[] snapc=0=[]) v11 -- ?+9021 0x26942000 con 0x22fef000 -152> 2014-11-03 11:37:57.708309 7f28ff81a700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.708308, event: commit_queued_for_journal_write, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -151> 2014-11-03 11:37:57.708515 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 57 ==== osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (292720611 0 3511520646) 0x2493cc00 con 0x2301fb00 -150> 2014-11-03 11:37:57.708565 7f28f31bd700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.708336, event: header_read, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -149> 2014-11-03 11:37:57.708586 7f28f31bd700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.708338, event: throttled, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -148> 2014-11-03 11:37:57.708604 7f28f31bd700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.708494, event: all_read, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -147> 2014-11-03 11:37:57.708621 7f28f31bd700 5 -- op tracker -- seq: 3330, time: 0.000000, event: dispatched, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -146> 2014-11-03 11:37:57.708676 7f28fe818700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.708676, event: reached_pg, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -145> 2014-11-03 11:37:57.708702 7f28fe818700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.708702, event: started, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -144> 2014-11-03 11:37:57.708802 7f28fe818700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.708802, event: started, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -143> 2014-11-03 11:37:57.708841 7f28fe818700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.708841, event: commit_queued_for_journal_write, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -142> 2014-11-03 11:37:57.710138 7f28f39c5700 1 -- 192.168.5.2:6803/7606 <== osd.77 192.168.5.6:6841/32590 19 ==== osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (1720370758 0 0) 0x28b17600 con 0x22fef000 -141> 2014-11-03 11:37:57.710174 7f28f39c5700 5 -- op tracker -- seq: 3331, time: 2014-11-03 11:37:57.710045, event: header_read, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -140> 2014-11-03 11:37:57.710190 7f28f39c5700 5 -- op tracker -- seq: 3331, time: 2014-11-03 11:37:57.710046, event: throttled, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -139> 2014-11-03 11:37:57.710201 7f28f39c5700 5 -- op tracker -- seq: 3331, time: 2014-11-03 11:37:57.710104, event: all_read, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -138> 2014-11-03 11:37:57.710212 7f28f39c5700 5 -- op tracker -- seq: 3331, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -137> 2014-11-03 11:37:57.710266 7f290201f700 5 -- op tracker -- seq: 3331, time: 2014-11-03 11:37:57.710266, event: reached_pg, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -136> 2014-11-03 11:37:57.710293 7f290201f700 5 -- op tracker -- seq: 3331, time: 2014-11-03 11:37:57.710293, event: started, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -135> 2014-11-03 11:37:57.710309 7f290201f700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.710309, event: sub_op_commit_rec, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -134> 2014-11-03 11:37:57.710330 7f290201f700 5 -- op tracker -- seq: 3331, time: 2014-11-03 11:37:57.710330, event: done, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -133> 2014-11-03 11:37:57.710369 7f28f4cd8700 1 -- 192.168.5.2:6803/7606 <== osd.61 192.168.5.5:6831/12481 37 ==== osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) v2 ==== 178+0+0 (762755870 0 0) 0xa23f340 con 0x22fe0840 -132> 2014-11-03 11:37:57.710399 7f28f4cd8700 5 -- op tracker -- seq: 3332, time: 2014-11-03 11:37:57.710308, event: header_read, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -131> 2014-11-03 11:37:57.710415 7f28f4cd8700 5 -- op tracker -- seq: 3332, time: 2014-11-03 11:37:57.710310, event: throttled, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -130> 2014-11-03 11:37:57.710426 7f28f4cd8700 5 -- op tracker -- seq: 3332, time: 2014-11-03 11:37:57.710358, event: all_read, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -129> 2014-11-03 11:37:57.710438 7f28f4cd8700 5 -- op tracker -- seq: 3332, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -128> 2014-11-03 11:37:57.710427 7f28f31bd700 1 -- 192.168.5.2:6803/7606 <== osd.52 192.168.5.4:6857/3307 58 ==== osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+39 (3502414780 0 3511520646) 0x3125d400 con 0x2301fb00 -127> 2014-11-03 11:37:57.710469 7f28f31bd700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.710290, event: header_read, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -126> 2014-11-03 11:37:57.710481 7f28ff81a700 5 -- op tracker -- seq: 3332, time: 2014-11-03 11:37:57.710481, event: reached_pg, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -125> 2014-11-03 11:37:57.710493 7f28f31bd700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.710292, event: throttled, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -124> 2014-11-03 11:37:57.710507 7f28f31bd700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.710412, event: all_read, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -123> 2014-11-03 11:37:57.710507 7f28ff81a700 5 -- op tracker -- seq: 3332, time: 2014-11-03 11:37:57.710507, event: started, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -122> 2014-11-03 11:37:57.710536 7f28f31bd700 5 -- op tracker -- seq: 3333, time: 0.000000, event: dispatched, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -121> 2014-11-03 11:37:57.710556 7f28ff81a700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.710555, event: sub_op_commit_rec, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -120> 2014-11-03 11:37:57.710580 7f28ff81a700 5 -- op tracker -- seq: 3332, time: 2014-11-03 11:37:57.710580, event: done, op: osd_sub_op_reply(client.7916306.0:11671652 24.348 31885348/rbd_data.5b64062ae8944a.0000000000000929/head//24 [] ondisk, result = 0) -119> 2014-11-03 11:37:57.710604 7f290101d700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.710604, event: reached_pg, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -118> 2014-11-03 11:37:57.710630 7f290101d700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.710629, event: started, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -117> 2014-11-03 11:37:57.710708 7f290101d700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.710708, event: started, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -116> 2014-11-03 11:37:57.710736 7f290101d700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.710736, event: commit_queued_for_journal_write, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -115> 2014-11-03 11:37:57.715038 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.76 192.168.5.6:0/22493 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.714496) v2 ==== 47+0+0 (2340165601 0 0) 0x23cbea00 con 0x23b1c160 -114> 2014-11-03 11:37:57.715081 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.714496) v2 -- ?+0 0x13defe00 con 0x23b1c160 -113> 2014-11-03 11:37:57.715139 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.76 192.168.5.6:0/22493 11 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.714496) v2 ==== 47+0+0 (2340165601 0 0) 0x30c1400 con 0x24229dc0 -112> 2014-11-03 11:37:57.715175 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.6:0/22493 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.714496) v2 -- ?+0 0x24dcf200 con 0x24229dc0 -111> 2014-11-03 11:37:57.719080 7f28c4694700 1 -- 192.168.5.2:6802/7606 >> :/0 pipe(0x29059b00 sd=228 :6802 s=0 pgs=0 cs=0 l=0 c=0x19180840).accept sd=228 192.168.5.2:43036/0 -110> 2014-11-03 11:37:57.720084 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450961 192.168.5.2:0/1006695 1 ==== command(tid 1: {"prefix": "get_command_descriptions", "pgid": "6.9d8"}) v1 ==== 79+0+0 (2316469674 0 0) 0x23fdf600 con 0x19180840 -109> 2014-11-03 11:37:57.720579 7f28fb812700 0 osd.21 105950 do_command r=0 -108> 2014-11-03 11:37:57.720598 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006695 -- command_reply(tid 1: 0 ) v1 -- ?+3143 0x24500000 con 0x19180840 -107> 2014-11-03 11:37:57.730019 7f290c834700 1 -- 192.168.5.2:6802/7606 <== client.12450961 192.168.5.2:0/1006695 2 ==== command(tid 2: {"prefix": "pg", "cmd": "query", "pgid": "6.9d8"}) v1 ==== 73+0+0 (1509150634 0 0) 0x2be31000 con 0x19180840 -106> 2014-11-03 11:37:57.730820 7f28fb812700 0 osd.21 105950 do_command r=0 -105> 2014-11-03 11:37:57.730836 7f28fb812700 1 -- 192.168.5.2:6802/7606 --> 192.168.5.2:0/1006695 -- command_reply(tid 2: 0 ) v1 -- ?+14290 0x2e01be00 con 0x19180840 -104> 2014-11-03 11:37:57.737233 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006695 pipe(0x29059b00 sd=228 :6802 s=2 pgs=2 cs=1 l=1 c=0x19180840).reader couldn't read tag, (0) Success -103> 2014-11-03 11:37:57.737283 7f28c4694700 2 -- 192.168.5.2:6802/7606 >> 192.168.5.2:0/1006695 pipe(0x29059b00 sd=228 :6802 s=2 pgs=2 cs=1 l=1 c=0x19180840).fault (0) Success -102> 2014-11-03 11:37:57.737391 7f290c834700 1 osd.21 105950 ms_handle_reset con 0x19180840 session 0x100dc780 -101> 2014-11-03 11:37:57.784612 7f2907029700 1 -- 192.168.5.2:6804/7606 <== osd.103 192.168.5.8:0/29722 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.784010) v2 ==== 47+0+0 (1205655026 0 0) 0x2b950800 con 0x23b1cc60 -100> 2014-11-03 11:37:57.784651 7f290882c700 1 -- 192.168.5.2:6805/7606 <== osd.103 192.168.5.8:0/29722 10 ==== osd_ping(ping e105950 stamp 2014-11-03 11:37:57.784010) v2 ==== 47+0+0 (1205655026 0 0) 0x1adc4800 con 0x239f8c60 -99> 2014-11-03 11:37:57.784678 7f2907029700 1 -- 192.168.5.2:6804/7606 --> 192.168.5.8:0/29722 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.784010) v2 -- ?+0 0x23cbea00 con 0x23b1cc60 -98> 2014-11-03 11:37:57.784731 7f290882c700 1 -- 192.168.5.2:6805/7606 --> 192.168.5.8:0/29722 -- osd_ping(ping_reply e105950 stamp 2014-11-03 11:37:57.784010) v2 -- ?+0 0x30c1400 con 0x239f8c60 -97> 2014-11-03 11:37:57.788930 7f2912507700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.788929, event: sub_op_applied, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -96> 2014-11-03 11:37:57.789001 7f2912507700 5 -- op tracker -- seq: 3324, time: 2014-11-03 11:37:57.789001, event: done, op: osd_sub_op(client.7912219.0:1154965 6.24d d0cbe24d/rbd_data.49dd0e2ae8944a.0000000000000edb/head//6 [] v 105950'12187828 snapset=0=[]:[] snapc=0=[]) -95> 2014-11-03 11:37:57.789136 7f2912507700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.789136, event: sub_op_applied, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -94> 2014-11-03 11:37:57.789184 7f2912507700 5 -- op tracker -- seq: 3325, time: 2014-11-03 11:37:57.789184, event: done, op: osd_sub_op(client.7928471.0:2008410 6.323 ea2e6b23/rbd_data.78fa7f226987ae.00000000000002f9/head//6 [] v 105950'14134379 snapset=0=[]:[] snapc=0=[]) -93> 2014-11-03 11:37:57.804431 7f28f57e3700 1 -- 192.168.5.2:6803/7606 <== osd.40 192.168.5.3:6868/25675 44 ==== osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+9021 (4100671757 0 4028335510) 0x26943800 con 0x22fea580 -92> 2014-11-03 11:37:57.804465 7f28f57e3700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.804310, event: header_read, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -91> 2014-11-03 11:37:57.804489 7f28f57e3700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.804313, event: throttled, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -90> 2014-11-03 11:37:57.804544 7f28f57e3700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.804414, event: all_read, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -89> 2014-11-03 11:37:57.804561 7f28f57e3700 5 -- op tracker -- seq: 3334, time: 0.000000, event: dispatched, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -88> 2014-11-03 11:37:57.804641 7f28fe818700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.804641, event: reached_pg, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -87> 2014-11-03 11:37:57.804672 7f28fe818700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.804672, event: started, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -86> 2014-11-03 11:37:57.804812 7f28fe818700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.804812, event: started, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -85> 2014-11-03 11:37:57.804848 7f28fe818700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.804848, event: commit_queued_for_journal_write, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -84> 2014-11-03 11:37:57.838168 7f2914d0c700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.838168, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -83> 2014-11-03 11:37:57.838193 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- pg_trim(6.9d8 to 105871'7940028 e105950) v1 -- ?+0 0x22a08400 con 0x22fb7160 -82> 2014-11-03 11:37:57.838292 7f2914d0c700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.838292, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -81> 2014-11-03 11:37:57.838360 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- pg_trim(6.9d8 to 105871'7940028 e105950) v1 -- ?+0 0x2a347c00 con 0x22fe0420 -80> 2014-11-03 11:37:57.838400 7f2914d0c700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.838400, event: write_thread_in_journal_buffer, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -79> 2014-11-03 11:37:57.838472 7f2914d0c700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.838471, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -78> 2014-11-03 11:37:57.838548 7f2914d0c700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.838548, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -77> 2014-11-03 11:37:57.838600 7f2914d0c700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.838600, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -76> 2014-11-03 11:37:57.838960 7f291450b700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.838960, event: journaled_completion_queued, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -75> 2014-11-03 11:37:57.839011 7f291450b700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.839011, event: journaled_completion_queued, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -74> 2014-11-03 11:37:57.839049 7f2911d06700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.839048, event: commit_sent, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -73> 2014-11-03 11:37:57.839072 7f291450b700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.839072, event: journaled_completion_queued, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -72> 2014-11-03 11:37:57.839090 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] ondisk, result = 0) v2 -- ?+0 0x270d38c0 con 0x2301fb00 -71> 2014-11-03 11:37:57.839128 7f291450b700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.839128, event: journaled_completion_queued, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -70> 2014-11-03 11:37:57.839173 7f291450b700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.839173, event: journaled_completion_queued, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -69> 2014-11-03 11:37:57.839196 7f2911d06700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.839196, event: commit_sent, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -68> 2014-11-03 11:37:57.839215 7f291450b700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.839215, event: journaled_completion_queued, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -67> 2014-11-03 11:37:57.839254 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.6:6826/9268 -- osd_sub_op_reply(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] ondisk, result = 0) v2 -- ?+0 0x27803000 con 0x2308bdc0 -66> 2014-11-03 11:37:57.839304 7f2911d06700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.839303, event: op_commit, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -65> 2014-11-03 11:37:57.839355 7f2911d06700 1 -- 192.168.5.2:6802/7606 --> 192.168.9.3:0/1014472 -- osd_op_reply(11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] v105950'303250 uv303250 ondisk = 0) v6 -- ?+0 0x2650d2c0 con 0x191809a0 -64> 2014-11-03 11:37:57.839397 7f2911d06700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.839396, event: commit_sent, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -63> 2014-11-03 11:37:57.839443 7f2911d06700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.839443, event: commit_sent, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -62> 2014-11-03 11:37:57.839474 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] ondisk, result = 0) v2 -- ?+0 0x27803b00 con 0x2301fb00 -61> 2014-11-03 11:37:57.839514 7f2911d06700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.839514, event: commit_sent, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -60> 2014-11-03 11:37:57.839817 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6857/3307 -- osd_sub_op_reply(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] ondisk, result = 0) v2 -- ?+0 0x27804080 con 0x2301fb00 -59> 2014-11-03 11:37:57.839908 7f2911d06700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.839907, event: commit_sent, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -58> 2014-11-03 11:37:57.839970 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6868/25675 -- osd_sub_op_reply(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] ondisk, result = 0) v2 -- ?+0 0x239268c0 con 0x22fea580 -57> 2014-11-03 11:37:57.845293 7f2912507700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- MOSDPGPushReply(24.4f6 105950 [PushReplyOp(b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24)]) v2 -- ?+0 0x17ae8400 con 0x2308b840 -56> 2014-11-03 11:37:57.845643 7f2912507700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.845643, event: sub_op_applied, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -55> 2014-11-03 11:37:57.845743 7f2912507700 5 -- op tracker -- seq: 3327, time: 2014-11-03 11:37:57.845742, event: done, op: osd_sub_op(client.7916306.0:11671622 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027710 snapset=0=[]:[] snapc=0=[]) -54> 2014-11-03 11:37:57.846160 7f2912507700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.846159, event: sub_op_applied, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -53> 2014-11-03 11:37:57.846233 7f2912507700 5 -- op tracker -- seq: 3328, time: 2014-11-03 11:37:57.846233, event: done, op: osd_sub_op(client.7916306.0:11671644 24.737 b1313f37/rbd_data.5b64062ae8944a.0000000000000ac3/head//24 [] v 105950'258046 snapset=0=[]:[] snapc=0=[]) -52> 2014-11-03 11:37:57.846545 7f2912507700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.846545, event: op_applied, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -51> 2014-11-03 11:37:57.846643 7f2912507700 5 -- op tracker -- seq: 3329, time: 2014-11-03 11:37:57.846631, event: done, op: osd_op(client.7916306.0:11671652 rbd_data.5b64062ae8944a.0000000000000929 [set-alloc-hint object_size 4194304 write_size 4194304,write 3870720~8192] 24.31885348 ack+ondisk+write e105950) -50> 2014-11-03 11:37:57.846730 7f2912507700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.846730, event: sub_op_applied, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -49> 2014-11-03 11:37:57.846758 7f2912507700 5 -- op tracker -- seq: 3330, time: 2014-11-03 11:37:57.846758, event: done, op: osd_sub_op(client.7916306.0:11671623 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027711 snapset=0=[]:[] snapc=0=[]) -48> 2014-11-03 11:37:57.846972 7f2912507700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.846972, event: sub_op_applied, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -47> 2014-11-03 11:37:57.847021 7f2912507700 5 -- op tracker -- seq: 3333, time: 2014-11-03 11:37:57.847021, event: done, op: osd_sub_op(client.7916306.0:11671624 24.98a 7aac598a/rbd_data.5b64062ae8944a.0000000000000bba/head//24 [] v 105950'1027712 snapset=0=[]:[] snapc=0=[]) -46> 2014-11-03 11:37:57.847319 7f2912507700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.847319, event: sub_op_applied, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -45> 2014-11-03 11:37:57.847366 7f2912507700 5 -- op tracker -- seq: 3334, time: 2014-11-03 11:37:57.847366, event: done, op: osd_sub_op(client.7869971.0:1408223 24.5cf d5a0a5cf/rbd_data.4b4d672ae8944a.0000000000001100/head//24 [] v 105950'161803 snapset=0=[]:[] snapc=0=[]) -44> 2014-11-03 11:37:57.847571 7f28ef985700 1 -- 192.168.5.2:6803/7606 <== osd.102 192.168.5.8:6808/21153 107 ==== osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) v11 ==== 1404+0+4925 (3312040104 0 383899410) 0x2c690c00 con 0x2308b840 -43> 2014-11-03 11:37:57.847666 7f28ef985700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.847317, event: header_read, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -42> 2014-11-03 11:37:57.847700 7f28ef985700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.847319, event: throttled, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -41> 2014-11-03 11:37:57.847738 7f28ef985700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.847550, event: all_read, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -40> 2014-11-03 11:37:57.847755 7f28ef985700 5 -- op tracker -- seq: 3335, time: 0.000000, event: dispatched, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -39> 2014-11-03 11:37:57.847822 7f290201f700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.847821, event: reached_pg, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -38> 2014-11-03 11:37:57.847852 7f290201f700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.847852, event: started, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -37> 2014-11-03 11:37:57.847954 7f290201f700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.847954, event: started, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -36> 2014-11-03 11:37:57.847984 7f290201f700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.847984, event: commit_queued_for_journal_write, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -35> 2014-11-03 11:37:57.848146 7f2914d0c700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.848145, event: write_thread_in_journal_buffer, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -34> 2014-11-03 11:37:57.849373 7f291450b700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.849373, event: journaled_completion_queued, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -33> 2014-11-03 11:37:57.849459 7f2911d06700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.849458, event: commit_sent, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -32> 2014-11-03 11:37:57.849515 7f2911d06700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.8:6808/21153 -- osd_sub_op_reply(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] ondisk, result = 0) v2 -- ?+0 0x23925dc0 con 0x2308b840 -31> 2014-11-03 11:37:57.849838 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- MRecoveryReserve RELEASE pgid: 6.9d8, query_epoch: 105950 v2 -- ?+0 0x23576c00 con 0x22fb7160 -30> 2014-11-03 11:37:57.849877 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- MRecoveryReserve RELEASE pgid: 6.9d8, query_epoch: 105950 v2 -- ?+0 0x234dcc00 con 0x22fe0420 -29> 2014-11-03 11:37:57.849910 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105950'7950027 active+degraded snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] exit Started/Primary/Active/Recovering 1.896762 12 0.000152 -28> 2014-11-03 11:37:57.849965 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105950'7950027 active+degraded snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] enter Started/Primary/Active/Recovered -27> 2014-11-03 11:37:57.850030 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105950'7950027 active snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] exit Started/Primary/Active/Recovered 0.000065 0 0.000000 -26> 2014-11-03 11:37:57.850068 7f2903021700 5 osd.21 pg_epoch: 105950 pg[6.9d8( v 105950'7950028 (105871'7940027,105950'7950028] local-les=105948 n=4523 ec=164 les/c 105948/105943 105947/105947/105947) [21,93,49] r=0 lpr=105947 pi=105939-105946/2 crt=105946'7950023 mlcod 105950'7950027 active snaptrimq=[1907~1,193e~2,1941~6,1978~1,19ef~2,19f2~1,19f4~3,19fa~5,1a01~5]] enter Started/Primary/Active/Clean -25> 2014-11-03 11:37:57.850249 7f2912507700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.850249, event: sub_op_applied, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -24> 2014-11-03 11:37:57.850317 7f2912507700 5 -- op tracker -- seq: 3335, time: 2014-11-03 11:37:57.850317, event: done, op: osd_sub_op(client.7916306.0:11671587 24.4f6 b768bcf6/rbd_data.5b64062ae8944a.00000000000008b7/head//24 [] v 105950'217150 snapset=0=[]:[] snapc=0=[]) -23> 2014-11-03 11:37:57.850280 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.9e4( v 105946'113392 lc 105946'113391 (103622'109598,105946'113392] local-les=105948 n=88 ec=25000 les/c 105948/105943 105947/105947/105947) [21,112,33] r=0 lpr=105947 pi=105933-105946/4 crt=105946'113392 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[303~3,307~1]] exit Started/Primary/Active/WaitLocalRecoveryReserved 29.743468 2 0.000083 -22> 2014-11-03 11:37:57.850412 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.9e4( v 105946'113392 lc 105946'113391 (103622'109598,105946'113392] local-les=105948 n=88 ec=25000 les/c 105948/105943 105947/105947/105947) [21,112,33] r=0 lpr=105947 pi=105933-105946/4 crt=105946'113392 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[303~3,307~1]] enter Started/Primary/Active/WaitRemoteRecoveryReserved -21> 2014-11-03 11:37:57.850468 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.3:6839/29378 -- MRecoveryReserve REQUEST pgid: 24.9e4, query_epoch: 105950 v2 -- ?+0 0x187f4a00 con 0x22ffd000 -20> 2014-11-03 11:37:57.851709 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.33 192.168.5.3:6839/29378 8 ==== MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 v2 ==== 26+0+0 (124988897 0 0) 0x187f4a00 con 0x22ffd000 -19> 2014-11-03 11:37:57.851751 7f290b031700 5 -- op tracker -- seq: 3336, time: 2014-11-03 11:37:57.851582, event: header_read, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -18> 2014-11-03 11:37:57.851778 7f290b031700 5 -- op tracker -- seq: 3336, time: 2014-11-03 11:37:57.851584, event: throttled, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -17> 2014-11-03 11:37:57.851794 7f290b031700 5 -- op tracker -- seq: 3336, time: 2014-11-03 11:37:57.851653, event: all_read, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -16> 2014-11-03 11:37:57.851809 7f290b031700 5 -- op tracker -- seq: 3336, time: 2014-11-03 11:37:57.851745, event: dispatched, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -15> 2014-11-03 11:37:57.851826 7f290b031700 5 -- op tracker -- seq: 3336, time: 2014-11-03 11:37:57.851825, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -14> 2014-11-03 11:37:57.851888 7f290b031700 5 -- op tracker -- seq: 3336, time: 2014-11-03 11:37:57.851887, event: done, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -13> 2014-11-03 11:37:57.851953 7f2902820700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6806/24552 -- MRecoveryReserve REQUEST pgid: 24.9e4, query_epoch: 105950 v2 -- ?+0 0x187f4c00 con 0x22fb7000 -12> 2014-11-03 11:37:57.852887 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.4:6841/21769 -- pg_info(1 pgs e105950:6.9d8) v4 -- ?+0 0x2a347a00 con 0x22fb7160 -11> 2014-11-03 11:37:57.852941 7f2903021700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.7:6839/7095 -- pg_info(1 pgs e105950:6.9d8) v4 -- ?+0 0x1ae44e00 con 0x22fe0420 -10> 2014-11-03 11:37:57.853369 7f290b031700 1 -- 192.168.5.2:6803/7606 <== osd.112 192.168.5.9:6806/24552 41 ==== MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 v2 ==== 26+0+0 (124988897 0 0) 0x187f4c00 con 0x22fb7000 -9> 2014-11-03 11:37:57.853402 7f290b031700 5 -- op tracker -- seq: 3337, time: 2014-11-03 11:37:57.853157, event: header_read, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -8> 2014-11-03 11:37:57.853420 7f290b031700 5 -- op tracker -- seq: 3337, time: 2014-11-03 11:37:57.853158, event: throttled, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -7> 2014-11-03 11:37:57.853434 7f290b031700 5 -- op tracker -- seq: 3337, time: 2014-11-03 11:37:57.853302, event: all_read, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -6> 2014-11-03 11:37:57.853447 7f290b031700 5 -- op tracker -- seq: 3337, time: 2014-11-03 11:37:57.853398, event: dispatched, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -5> 2014-11-03 11:37:57.853461 7f290b031700 5 -- op tracker -- seq: 3337, time: 2014-11-03 11:37:57.853461, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -4> 2014-11-03 11:37:57.853495 7f290b031700 5 -- op tracker -- seq: 3337, time: 2014-11-03 11:37:57.853495, event: done, op: MRecoveryReserve GRANT pgid: 24.9e4, query_epoch: 105950 -3> 2014-11-03 11:37:57.853539 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.9e4( v 105946'113392 lc 105946'113391 (103622'109598,105946'113392] local-les=105948 n=88 ec=25000 les/c 105948/105943 105947/105947/105947) [21,112,33] r=0 lpr=105947 pi=105933-105946/4 crt=105946'113392 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[303~3,307~1]] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.003128 2 0.000319 -2> 2014-11-03 11:37:57.853585 7f2902820700 5 osd.21 pg_epoch: 105950 pg[24.9e4( v 105946'113392 lc 105946'113391 (103622'109598,105946'113392] local-les=105948 n=88 ec=25000 les/c 105948/105943 105947/105947/105947) [21,112,33] r=0 lpr=105947 pi=105933-105946/4 crt=105946'113392 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded m=1 snaptrimq=[303~3,307~1]] enter Started/Primary/Active/Recovering -1> 2014-11-03 11:37:57.853735 7f28fc814700 1 -- 192.168.5.2:6803/7606 --> 192.168.5.9:6806/24552 -- MOSDPGPull(24.9e4 105950 [PullOp(5abb99e4/rbd_data.5dd32f2ae8944a.0000000000000165/head//24, recovery_info: ObjectRecoveryInfo(5abb99e4/rbd_data.5dd32f2ae8944a.0000000000000165/head//24@105946'113392, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x229e7e00 con 0x22fb7000 0> 2014-11-03 11:37:57.856578 7f28fc013700 -1 *** Caught signal (Segmentation fault) ** 2014-11-03 11:37:57.871674 7f28fb812700 0 osd.21 105950 do_command r=0 ceph version 0.87-6-gdba7def (dba7defc623474ad17263c9fccfec60fe7a439f0) 1: /usr/bin/ceph-osd() [0x9b6725] 2: (()+0xfcb0) [0x7f291fc2acb0] 3: (ReplicatedPG::trim_object(hobject_t const&)+0x395) [0x811b55] 4: (ReplicatedPG::TrimmingObjects::react(ReplicatedPG::SnapTrim const&)+0x43e) [0x82b9be] 5: (boost::statechart::simple_state<ReplicatedPG::TrimmingObjects, ReplicatedPG::SnapTrimmer, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0xc0) [0x870ce0] 6: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_exception_translator>::process_queued_events()+0xfb) [0x85618b] 7: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x1e) [0x85633e] 8: (ReplicatedPG::snap_trimmer()+0x4f8) [0x7d5ef8] 9: (OSD::SnapTrimWQ::_process(PG*)+0x14) [0x673ab4] 10: (ThreadPool::worker(ThreadPool::WorkThread*)+0x48e) [0xa8fade] 11: (ThreadPool::WorkThread::entry()+0x10) [0xa92870] 12: (()+0x7e9a) [0x7f291fc22e9a] 13: (clone()+0x6d) [0x7f291e5ed31d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.21.log --- end dump of recent events --- 2014-11-03 11:37:57.871674 7f28fb812700 0 osd.21 105950 do_command r=0
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com