On Thu, Apr 09, 2015 at 06:43:45AM -0700, Gregory Farnum wrote:
> You can turn up debugging ("debug osd = 10" and "debug filestore = 10"
> are probably enough, or maybe 20 each) and see what comes out to get
> more information about why the threads are stuck.
>
> But just from the log my answer is the same as before, and now I don't
> trust that controller (or maybe its disks), regardless of what it's
> admitting to. ;)
> -Greg
>
Ran with osd and filestore debug both at 20; still nothing jumping out at me. Logfile attached as it got huge fairly quickly, but mostly seems to be the same extra lines. I tried running some test I/O on the drives in question to try and provoke some kind of problem, but they seem fine now...
2015-04-09 15:54:59.661982 7f829bf91700 5 osd.15 3814 tick
2015-04-09 15:54:59.662005 7f829bf91700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:54:59.662009 7f829bf91700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:01.350393 7f829bf91700 5 osd.15 3814 tick
2015-04-09 15:55:01.350421 7f829bf91700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:01.350426 7f829bf91700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:01.350763 7f828a025700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:01.350796 7f828a025700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:02.350505 7f829bf91700 5 osd.15 3814 tick
2015-04-09 15:55:02.350522 7f829bf91700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:02.350524 7f829bf91700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:02.361247 7f828671d700 -1 osd.15 3814 *** Got signal Terminated ***
2015-04-09 15:55:02.361318 7f828671d700 0 osd.15 3814 prepare_to_stop starting shutdown
2015-04-09 15:55:02.361332 7f828671d700 -1 osd.15 3814 shutdown
2015-04-09 15:55:02.361941 7f828671d700 10 osd.15 3814 recovery tp stopped
2015-04-09 15:55:02.362050 7f828671d700 10 osd.15 3814 osd tp stopped
2015-04-09 15:55:02.362106 7f828671d700 10 osd.15 3814 command tp stopped
2015-04-09 15:55:02.362187 7f828671d700 10 osd.15 3814 disk tp paused (new)
2015-04-09 15:55:02.362194 7f828671d700 10 osd.15 3814 stopping agent
2015-04-09 15:55:02.362209 7f828701f700 10 osd.15 3814 agent_entry finish
2015-04-09 15:55:02.362244 7f828671d700 10 osd.15 3814 reset_heartbeat_peers
2015-04-09 15:55:02.362387 7f828671d700 10 osd.15 3814 noting clean unmount in epoch 3814
2015-04-09 15:55:02.362396 7f828671d700 10 osd.15 3814 write_superblock sb(71783e7e-b856-4b96-a478-27191a56bf17 osd.15 e411ce75-c0f6-47f6-b58f-804a093fae81 e3814 [2527,3814] lci=[0,3814])
2015-04-09 15:55:02.362452 7f828671d700 5 filestore(/var/lib/ceph/osd/osd.15) queue_transactions new osr(default 0x2328b98)/0x2328b98
2015-04-09 15:55:02.362466 7f828671d700 10 journal op_submit_start 26
2015-04-09 15:55:02.362468 7f828671d700 5 filestore(/var/lib/ceph/osd/osd.15) queue_transactions (writeahead) 26 0x7f828671c8f0
2015-04-09 15:55:02.362472 7f828671d700 10 journal op_journal_transactions 26 0x7f828671c8f0
2015-04-09 15:55:02.362482 7f828671d700 5 journal submit_entry seq 26 len 492 (0x2451d20)
2015-04-09 15:55:02.362485 7f828671d700 30 journal XXX throttle take 492
2015-04-09 15:55:02.362498 7f828671d700 10 journal op_submit_finish 26
2015-04-09 15:55:02.362514 7f829878a700 20 journal write_thread_entry woke up
2015-04-09 15:55:02.362519 7f829878a700 10 journal room 15999168511 max_size 15999172608 pos 44724224 header.start 44724224 top 4096
2015-04-09 15:55:02.362523 7f829878a700 10 journal check_for_full at 44724224 : 4096 < 15999168511
2015-04-09 15:55:02.362525 7f829878a700 15 journal prepare_single_write 1 will write 44724224 : seq 26 len 492 -> 4096 (head 40 pre_pad 0 ebl 492 post_pad 3524 tail 40) (ebl alignment -1)
2015-04-09 15:55:02.362543 7f829878a700 20 journal prepare_multi_write queue_pos now 44728320
2015-04-09 15:55:02.362546 7f829878a700 15 journal do_aio_write writing 44724224~4096
2015-04-09 15:55:02.362559 7f829878a700 20 journal write_aio_bl 44724224~4096 seq 26
2015-04-09 15:55:02.362563 7f829878a700 20 journal write_aio_bl .. 44724224~4096 in 1
2015-04-09 15:55:02.362613 7f829878a700 5 journal put_throttle finished 1 ops and 492 bytes, now 0 ops and 0 bytes
2015-04-09 15:55:02.362618 7f829878a700 20 journal write_thread_entry going to sleep
2015-04-09 15:55:02.362641 7f8297f89700 20 journal write_finish_thread_entry waiting for aio(s)
2015-04-09 15:55:02.362702 7f8297f89700 10 journal write_finish_thread_entry aio 44724224~4096 done
2015-04-09 15:55:02.362712 7f8297f89700 20 journal check_aio_completion
2015-04-09 15:55:02.362715 7f8297f89700 20 journal check_aio_completion completed seq 26 44724224~4096
2015-04-09 15:55:02.362724 7f8297f89700 20 journal check_aio_completion queueing finishers through seq 26
2015-04-09 15:55:02.362735 7f8297f89700 10 journal queue_completions_thru seq 26 queueing seq 26 0x2451d20 lat 0.000243
2015-04-09 15:55:02.362754 7f8297f89700 20 journal write_finish_thread_entry sleeping
2015-04-09 15:55:02.362778 7f8297788700 5 filestore(/var/lib/ceph/osd/osd.15) _journaled_ahead 0x25a2000 seq 26 osr(default 0x2328b98) 0x7f828671c8f0
2015-04-09 15:55:02.362795 7f8297788700 5 filestore(/var/lib/ceph/osd/osd.15) queue_op 0x25a2000 seq 26 osr(default 0x2328b98) 486 bytes (queue has 1 ops and 486 bytes)
2015-04-09 15:55:02.362841 7f8296f87700 10 journal op_apply_start 26 open_ops 0 -> 1
2015-04-09 15:55:02.362860 7f8296f87700 5 filestore(/var/lib/ceph/osd/osd.15) _do_op 0x25a2000 seq 26 osr(default 0x2328b98)/0x2328b98 start
2015-04-09 15:55:02.362889 7f8296f87700 10 filestore(/var/lib/ceph/osd/osd.15) _do_transaction on 0x7f828671c8f0
2015-04-09 15:55:02.362927 7f8296f87700 15 filestore(/var/lib/ceph/osd/osd.15) write meta/23c2fcde/osd_superblock/0//-1 0~358
2015-04-09 15:55:02.363061 7f8296f87700 10 filestore(/var/lib/ceph/osd/osd.15) write meta/23c2fcde/osd_superblock/0//-1 0~358 = 358
2015-04-09 15:55:02.363078 7f8296f87700 10 journal op_apply_finish 26 open_ops 1 -> 0, max_applied_seq 25 -> 26
2015-04-09 15:55:02.363083 7f8296f87700 10 filestore(/var/lib/ceph/osd/osd.15) _do_op 0x25a2000 seq 26 r = 0, finisher 0x235e960 0
2015-04-09 15:55:02.363091 7f8296f87700 10 filestore(/var/lib/ceph/osd/osd.15) _finish_op 0x25a2000 seq 26 osr(default 0x2328b98)/0x2328b98
2015-04-09 15:55:02.363174 7f828671d700 10 osd.15 3814 syncing store
2015-04-09 15:55:02.363188 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) flush
2015-04-09 15:55:02.363192 7f828671d700 10 journal waiting for completions to empty
2015-04-09 15:55:02.363195 7f828671d700 10 journal flush waiting for finisher
2015-04-09 15:55:02.363198 7f828671d700 10 journal flush done
2015-04-09 15:55:02.363203 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) flush draining ondisk finisher
2015-04-09 15:55:02.363207 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) _flush_op_queue draining op tp
2015-04-09 15:55:02.363210 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) _flush_op_queue waiting for apply finisher
2015-04-09 15:55:02.363213 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) flush complete
2015-04-09 15:55:02.363223 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) start_sync
2015-04-09 15:55:02.363226 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) sync waiting
2015-04-09 15:55:02.363255 7f8298f8b700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 3.189084
2015-04-09 15:55:02.363268 7f8298f8b700 10 journal commit_start max_applied_seq 26, open_ops 0
2015-04-09 15:55:02.363272 7f8298f8b700 10 journal commit_start blocked, all open_ops have completed
2015-04-09 15:55:02.363275 7f8298f8b700 10 journal commit_start committing 26, still blocked
2015-04-09 15:55:02.363278 7f8298f8b700 10 journal commit_start
2015-04-09 15:55:02.363286 7f8298f8b700 15 filestore(/var/lib/ceph/osd/osd.15) sync_entry committing 26 sync_epoch 1
2015-04-09 15:55:02.363294 7f8298f8b700 10 journal commit_started committing 26, unblocking
2015-04-09 15:55:02.363305 7f8298f8b700 15 genericfilestorebackend(/var/lib/ceph/osd/osd.15) syncfs: doing a full sync (syncfs(2) if possible)
2015-04-09 15:55:02.365075 7f8298f8b700 10 filestore(/var/lib/ceph/osd/osd.15) sync_entry commit took 0.001793, interval was 3.190904
2015-04-09 15:55:02.365091 7f8298f8b700 10 journal commit_finish thru 26
2015-04-09 15:55:02.365098 7f8298f8b700 5 journal committed_thru 26 (last_committed_seq 25)
2015-04-09 15:55:02.365104 7f8298f8b700 10 journal header: block_size 4096 alignment 4096 max_size 15999172608
2015-04-09 15:55:02.365107 7f8298f8b700 10 journal header: start 44728320
2015-04-09 15:55:02.365110 7f8298f8b700 10 journal write_pos 44728320
2015-04-09 15:55:02.365112 7f8298f8b700 10 journal committed_thru done
2015-04-09 15:55:02.365135 7f8298f8b700 15 filestore(/var/lib/ceph/osd/osd.15) sync_entry committed to op_seq 26
2015-04-09 15:55:02.365153 7f8298f8b700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:02.365168 7f828671d700 10 filestore(/var/lib/ceph/osd/osd.15) sync done
2015-04-09 15:55:02.365172 7f828671d700 5 filestore(/var/lib/ceph/osd/osd.15) umount /var/lib/ceph/osd/osd.15
2015-04-09 15:55:02.365191 7f8298f8b700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry force_sync set
2015-04-09 15:55:02.365198 7f8298f8b700 10 journal commit_start max_applied_seq 26, open_ops 0
2015-04-09 15:55:02.365202 7f8298f8b700 10 journal commit_start blocked, all open_ops have completed
2015-04-09 15:55:02.365204 7f8298f8b700 10 journal commit_start nothing to do
2015-04-09 15:55:02.365207 7f8298f8b700 10 journal commit_start
2015-04-09 15:55:02.365539 7f828671d700 10 journal journal_stop
2015-04-09 15:55:02.365623 7f828671d700 1 journal close /dev/sdj4
2015-04-09 15:55:02.365674 7f829878a700 20 journal write_thread_entry woke up
2015-04-09 15:55:02.365686 7f829878a700 20 journal prepare_multi_write queue_pos now 44728320
2015-04-09 15:55:02.365682 7f8297f89700 10 journal write_finish_thread_entry exit
2015-04-09 15:55:02.365729 7f829878a700 15 journal do_write writing 44728320~0 + header
2015-04-09 15:55:02.365867 7f829878a700 20 journal do_write latency 0.000129
2015-04-09 15:55:02.365882 7f829878a700 20 journal do_write queueing finishers through seq 26
2015-04-09 15:55:02.365893 7f829878a700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2015-04-09 15:55:02.365901 7f829878a700 10 journal write_thread_entry finish
2015-04-09 15:55:02.366733 7f828671d700 10 osd.15 3814 Store synced
2015-04-09 15:55:02.367007 7f828671d700 1 -- 192.168.6.152:6805/32384 mark_down 0x245a9a0 -- 0x2350c80
2015-04-09 15:55:02.367032 7f828671d700 10 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=2 pgs=2396044 cs=1 l=1 c=0x245a9a0).unregister_pipe
2015-04-09 15:55:02.367063 7f828671d700 10 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=2 pgs=2396044 cs=1 l=1 c=0x245a9a0).stop
2015-04-09 15:55:02.367149 7f828681e700 2 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).reader couldn't read tag, (0) Success
2015-04-09 15:55:02.367187 7f828681e700 2 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).fault (0) Success
2015-04-09 15:55:02.367208 7f828681e700 10 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).fault already closed|closing
2015-04-09 15:55:02.367222 7f828681e700 10 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).reader done
2015-04-09 15:55:02.367307 7f82a2af7700 20 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).writer finishing
2015-04-09 15:55:02.367364 7f82a2af7700 10 -- 192.168.6.152:6805/32384 queue_reap 0x2350c80
2015-04-09 15:55:02.367380 7f82a2af7700 10 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).writer done
2015-04-09 15:55:02.367449 7f829978c700 10 -- 192.168.6.152:6805/32384 reaper
2015-04-09 15:55:02.367466 7f829978c700 10 -- 192.168.6.152:6805/32384 reaper reaping pipe 0x2350c80 192.168.6.146:6789/0
2015-04-09 15:55:02.367474 7f829978c700 10 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).discard_queue
2015-04-09 15:55:02.367485 7f829978c700 10 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).unregister_pipe - not registered
2015-04-09 15:55:02.367537 7f829978c700 20 -- 192.168.6.152:6805/32384 >> 192.168.6.146:6789/0 pipe(0x2350c80 sd=25 :41241 s=4 pgs=2396044 cs=1 l=1 c=0x245a9a0).join
2015-04-09 15:55:02.367561 7f829978c700 10 -- 192.168.6.152:6805/32384 reaper reaped pipe 0x2350c80 192.168.6.146:6789/0
2015-04-09 15:55:02.367571 7f829978c700 10 -- 192.168.6.152:6805/32384 reaper deleted pipe 0x2350c80
2015-04-09 15:55:02.367576 7f829978c700 10 -- 192.168.6.152:6805/32384 reaper done
2015-04-09 15:55:02.368551 7f828671d700 10 -- 192.168.6.152:6805/32384 shutdown 192.168.6.152:6805/32384
2015-04-09 15:55:02.368577 7f828671d700 1 -- 192.168.6.152:6805/32384 mark_down_all
2015-04-09 15:55:02.368597 7f828671d700 10 -- 192.168.6.152:6806/32384 shutdown 192.168.6.152:6806/32384
2015-04-09 15:55:02.368607 7f828671d700 1 -- 192.168.6.152:6806/32384 mark_down_all
2015-04-09 15:55:02.368619 7f828671d700 10 -- :/32384 shutdown :/32384
2015-04-09 15:55:02.368626 7f828671d700 1 -- :/32384 mark_down_all
2015-04-09 15:55:02.368636 7f828671d700 10 -- 192.168.6.152:6809/32384 shutdown 192.168.6.152:6809/32384
2015-04-09 15:55:02.368645 7f828671d700 1 -- 192.168.6.152:6809/32384 mark_down_all
2015-04-09 15:55:02.368660 7f828671d700 10 -- 192.168.6.152:6808/32384 shutdown 192.168.6.152:6808/32384
2015-04-09 15:55:02.368669 7f828671d700 1 -- 192.168.6.152:6808/32384 mark_down_all
2015-04-09 15:55:02.368687 7f828671d700 10 -- 192.168.6.152:6807/32384 shutdown 192.168.6.152:6807/32384
2015-04-09 15:55:02.368696 7f828671d700 1 -- 192.168.6.152:6807/32384 mark_down_all
2015-04-09 15:55:02.370034 7f82a2b31780 10 -- 192.168.6.152:6805/32384 wait: dispatch queue is stopped
2015-04-09 15:55:02.370053 7f82a2b31780 20 -- 192.168.6.152:6805/32384 wait: stopping accepter thread
2015-04-09 15:55:02.370062 7f82a2b31780 10 accepter.stop accepter
2015-04-09 15:55:02.370148 7f8292836700 20 accepter.accepter poll got 1
2015-04-09 15:55:02.370176 7f8292836700 20 accepter.accepter closing
2015-04-09 15:55:02.370218 7f8292836700 10 accepter.accepter stopping
2015-04-09 15:55:02.370333 7f82a2b31780 20 -- 192.168.6.152:6805/32384 wait: stopped accepter thread
2015-04-09 15:55:02.370347 7f82a2b31780 20 -- 192.168.6.152:6805/32384 wait: stopping reaper thread
2015-04-09 15:55:02.370374 7f829978c700 10 -- 192.168.6.152:6805/32384 reaper_entry done
2015-04-09 15:55:02.370448 7f82a2b31780 20 -- 192.168.6.152:6805/32384 wait: stopped reaper thread
2015-04-09 15:55:02.370459 7f82a2b31780 10 -- 192.168.6.152:6805/32384 wait: closing pipes
2015-04-09 15:55:02.370464 7f82a2b31780 10 -- 192.168.6.152:6805/32384 reaper
2015-04-09 15:55:02.370472 7f82a2b31780 10 -- 192.168.6.152:6805/32384 reaper done
2015-04-09 15:55:02.370476 7f82a2b31780 10 -- 192.168.6.152:6805/32384 wait: waiting for pipes to close
2015-04-09 15:55:02.370481 7f82a2b31780 10 -- 192.168.6.152:6805/32384 wait: done.
2015-04-09 15:55:02.370487 7f82a2b31780 1 -- 192.168.6.152:6805/32384 shutdown complete.
2015-04-09 15:55:02.370492 7f82a2b31780 10 -- :/32384 wait: waiting for dispatch queue
2015-04-09 15:55:02.370514 7f82a2b31780 10 -- :/32384 wait: dispatch queue is stopped
2015-04-09 15:55:02.370519 7f82a2b31780 20 -- :/32384 wait: stopping reaper thread
2015-04-09 15:55:02.370544 7f8299f8d700 10 -- :/32384 reaper_entry done
2015-04-09 15:55:02.370619 7f82a2b31780 20 -- :/32384 wait: stopped reaper thread
2015-04-09 15:55:02.370628 7f82a2b31780 10 -- :/32384 wait: closing pipes
2015-04-09 15:55:02.370631 7f82a2b31780 10 -- :/32384 reaper
2015-04-09 15:55:02.370633 7f82a2b31780 10 -- :/32384 reaper done
2015-04-09 15:55:02.370636 7f82a2b31780 10 -- :/32384 wait: waiting for pipes to close
2015-04-09 15:55:02.370639 7f82a2b31780 10 -- :/32384 wait: done.
2015-04-09 15:55:02.370645 7f82a2b31780 1 -- :/32384 shutdown complete.
2015-04-09 15:55:02.370649 7f82a2b31780 10 -- 192.168.6.152:6808/32384 wait: waiting for dispatch queue
2015-04-09 15:55:02.370680 7f82a2b31780 10 -- 192.168.6.152:6808/32384 wait: dispatch queue is stopped
2015-04-09 15:55:02.370689 7f82a2b31780 20 -- 192.168.6.152:6808/32384 wait: stopping accepter thread
2015-04-09 15:55:02.370696 7f82a2b31780 10 accepter.stop accepter
2015-04-09 15:55:02.370723 7f8290031700 20 accepter.accepter poll got 1
2015-04-09 15:55:02.370734 7f8290031700 20 accepter.accepter closing
2015-04-09 15:55:02.370759 7f8290031700 10 accepter.accepter stopping
2015-04-09 15:55:02.370813 7f82a2b31780 20 -- 192.168.6.152:6808/32384 wait: stopped accepter thread
2015-04-09 15:55:02.370826 7f82a2b31780 20 -- 192.168.6.152:6808/32384 wait: stopping reaper thread
2015-04-09 15:55:02.370887 7f829a78e700 10 -- 192.168.6.152:6808/32384 reaper_entry done
2015-04-09 15:55:02.370994 7f82a2b31780 20 -- 192.168.6.152:6808/32384 wait: stopped reaper thread
2015-04-09 15:55:02.371008 7f82a2b31780 10 -- 192.168.6.152:6808/32384 wait: closing pipes
2015-04-09 15:55:02.371015 7f82a2b31780 10 -- 192.168.6.152:6808/32384 reaper
2015-04-09 15:55:02.371022 7f82a2b31780 10 -- 192.168.6.152:6808/32384 reaper done
2015-04-09 15:55:02.371028 7f82a2b31780 10 -- 192.168.6.152:6808/32384 wait: waiting for pipes to close
2015-04-09 15:55:02.371034 7f82a2b31780 10 -- 192.168.6.152:6808/32384 wait: done.
2015-04-09 15:55:02.371039 7f82a2b31780 1 -- 192.168.6.152:6808/32384 shutdown complete.
2015-04-09 15:55:02.371045 7f82a2b31780 10 -- 192.168.6.152:6807/32384 wait: waiting for dispatch queue
2015-04-09 15:55:02.371069 7f82a2b31780 10 -- 192.168.6.152:6807/32384 wait: dispatch queue is stopped
2015-04-09 15:55:02.371082 7f82a2b31780 20 -- 192.168.6.152:6807/32384 wait: stopping accepter thread
2015-04-09 15:55:02.371090 7f82a2b31780 10 accepter.stop accepter
2015-04-09 15:55:02.371156 7f828f02f700 20 accepter.accepter poll got 1
2015-04-09 15:55:02.371168 7f828f02f700 20 accepter.accepter closing
2015-04-09 15:55:02.371185 7f828f02f700 10 accepter.accepter stopping
2015-04-09 15:55:02.371234 7f82a2b31780 20 -- 192.168.6.152:6807/32384 wait: stopped accepter thread
2015-04-09 15:55:02.371248 7f82a2b31780 20 -- 192.168.6.152:6807/32384 wait: stopping reaper thread
2015-04-09 15:55:02.371284 7f829c792700 10 -- 192.168.6.152:6807/32384 reaper_entry done
2015-04-09 15:55:02.371391 7f82a2b31780 20 -- 192.168.6.152:6807/32384 wait: stopped reaper thread
2015-04-09 15:55:02.371404 7f82a2b31780 10 -- 192.168.6.152:6807/32384 wait: closing pipes
2015-04-09 15:55:02.371410 7f82a2b31780 10 -- 192.168.6.152:6807/32384 reaper
2015-04-09 15:55:02.371416 7f82a2b31780 10 -- 192.168.6.152:6807/32384 reaper done
2015-04-09 15:55:02.371420 7f82a2b31780 10 -- 192.168.6.152:6807/32384 wait: waiting for pipes to close
2015-04-09 15:55:02.371425 7f82a2b31780 10 -- 192.168.6.152:6807/32384 wait: done.
2015-04-09 15:55:02.371428 7f82a2b31780 1 -- 192.168.6.152:6807/32384 shutdown complete.
2015-04-09 15:55:02.371433 7f82a2b31780 10 -- 192.168.6.152:6806/32384 wait: waiting for dispatch queue
2015-04-09 15:55:02.371469 7f82a2b31780 10 -- 192.168.6.152:6806/32384 wait: dispatch queue is stopped
2015-04-09 15:55:02.371479 7f82a2b31780 20 -- 192.168.6.152:6806/32384 wait: stopping accepter thread
2015-04-09 15:55:02.372423 7f82a2b31780 10 accepter.stop accepter
2015-04-09 15:55:02.372499 7f8291834700 20 accepter.accepter poll got 1
2015-04-09 15:55:02.372516 7f8291834700 20 accepter.accepter closing
2015-04-09 15:55:02.372549 7f8291834700 10 accepter.accepter stopping
2015-04-09 15:55:02.372612 7f82a2b31780 20 -- 192.168.6.152:6806/32384 wait: stopped accepter thread
2015-04-09 15:55:02.372622 7f82a2b31780 20 -- 192.168.6.152:6806/32384 wait: stopping reaper thread
2015-04-09 15:55:02.372656 7f829da7f700 10 -- 192.168.6.152:6806/32384 reaper_entry done
2015-04-09 15:55:02.372702 7f82a2b31780 20 -- 192.168.6.152:6806/32384 wait: stopped reaper thread
2015-04-09 15:55:02.372710 7f82a2b31780 10 -- 192.168.6.152:6806/32384 wait: closing pipes
2015-04-09 15:55:02.372714 7f82a2b31780 10 -- 192.168.6.152:6806/32384 reaper
2015-04-09 15:55:02.372718 7f82a2b31780 10 -- 192.168.6.152:6806/32384 reaper done
2015-04-09 15:55:02.372722 7f82a2b31780 10 -- 192.168.6.152:6806/32384 wait: waiting for pipes to close
2015-04-09 15:55:02.372726 7f82a2b31780 10 -- 192.168.6.152:6806/32384 wait: done.
2015-04-09 15:55:02.372730 7f82a2b31780 1 -- 192.168.6.152:6806/32384 shutdown complete.
2015-04-09 15:55:02.372735 7f82a2b31780 10 -- 192.168.6.152:6809/32384 wait: waiting for dispatch queue
2015-04-09 15:55:02.372755 7f82a2b31780 10 -- 192.168.6.152:6809/32384 wait: dispatch queue is stopped
2015-04-09 15:55:02.372761 7f82a2b31780 20 -- 192.168.6.152:6809/32384 wait: stopping accepter thread
2015-04-09 15:55:02.372765 7f82a2b31780 10 accepter.stop accepter
2015-04-09 15:55:02.372797 7f828e02d700 20 accepter.accepter poll got 1
2015-04-09 15:55:02.372806 7f828e02d700 20 accepter.accepter closing
2015-04-09 15:55:02.372818 7f828e02d700 10 accepter.accepter stopping
2015-04-09 15:55:02.372865 7f82a2b31780 20 -- 192.168.6.152:6809/32384 wait: stopped accepter thread
2015-04-09 15:55:02.372924 7f82a2b31780 20 -- 192.168.6.152:6809/32384 wait: stopping reaper thread
2015-04-09 15:55:02.372981 7f829d27e700 10 -- 192.168.6.152:6809/32384 reaper_entry done
2015-04-09 15:55:02.373047 7f82a2b31780 20 -- 192.168.6.152:6809/32384 wait: stopped reaper thread
2015-04-09 15:55:02.373055 7f82a2b31780 10 -- 192.168.6.152:6809/32384 wait: closing pipes
2015-04-09 15:55:02.373059 7f82a2b31780 10 -- 192.168.6.152:6809/32384 reaper
2015-04-09 15:55:02.373063 7f82a2b31780 10 -- 192.168.6.152:6809/32384 reaper done
2015-04-09 15:55:02.373067 7f82a2b31780 10 -- 192.168.6.152:6809/32384 wait: waiting for pipes to close
2015-04-09 15:55:02.373071 7f82a2b31780 10 -- 192.168.6.152:6809/32384 wait: done.
2015-04-09 15:55:02.373074 7f82a2b31780 1 -- 192.168.6.152:6809/32384 shutdown complete.
2015-04-09 15:55:05.980649 7f70b1507780 0 ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047), process ceph-osd, pid 19248
2015-04-09 15:55:05.986291 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) dump_stop
2015-04-09 15:55:05.986576 7f70b1507780 5 filestore(/var/lib/ceph/osd/osd.15) basedir /var/lib/ceph/osd/osd.15 journal /dev/sdj4
2015-04-09 15:55:05.986676 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) mount fsid is e411ce75-c0f6-47f6-b58f-804a093fae81
2015-04-09 15:55:05.986811 7f70b1507780 0 filestore(/var/lib/ceph/osd/osd.15) mount detected xfs (libxfs)
2015-04-09 15:55:05.986830 7f70b1507780 1 filestore(/var/lib/ceph/osd/osd.15) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs
2015-04-09 15:55:05.990305 7f70b1507780 0 genericfilestorebackend(/var/lib/ceph/osd/osd.15) detect_features: FIEMAP ioctl is supported and appears to work
2015-04-09 15:55:05.990332 7f70b1507780 0 genericfilestorebackend(/var/lib/ceph/osd/osd.15) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-04-09 15:55:05.991853 7f70b1507780 0 genericfilestorebackend(/var/lib/ceph/osd/osd.15) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2015-04-09 15:55:05.992047 7f70b1507780 0 xfsfilestorebackend(/var/lib/ceph/osd/osd.15) detect_feature: extsize is disabled by conf
2015-04-09 15:55:05.993276 7f70b1507780 5 filestore(/var/lib/ceph/osd/osd.15) mount op_seq is 26
2015-04-09 15:55:05.996541 7f70b1507780 20 filestore (init)dbobjectmap: seq is 1
2015-04-09 15:55:05.996578 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) open_journal at /dev/sdj4
2015-04-09 15:55:05.996647 7f70b1507780 0 filestore(/var/lib/ceph/osd/osd.15) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2015-04-09 15:55:05.996656 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) list_collections
2015-04-09 15:55:05.996919 7f70ac16c700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:06.001323 7f70b1507780 1 journal _open /dev/sdj4 fd 21: 15999172608 bytes, block size 4096 bytes, directio = 1, aio = 1
2015-04-09 15:55:06.006610 7f70b1507780 1 journal _open /dev/sdj4 fd 21: 15999172608 bytes, block size 4096 bytes, directio = 1, aio = 1
2015-04-09 15:55:06.007218 7f70b1507780 5 filestore(/var/lib/ceph/osd/osd.15) umount /var/lib/ceph/osd/osd.15
2015-04-09 15:55:06.007250 7f70ac16c700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry force_sync set
2015-04-09 15:55:06.007864 7f70b1507780 1 journal close /dev/sdj4
2015-04-09 15:55:06.009705 7f70b1507780 10 ErasureCodePluginSelectJerasure: SSE4 plugin
2015-04-09 15:55:06.018251 7f70b1507780 10 ErasureCodeJerasure: technique=reed_sol_van
2015-04-09 15:55:06.018273 7f70b1507780 10 ErasureCodeJerasure: k defaults to 7
2015-04-09 15:55:06.018277 7f70b1507780 10 ErasureCodeJerasure: m defaults to 3
2015-04-09 15:55:06.018280 7f70b1507780 10 ErasureCodeJerasure: w defaults to 8
2015-04-09 15:55:06.018312 7f70b1507780 10 load: jerasure
2015-04-09 15:55:06.019143 7f70b1507780 5 filestore(/var/lib/ceph/osd/osd.15) test_mount basedir /var/lib/ceph/osd/osd.15 journal /dev/sdj4
2015-04-09 15:55:06.020047 7f70b1507780 2 osd.15 0 mounting /var/lib/ceph/osd/osd.15 /dev/sdj4
2015-04-09 15:55:06.020057 7f70b1507780 5 filestore(/var/lib/ceph/osd/osd.15) basedir /var/lib/ceph/osd/osd.15 journal /dev/sdj4
2015-04-09 15:55:06.020074 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) mount fsid is e411ce75-c0f6-47f6-b58f-804a093fae81
2015-04-09 15:55:06.020109 7f70b1507780 0 filestore(/var/lib/ceph/osd/osd.15) mount detected xfs (libxfs)
2015-04-09 15:55:06.022026 7f70b1507780 0 genericfilestorebackend(/var/lib/ceph/osd/osd.15) detect_features: FIEMAP ioctl is supported and appears to work
2015-04-09 15:55:06.022035 7f70b1507780 0 genericfilestorebackend(/var/lib/ceph/osd/osd.15) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-04-09 15:55:06.022766 7f70b1507780 0 genericfilestorebackend(/var/lib/ceph/osd/osd.15) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2015-04-09 15:55:06.022843 7f70b1507780 0 xfsfilestorebackend(/var/lib/ceph/osd/osd.15) detect_feature: extsize is disabled by conf
2015-04-09 15:55:06.023665 7f70b1507780 5 filestore(/var/lib/ceph/osd/osd.15) mount op_seq is 26
2015-04-09 15:55:06.026127 7f70b1507780 20 filestore (init)dbobjectmap: seq is 1
2015-04-09 15:55:06.026176 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) open_journal at /dev/sdj4
2015-04-09 15:55:06.026213 7f70b1507780 0 filestore(/var/lib/ceph/osd/osd.15) mount: WRITEAHEAD journal mode explicitly enabled in conf
2015-04-09 15:55:06.026219 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) list_collections
2015-04-09 15:55:06.026582 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:06.031094 7f70b1507780 1 journal _open /dev/sdj4 fd 22: 15999172608 bytes, block size 4096 bytes, directio = 1, aio = 1
2015-04-09 15:55:06.034102 7f70b1507780 1 journal _open /dev/sdj4 fd 22: 15999172608 bytes, block size 4096 bytes, directio = 1, aio = 1
2015-04-09 15:55:06.034540 7f70b1507780 2 osd.15 0 boot
2015-04-09 15:55:06.034561 7f70b1507780 15 filestore(/var/lib/ceph/osd/osd.15) read meta/23c2fcde/osd_superblock/0//-1 0~0
2015-04-09 15:55:06.034681 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) FileStore::read meta/23c2fcde/osd_superblock/0//-1 0~358/358
2015-04-09 15:55:06.034704 7f70b1507780 10 osd.15 0 read_superblock sb(71783e7e-b856-4b96-a478-27191a56bf17 osd.15 e411ce75-c0f6-47f6-b58f-804a093fae81 e3814 [2527,3814] lci=[0,3814])
2015-04-09 15:55:06.034794 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) stat meta/16ef7597/infos/head//-1 = 0 (size 0)
2015-04-09 15:55:06.034838 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) stat meta/a468ec03/snapmapper/0//-1 = 0 (size 0)
2015-04-09 15:55:06.034853 7f70b1507780 10 open_all_classes
2015-04-09 15:55:06.034897 7f70b1507780 10 open_all_classes found statelog
2015-04-09 15:55:06.034914 7f70b1507780 10 _get_class adding new class name statelog 0x2db31e8
2015-04-09 15:55:06.034919 7f70b1507780 10 _load_class statelog from /usr/lib/rados-classes/libcls_statelog.so
2015-04-09 15:55:06.035180 7f70b1507780 10 register_class statelog status 3
2015-04-09 15:55:06.035183 7f70b1507780 10 register_cxx_method statelog.add flags 3 0x7f70ab24c290
2015-04-09 15:55:06.035188 7f70b1507780 10 register_cxx_method statelog.list flags 1 0x7f70ab24c5b0
2015-04-09 15:55:06.035191 7f70b1507780 10 register_cxx_method statelog.remove flags 3 0x7f70ab24b210
2015-04-09 15:55:06.035194 7f70b1507780 10 register_cxx_method statelog.check_state flags 1 0x7f70ab24ba00
2015-04-09 15:55:06.035197 7f70b1507780 10 _load_class statelog success
2015-04-09 15:55:06.035199 7f70b1507780 10 open_all_classes found hello
2015-04-09 15:55:06.035201 7f70b1507780 10 _get_class adding new class name hello 0x2db3108
2015-04-09 15:55:06.035204 7f70b1507780 10 _load_class hello from /usr/lib/rados-classes/libcls_hello.so
2015-04-09 15:55:06.035377 7f70b1507780 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello
2015-04-09 15:55:06.035387 7f70b1507780 10 register_class hello status 3
2015-04-09 15:55:06.035389 7f70b1507780 10 register_cxx_method hello.say_hello flags 1 0x7f70a2f51f50
2015-04-09 15:55:06.035393 7f70b1507780 10 register_cxx_method hello.record_hello flags 2 0x7f70a2f52f90
2015-04-09 15:55:06.035396 7f70b1507780 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7f70a2f52270
2015-04-09 15:55:06.035401 7f70b1507780 10 register_cxx_method hello.replay flags 1 0x7f70a2f51f20
2015-04-09 15:55:06.035404 7f70b1507780 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7f70a2f52510
2015-04-09 15:55:06.035407 7f70b1507780 10 register_cxx_method hello.bad_reader flags 2 0x7f70a2f51f10
2015-04-09 15:55:06.035409 7f70b1507780 10 register_cxx_method hello.bad_writer flags 1 0x7f70a2f51f00
2015-04-09 15:55:06.035411 7f70b1507780 10 _load_class hello success
2015-04-09 15:55:06.035412 7f70b1507780 10 open_all_classes found refcount
2015-04-09 15:55:06.035415 7f70b1507780 10 _get_class adding new class name refcount 0x2db3488
2015-04-09 15:55:06.035417 7f70b1507780 10 _load_class refcount from /usr/lib/rados-classes/libcls_refcount.so
2015-04-09 15:55:06.038170 7f70b1507780 10 register_class refcount status 3
2015-04-09 15:55:06.038172 7f70b1507780 10 register_cxx_method refcount.get flags 3 0x7f70a2c957d0
2015-04-09 15:55:06.038175 7f70b1507780 10 register_cxx_method refcount.put flags 3 0x7f70a2c946d0
2015-04-09 15:55:06.038180 7f70b1507780 10 register_cxx_method refcount.set flags 3 0x7f70a2c95260
2015-04-09 15:55:06.038182 7f70b1507780 10 register_cxx_method refcount.read flags 1 0x7f70a2c94d20
2015-04-09 15:55:06.038185 7f70b1507780 10 _load_class refcount success
2015-04-09 15:55:06.038198 7f70b1507780 10 open_all_classes found version
2015-04-09 15:55:06.038201 7f70b1507780 10 _get_class adding new class name version 0x2db33a8
2015-04-09 15:55:06.038203 7f70b1507780 10 _load_class version from /usr/lib/rados-classes/libcls_version.so
2015-04-09 15:55:06.038363 7f70b1507780 10 register_class version status 3
2015-04-09 15:55:06.038366 7f70b1507780 10 register_cxx_method version.set flags 3 0x7f70a29f22a0
2015-04-09 15:55:06.038370 7f70b1507780 10 register_cxx_method version.inc flags 3 0x7f70a29f2f30
2015-04-09 15:55:06.038374 7f70b1507780 10 register_cxx_method version.inc_conds flags 3 0x7f70a29f2f30
2015-04-09 15:55:06.038376 7f70b1507780 10 register_cxx_method version.read flags 1 0x7f70a29f2ae0
2015-04-09 15:55:06.038378 7f70b1507780 10 register_cxx_method version.check_conds flags 1 0x7f70a29f38d0
2015-04-09 15:55:06.038381 7f70b1507780 10 _load_class version success
2015-04-09 15:55:06.038382 7f70b1507780 10 open_all_classes found lock
2015-04-09 15:55:06.038385 7f70b1507780 10 _get_class adding new class name lock 0x2db32c8
2015-04-09 15:55:06.038387 7f70b1507780 10 _load_class lock from /usr/lib/rados-classes/libcls_lock.so
2015-04-09 15:55:06.038548 7f70b1507780 10 register_class lock status 3
2015-04-09 15:55:06.038552 7f70b1507780 10 register_cxx_method lock.lock flags 3 0x7f70a27e3d80
2015-04-09 15:55:06.038555 7f70b1507780 10 register_cxx_method lock.unlock flags 3 0x7f70a27e53f0
2015-04-09 15:55:06.038559 7f70b1507780 10 register_cxx_method lock.break_lock flags 3 0x7f70a27e5800
2015-04-09 15:55:06.038562 7f70b1507780 10 register_cxx_method lock.get_info flags 1 0x7f70a27e5bb0
2015-04-09 15:55:06.038573 7f70b1507780 10 register_cxx_method lock.list_locks flags 1 0x7f70a27e32f0
2015-04-09 15:55:06.038576 7f70b1507780 10 _load_class lock success
2015-04-09 15:55:06.038579 7f70b1507780 10 open_all_classes found rbd
2015-04-09 15:55:06.038582 7f70b1507780 10 _get_class adding new class name rbd 0x2db3808
2015-04-09 15:55:06.038585 7f70b1507780 10 _load_class rbd from /usr/lib/rados-classes/libcls_rbd.so
2015-04-09 15:55:06.038833 7f70b1507780 10 register_class rbd status 3
2015-04-09 15:55:06.038836 7f70b1507780 10 register_cxx_method rbd.create flags 3 0x7f70a25cae30
2015-04-09 15:55:06.038840 7f70b1507780 10 register_cxx_method rbd.get_features flags 1 0x7f70a25cc290
2015-04-09 15:55:06.038851 7f70b1507780 10 register_cxx_method rbd.get_size flags 1 0x7f70a25cc660
2015-04-09 15:55:06.038855 7f70b1507780 10 register_cxx_method rbd.set_size flags 3 0x7f70a25c9b60
2015-04-09 15:55:06.038858 7f70b1507780 10 register_cxx_method rbd.get_snapcontext flags 1 0x7f70a25cb8c0
2015-04-09 15:55:06.038862 7f70b1507780 10 register_cxx_method rbd.get_object_prefix flags 1 0x7f70a25c78a0
2015-04-09 15:55:06.038865 7f70b1507780 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7f70a25cd7a0
2015-04-09 15:55:06.038867 7f70b1507780 10 register_cxx_method rbd.snapshot_add flags 3 0x7f70a25cef00
2015-04-09 15:55:06.038869 7f70b1507780 10 register_cxx_method rbd.snapshot_remove flags 3 0x7f70a25cda80
2015-04-09 15:55:06.038872 7f70b1507780 10 register_cxx_method rbd.get_all_features flags 1 0x7f70a25c5750
2015-04-09 15:55:06.038874 7f70b1507780 10 register_cxx_method rbd.copyup flags 3 0x7f70a25c5680
2015-04-09 15:55:06.038877 7f70b1507780 10 register_cxx_method rbd.get_parent flags 1 0x7f70a25cd360
2015-04-09 15:55:06.038879 7f70b1507780 10 register_cxx_method rbd.set_parent flags 3 0x7f70a25ca300
2015-04-09 15:55:06.038889 7f70b1507780 10 register_cxx_method rbd.remove_parent flags 3 0x7f70a25c9180
2015-04-09 15:55:06.038893 7f70b1507780 10 register_cxx_method rbd.set_protection_status flags 3 0x7f70a25ccea0
2015-04-09 15:55:06.038905 7f70b1507780 10 register_cxx_method rbd.get_protection_status flags 1 0x7f70a25ccb40
2015-04-09 15:55:06.038910 7f70b1507780 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7f70a25c8b10
2015-04-09 15:55:06.038914 7f70b1507780 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7f70a25c9610
2015-04-09 15:55:06.038918 7f70b1507780 10 register_cxx_method rbd.add_child flags 3 0x7f70a25ce010
2015-04-09 15:55:06.038921 7f70b1507780 10 register_cxx_method rbd.remove_child flags 3 0x7f70a25ce8c0
2015-04-09 15:55:06.038926 7f70b1507780 10 register_cxx_method rbd.get_children flags 1 0x7f70a25ce580
2015-04-09 15:55:06.038928 7f70b1507780 10 register_cxx_method rbd.get_id flags 1 0x7f70a25c9360
2015-04-09 15:55:06.038931 7f70b1507780 10 register_cxx_method rbd.set_id flags 3 0x7f70a25cab30
2015-04-09 15:55:06.038933 7f70b1507780 10 register_cxx_method rbd.dir_get_id flags 1 0x7f70a25c7a70
2015-04-09 15:55:06.038937 7f70b1507780 10 register_cxx_method rbd.dir_get_name flags 1 0x7f70a25c7d60
2015-04-09 15:55:06.038941 7f70b1507780 10 register_cxx_method rbd.dir_list flags 1 0x7f70a25d0f00
2015-04-09 15:55:06.038944 7f70b1507780 10 register_cxx_method rbd.dir_add_image flags 3 0x7f70a25d0cd0
2015-04-09 15:55:06.038947 7f70b1507780 10 register_cxx_method rbd.dir_remove_image flags 3 0x7f70a25c8470
2015-04-09 15:55:06.038950 7f70b1507780 10 register_cxx_method rbd.dir_rename_image flags 3 0x7f70a25d09e0
2015-04-09 15:55:06.038953 7f70b1507780 10 register_cxx_method rbd.snap_list flags 1 0x7f70a25c65e0
2015-04-09 15:55:06.038957 7f70b1507780 10 register_cxx_method rbd.snap_add flags 3 0x7f70a25c7040
2015-04-09 15:55:06.038960 7f70b1507780 10 register_cxx_method rbd.snap_remove flags 3 0x7f70a25c68c0
2015-04-09 15:55:06.038971 7f70b1507780 10 _load_class rbd success
2015-04-09 15:55:06.038973 7f70b1507780 10 open_all_classes found replica_log
2015-04-09 15:55:06.038975 7f70b1507780 10 _get_class adding new class name replica_log 0x2db3728
2015-04-09 15:55:06.038978 7f70b1507780 10 _load_class replica_log from /usr/lib/rados-classes/libcls_replica_log.so
2015-04-09 15:55:06.039134 7f70b1507780 10 register_class replica_log status 3
2015-04-09 15:55:06.039136 7f70b1507780 10 register_cxx_method replica_log.set flags 3 0x7f70a23bcf50
2015-04-09 15:55:06.039140 7f70b1507780 10 register_cxx_method replica_log.get flags 1 0x7f70a23bbab0
2015-04-09 15:55:06.039142 7f70b1507780 10 register_cxx_method replica_log.delete flags 3 0x7f70a23bb4d0
2015-04-09 15:55:06.039145 7f70b1507780 10 _load_class replica_log success
2015-04-09 15:55:06.039147 7f70b1507780 10 open_all_classes found user
2015-04-09 15:55:06.039149 7f70b1507780 10 _get_class adding new class name user 0x2db3568
2015-04-09 15:55:06.039151 7f70b1507780 10 _load_class user from /usr/lib/rados-classes/libcls_user.so
2015-04-09 15:55:06.039300 7f70b1507780 10 register_class user status 3
2015-04-09 15:55:06.039302 7f70b1507780 10 register_cxx_method user.set_buckets_info flags 3 0x7f70a21acd60
2015-04-09 15:55:06.039306 7f70b1507780 10 register_cxx_method user.complete_stats_sync flags 3 0x7f70a21aa800
2015-04-09 15:55:06.039309 7f70b1507780 10 register_cxx_method user.remove_bucket flags 3 0x7f70a21abf50
2015-04-09 15:55:06.039312 7f70b1507780 10 register_cxx_method user.list_buckets flags 1 0x7f70a21ae240
2015-04-09 15:55:06.039324 7f70b1507780 10 register_cxx_method user.get_header flags 1 0x7f70a21aaee0
2015-04-09 15:55:06.039327 7f70b1507780 10 _load_class user success
2015-04-09 15:55:06.039330 7f70b1507780 10 open_all_classes found rgw
2015-04-09 15:55:06.039334 7f70b1507780 10 _get_class adding new class name rgw 0x2db3c68
2015-04-09 15:55:06.039336 7f70b1507780 10 _load_class rgw from /usr/lib/rados-classes/libcls_rgw.so
2015-04-09 15:55:06.041184 7f70b1507780 10 register_class rgw status 3
2015-04-09 15:55:06.041185 7f70b1507780 10 register_cxx_method rgw.bucket_init_index flags 3 0x7f70a1eda150
2015-04-09 15:55:06.041189 7f70b1507780 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7f70a1edcb70
2015-04-09 15:55:06.041191 7f70b1507780 10 register_cxx_method rgw.bucket_list flags 1 0x7f70a1ede640
2015-04-09 15:55:06.041194 7f70b1507780 10 register_cxx_method rgw.bucket_check_index flags 1 0x7f70a1edfdf0
2015-04-09 15:55:06.041196 7f70b1507780 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7f70a1ee0040
2015-04-09 15:55:06.041198 7f70b1507780 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7f70a1edde10
2015-04-09 15:55:06.041200 7f70b1507780 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7f70a1ee0340
2015-04-09 15:55:06.041202 7f70b1507780 10 register_cxx_method rgw.bi_log_list flags 1 0x7f70a1edc2f0
2015-04-09 15:55:06.041205 7f70b1507780 10 register_cxx_method rgw.bi_log_trim flags 3 0x7f70a1edc690
2015-04-09 15:55:06.041208 7f70b1507780 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7f70a1edd030
2015-04-09 15:55:06.041210 7f70b1507780 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7f70a1ee1560
2015-04-09 15:55:06.041212 7f70b1507780 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7f70a1edb320
2015-04-09 15:55:06.041222 7f70b1507780 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7f70a1edb580
2015-04-09 15:55:06.041230 7f70b1507780 10 register_cxx_method rgw.gc_set_entry flags 3 0x7f70a1ed9860
2015-04-09 15:55:06.041233 7f70b1507780 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7f70a1ed9aa0
2015-04-09 15:55:06.041235 7f70b1507780 10 register_cxx_method rgw.gc_list flags 1 0x7f70a1eda430
2015-04-09 15:55:06.041237 7f70b1507780 10 register_cxx_method rgw.gc_remove flags 3 0x7f70a1ed9da0
2015-04-09 15:55:06.041240 7f70b1507780 10 _load_class rgw success
2015-04-09 15:55:06.041242 7f70b1507780 10 open_all_classes found log
2015-04-09 15:55:06.041245 7f70b1507780 10 _get_class adding new class name log 0x2db3b88
2015-04-09 15:55:06.041247 7f70b1507780 10 _load_class log from /usr/lib/rados-classes/libcls_log.so
2015-04-09 15:55:06.041391 7f70b1507780 10 register_class log status 3
2015-04-09 15:55:06.041392 7f70b1507780 10 register_cxx_method log.add flags 3 0x7f70a1c2aec0
2015-04-09 15:55:06.041395 7f70b1507780 10 register_cxx_method log.list flags 1 0x7f70a1c2b8a0
2015-04-09 15:55:06.041397 7f70b1507780 10 register_cxx_method log.trim flags 3 0x7f70a1c2cf20
2015-04-09 15:55:06.041399 7f70b1507780 10 register_cxx_method log.info flags 1 0x7f70a1c2a9c0
2015-04-09 15:55:06.041401 7f70b1507780 10 _load_class log success
2015-04-09 15:55:06.041403 7f70b1507780 10 open_all_classes found kvs
2015-04-09 15:55:06.041405 7f70b1507780 10 _get_class adding new class name kvs 0x2db3aa8
2015-04-09 15:55:06.041407 7f70b1507780 10 _load_class kvs from /usr/lib/rados-classes/libcls_kvs.so
2015-04-09 15:55:06.041590 7f70b1507780 10 register_class kvs status 3
2015-04-09 15:55:06.041592 7f70b1507780 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7f70a1a14b70
2015-04-09 15:55:06.041601 7f70b1507780 10 register_cxx_method kvs.get_next_idata flags 1 0x7f70a1a16800
2015-04-09 15:55:06.041603 7f70b1507780 10 register_cxx_method kvs.get_prev_idata flags 1 0x7f70a1a177f0
2015-04-09 15:55:06.041605 7f70b1507780 10 register_cxx_method kvs.read_many flags 1 0x7f70a1a19cc0
2015-04-09 15:55:06.041607 7f70b1507780 10 register_cxx_method kvs.check_writable flags 3 0x7f70a1a14730
2015-04-09 15:55:06.041609 7f70b1507780 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7f70a1a14970
2015-04-09 15:55:06.041611 7f70b1507780 10 register_cxx_method kvs.omap_insert flags 2 0x7f70a1a1c0d0
2015-04-09 15:55:06.041614 7f70b1507780 10 register_cxx_method kvs.create_with_omap flags 2 0x7f70a1a1b810
2015-04-09 15:55:06.041616 7f70b1507780 10 register_cxx_method kvs.omap_remove flags 2 0x7f70a1a190a0
2015-04-09 15:55:06.041618 7f70b1507780 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7f70a1a1a8c0
2015-04-09 15:55:06.041620 7f70b1507780 10 _load_class kvs success
2015-04-09 15:55:06.041643 7f70b1507780 20 osd.15 0 get_map 3814 - loading and decoding 0x2dbf200
2015-04-09 15:55:06.041652 7f70b1507780 15 filestore(/var/lib/ceph/osd/osd.15) read meta/a080e38/osdmap.3814/0//-1 0~0
2015-04-09 15:55:06.041766 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) FileStore::read meta/a080e38/osdmap.3814/0//-1 0~38606/38606
2015-04-09 15:55:06.041772 7f70b1507780 10 osd.15 0 add_map_bl 3814 38606 bytes
2015-04-09 15:55:06.042376 7f70b1507780 0 osd.15 3814 crush map has features 1107558400, adjusting msgr requires for clients
2015-04-09 15:55:06.042384 7f70b1507780 0 osd.15 3814 crush map has features 1107558400 was 8705, adjusting msgr requires for mons
2015-04-09 15:55:06.042392 7f70b1507780 0 osd.15 3814 crush map has features 1107558400, adjusting msgr requires for osds
2015-04-09 15:55:06.042402 7f70b1507780 10 osd.15 3814 create_recoverystate_perf
2015-04-09 15:55:06.042416 7f70b1507780 0 osd.15 3814 load_pgs
2015-04-09 15:55:06.042421 7f70b1507780 10 filestore(/var/lib/ceph/osd/osd.15) list_collections
2015-04-09 15:55:06.042466 7f70b1507780 10 osd.15 3814 load_pgs ignoring unrecognized meta
2015-04-09 15:55:06.042469 7f70b1507780 0 osd.15 3814 load_pgs opened 0 pgs
2015-04-09 15:55:06.042475 7f70b1507780 10 osd.15 3814 build_past_intervals_parallel nothing to build
2015-04-09 15:55:06.042480 7f70b1507780 2 osd.15 3814 superblock: i am osd.15
2015-04-09 15:55:06.042482 7f70b1507780 10 osd.15 3814 create_logger
2015-04-09 15:55:06.045312 7f70b1507780 10 osd.15 3814 set_disk_tp_priority class priority -1
2015-04-09 15:55:06.045330 7f70b1507780 -1 osd.15 3814 set_disk_tp_priority(22) Invalid argument: osd_disk_thread_ioprio_class is but only the following values are allowed: idle, be or rt
2015-04-09 15:55:06.045547 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:06.045577 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:06.045791 7f70959f7700 10 osd.15 3814 agent_entry start
2015-04-09 15:55:06.045805 7f70959f7700 20 osd.15 3814 agent_entry empty queue
2015-04-09 15:55:06.046631 7f70b14cf700 10 osd.15 3814 OSD::ms_get_authorizer type=mon
2015-04-09 15:55:06.047124 7f70a1a0f700 10 osd.15 3814 ms_handle_connect on mon
2015-04-09 15:55:06.047146 7f70a1a0f700 10 osd.15 3814 send_alive up_thru currently 2773 want 0
2015-04-09 15:55:06.047153 7f70a1a0f700 20 osd.15 3814 send_pg_stats
2015-04-09 15:55:06.053572 7f70a1a0f700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:06.053577 7f70a1a0f700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:06.053579 7f70a1a0f700 20 osd.15 3814 _dispatch 0x2d85240 osd_map(3814..3814 src has 2527..3814) v3
2015-04-09 15:55:06.053598 7f70a1a0f700 0 osd.15 3814 ignoring osdmap until we have initialized
2015-04-09 15:55:06.053601 7f70a1a0f700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:06.053603 7f70a1a0f700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:06.053940 7f70a1a0f700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:06.053944 7f70a1a0f700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:06.053947 7f70a1a0f700 20 osd.15 3814 _dispatch 0x2ec8d80 osd_map(3814..3814 src has 2527..3814) v3
2015-04-09 15:55:06.053952 7f70a1a0f700 0 osd.15 3814 ignoring osdmap until we have initialized
2015-04-09 15:55:06.053955 7f70a1a0f700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:06.053957 7f70a1a0f700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:06.054119 7f70b1507780 10 osd.15 3814 ensuring pgs have consumed prior maps
2015-04-09 15:55:06.054122 7f70b1507780 7 osd.15 3814 consume_map version 3814
2015-04-09 15:55:06.054128 7f70b1507780 0 osd.15 3814 done with init, starting boot process
2015-04-09 15:55:06.054130 7f70b1507780 10 osd.15 3814 start_boot - have maps 2527..3814
2015-04-09 15:55:06.055613 7f709ba03700 10 osd.15 3814 _maybe_boot mon has osdmaps 2527..3814
2015-04-09 15:55:06.055654 7f709ba03700 10 osd.15 3814 _send_boot
2015-04-09 15:55:06.055660 7f709ba03700 10 osd.15 3814 assuming cluster_addr ip matches client_addr
2015-04-09 15:55:06.055663 7f709ba03700 10 osd.15 3814 assuming hb_back_addr ip matches cluster_addr
2015-04-09 15:55:06.055679 7f709ba03700 10 osd.15 3814 client_addr 192.168.6.152:6805/19248, cluster_addr 192.168.6.152:6806/19248, hb_back_addr 192.168.6.152:6807/19248, hb_front_addr 192.168.6.152:6808/19248
2015-04-09 15:55:06.096297 7f709ba03700 10 osd.15 3814 _collect_metadata {arch=x86_64,back_addr=192.168.6.152:6806/19248,ceph_version=ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047),cpu=AMD Opteron(tm) Processor 6320,distro=Ubuntu,distro_codename=precise,distro_description=Ubuntu 12.04.5 LTS,distro_version=12.04,front_addr=192.168.6.152:6805/19248,hb_back_addr=192.168.6.152:6807/19248,hb_front_addr=192.168.6.152:6808/19248,hostname=ceph02,kernel_description=#62~precise1-Ubuntu SMP Mon Aug 18 14:52:04 UTC 2014,kernel_version=3.13.0-35-generic,mem_swap_kb=7761400,mem_total_kb=36960444,os=Linux,osd_data=/var/lib/ceph/osd/osd.15,osd_journal=/dev/sdj4}
2015-04-09 15:55:11.200924 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:11.200959 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:11.201034 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.174450
2015-04-09 15:55:11.201062 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:12.056379 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:12.056422 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:12.056433 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:13.056640 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:13.056676 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:13.056678 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:14.056897 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:14.056938 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:14.056944 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:14.781804 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:14.782018 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:15.150042 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:15.150069 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:15.150077 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:16.337234 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.136169
2015-04-09 15:55:16.337278 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:16.337272 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:16.337297 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:16.337300 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:17.337383 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:17.337411 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:17.337413 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:18.337505 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:18.337537 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:18.337542 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:18.888083 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:18.888138 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:19.352739 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:19.352770 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:19.352781 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:20.396801 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:20.396827 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:20.396831 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:20.610921 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:20.610950 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:21.487204 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:21.487251 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:21.487257 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:21.487255 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.149975
2015-04-09 15:55:21.487283 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:22.654639 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:22.654666 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:22.654671 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:23.950117 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:23.950342 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:23.950348 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:24.950441 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:24.950477 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:24.950482 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:26.351952 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:26.351981 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:26.351987 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:26.488294 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.001009
2015-04-09 15:55:26.488349 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:26.514481 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:26.514515 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:27.352138 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:27.352164 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:27.352168 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:27.710874 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:27.710902 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:28.527591 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:28.527612 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:28.527616 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:29.416762 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:29.416794 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:29.614816 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:29.614841 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:29.614847 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:32.495336 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 6.006981
2015-04-09 15:55:32.495383 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:32.495548 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:32.495563 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:32.495568 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:34.215324 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:34.215353 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:34.215378 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:34.215729 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:34.215757 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:34.735943 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:34.735972 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:35.260415 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:35.260451 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:35.260456 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:36.265375 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:36.265395 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:36.265400 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:37.265522 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:37.265549 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:37.265555 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:39.337111 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:39.337133 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:39.337140 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:39.337593 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 6.842209
2015-04-09 15:55:39.337633 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:40.337225 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:40.337253 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:40.337268 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:40.798849 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:40.798879 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:41.346316 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:41.346340 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:41.346346 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:42.362235 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:42.362259 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:42.362264 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:43.125576 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:43.125623 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:43.368379 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:43.368404 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:43.368409 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:44.338858 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.001224
2015-04-09 15:55:44.338896 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:44.543607 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:44.543632 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:44.543636 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:45.543942 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:45.543975 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:45.543979 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:46.547279 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:46.547297 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:46.547303 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:47.552509 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:47.552534 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:47.552590 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:47.552595 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:47.552597 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:48.200297 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:48.200337 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:48.552706 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:48.552741 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:48.552747 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:50.304469 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.965572
2015-04-09 15:55:50.304511 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:50.305474 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:50.305504 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:50.305508 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:51.308438 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:51.308464 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:51.308469 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:52.308511 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:52.308548 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:52.308846 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:52.308863 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:52.308867 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:53.309036 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:53.309062 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:53.309070 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:54.015130 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:54.015171 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:54.309148 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:54.309172 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:54.309177 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:55.311576 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.007062
2015-04-09 15:55:55.311639 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:55:55.311767 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:55.311785 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:55.311790 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:56.957272 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:56.957297 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:56.957301 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:57.967459 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:57.967507 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:57.967513 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:58.135794 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:58.135826 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:58.967777 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:58.967804 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:58.967810 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:55:59.236088 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:59.236119 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:55:59.967905 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:55:59.967934 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:55:59.967943 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:00.311716 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000077
2015-04-09 15:56:00.311764 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:00.968023 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:00.968055 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:00.968060 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:02.105260 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:02.105286 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:02.105290 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:03.105543 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:03.105568 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:03.105572 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:04.106005 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:04.106032 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:04.106038 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:04.536274 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:04.536322 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:05.139456 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:05.139482 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:05.139487 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:05.311840 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000075
2015-04-09 15:56:05.311916 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:06.139823 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:06.139851 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:06.139854 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:07.150676 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:07.150712 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:07.150718 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:08.224373 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:08.224410 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:08.224416 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:10.258942 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:10.258965 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:10.258976 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:10.258982 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:10.258986 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:10.311983 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000067
2015-04-09 15:56:10.312015 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:11.262657 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:11.262859 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:11.262863 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:12.263038 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:12.263279 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:12.263293 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:13.266520 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:13.266543 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:13.266549 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:14.266662 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:14.266687 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:14.266692 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:15.687525 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.375509
2015-04-09 15:56:15.687567 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:15.687652 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:15.687664 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:15.687667 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:16.692856 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:16.692880 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:16.692884 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:16.692918 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:16.692931 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:17.699252 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:17.699274 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:17.699280 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:18.703002 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:18.703025 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:18.703031 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:18.703068 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:18.703080 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:19.212754 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:19.212777 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:20.216200 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:20.216222 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:20.216227 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:20.835748 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.148180
2015-04-09 15:56:20.835767 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:21.220285 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:21.220312 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:21.220318 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:21.220497 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:21.220521 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:22.220742 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:22.220770 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:22.220774 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:23.686541 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:23.686563 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:23.686567 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:23.686650 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:23.686670 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:24.709839 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:24.709870 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:24.709875 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:26.337352 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.501583
2015-04-09 15:56:26.337397 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:26.337763 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:26.337783 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:26.337787 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:27.337861 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:27.337885 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:27.337890 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:28.338947 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:28.338964 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:28.338968 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:29.339607 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:29.339633 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:29.339637 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:29.340957 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:29.340986 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:30.339711 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:30.339727 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:30.339732 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:33.548176 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:33.548211 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:33.548216 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:33.548373 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 7.210976
2015-04-09 15:56:33.548396 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:33.548519 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:33.548544 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:34.548635 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:34.548659 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:34.548667 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:35.551876 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:35.551895 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:35.551900 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:36.556697 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:36.556720 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:36.556725 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:37.556824 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:37.556849 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:37.556854 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:40.299338 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:40.299370 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:40.299792 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 6.751395
2015-04-09 15:56:40.299821 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:40.301011 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:40.301039 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:40.301044 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:41.302690 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:41.302711 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:41.302716 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:42.306967 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:42.306985 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:42.306989 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:42.307085 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:42.307097 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:43.311062 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:43.311082 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:43.311087 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:44.314850 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:44.314872 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:44.314878 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:44.314968 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:44.314980 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:45.342625 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.042802
2015-04-09 15:56:45.342667 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:45.343022 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:45.343045 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:45.343049 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:46.475076 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:46.475096 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:46.475100 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:47.644943 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:47.644967 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:47.644972 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:48.647187 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:48.647220 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:48.647246 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:49.615271 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:49.615313 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:49.647337 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:49.647367 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:49.647373 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:50.536365 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.193696
2015-04-09 15:56:50.536411 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:51.327510 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:51.327538 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:51.327543 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:52.338827 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:52.338904 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:52.338906 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:53.339131 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:53.339154 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:53.339159 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:55.222393 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:55.222410 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:55.222415 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:56:55.953116 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.416704
2015-04-09 15:56:55.953153 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:56:55.953185 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:55.953206 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:56:56.929778 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:56:56.929804 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:56:56.929809 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:01.469768 7f70a515e700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f70a515e700' had timed out after 4
2015-04-09 15:57:01.469805 7f709aa01700 1 heartbeat_map reset_timeout 'OSD::op_tp thread 0x7f709aa01700' had timed out after 4
2015-04-09 15:57:01.469832 7f70999ff700 1 heartbeat_map reset_timeout 'OSD::disk_tp thread 0x7f70999ff700' had timed out after 4
2015-04-09 15:57:01.469837 7f70ad96f700 1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f709a200700' had timed out after 4
2015-04-09 15:57:01.469865 7f70ad96f700 1 heartbeat_map is_healthy 'OSD::command_tp thread 0x7f70991fe700' had timed out after 4
2015-04-09 15:57:01.469870 7f70ad96f700 1 heartbeat_map is_healthy 'OSD::op_tp thread 0x7f709b202700' had timed out after 4
2015-04-09 15:57:01.469873 7f70ad96f700 1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f70a595f700' had timed out after 4
2015-04-09 15:57:01.469906 7f709a200700 1 heartbeat_map reset_timeout 'OSD::recovery_tp thread 0x7f709a200700' had timed out after 4
2015-04-09 15:57:01.470266 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.517113
2015-04-09 15:57:01.470285 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:01.470301 7f70a595f700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f70a595f700' had timed out after 4
2015-04-09 15:57:01.470793 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:01.470814 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:01.470819 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:01.470885 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:01.470904 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:01.471191 7f70991fe700 1 heartbeat_map reset_timeout 'OSD::command_tp thread 0x7f70991fe700' had timed out after 4
2015-04-09 15:57:01.471227 7f709b202700 1 heartbeat_map reset_timeout 'OSD::op_tp thread 0x7f709b202700' had timed out after 4
2015-04-09 15:57:03.565914 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:03.565930 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:03.565933 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:03.940031 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:03.940068 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:04.566147 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:04.566190 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:04.566194 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:05.572496 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:05.572511 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:05.572514 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:06.494582 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.024297
2015-04-09 15:57:06.494623 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:06.572588 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:06.572608 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:06.572613 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:07.572738 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:07.572762 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:07.572768 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:10.256037 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:10.256055 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:10.256060 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:10.256141 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:10.256175 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:11.259936 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:11.259955 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:11.259964 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:11.357142 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:11.357179 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:11.636706 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.142082
2015-04-09 15:57:11.636732 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:12.263133 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:12.263153 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:12.263156 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:13.270667 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:13.270686 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:13.270691 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:13.270743 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:13.270764 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:14.275334 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:14.275354 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:14.275359 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:15.339171 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:15.339204 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:15.339207 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:16.339283 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:16.339310 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:16.339315 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:16.636812 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000078
2015-04-09 15:57:16.636856 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:17.552708 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:17.552725 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:17.552728 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:18.555127 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:18.555161 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:18.555166 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:19.508448 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:19.508480 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:20.133526 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:20.133545 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:20.133549 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:21.508738 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:21.508759 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:21.508763 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:21.636950 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000093
2015-04-09 15:57:21.636986 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:21.810552 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:21.810599 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:22.509243 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:22.509260 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:22.509268 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:22.910738 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:22.910770 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:23.567887 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:23.567910 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:23.567915 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:25.855207 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:25.855408 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:25.855414 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:27.586734 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:27.586751 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:27.586756 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:27.586768 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.949782
2015-04-09 15:57:27.586840 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:28.934166 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:28.934206 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:28.934323 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:28.934342 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:28.934347 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:30.348116 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:30.348316 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:30.348319 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:31.915308 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:31.915326 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:31.915331 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:31.916672 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:31.916692 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:32.890301 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.303457
2015-04-09 15:57:32.890719 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:32.917152 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:32.917173 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:32.917178 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:34.328076 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:34.328097 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:34.328101 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:35.356926 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:35.356944 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:35.356949 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:36.357153 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:36.357174 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:36.357181 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:37.174024 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:37.174058 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:37.532391 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:37.532418 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:37.532424 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:38.338647 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.447927
2015-04-09 15:57:38.338699 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:38.794432 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:38.794451 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:38.794456 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:39.893345 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:39.893371 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:39.893376 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:40.935416 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:40.935448 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:40.935453 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:41.938540 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:41.938610 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:41.938615 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:41.942122 7f70a1a0f700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:41.942143 7f70a1a0f700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:41.942147 7f70a1a0f700 20 osd.15 3814 _dispatch 0x2ec86c0 osd_map(3814..3814 src has 2527..3814) v3
2015-04-09 15:57:41.943084 7f70a1a0f700 3 osd.15 3814 handle_osd_map epochs [3814,3814], i have 3814, src has [2527,3814]
2015-04-09 15:57:41.943105 7f70a1a0f700 10 osd.15 3814 no new maps here, dropping
2015-04-09 15:57:41.943111 7f70a1a0f700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:41.943114 7f70a1a0f700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:42.938701 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:42.938726 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:42.938731 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:42.938769 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:42.938790 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:43.671002 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.332301
2015-04-09 15:57:43.671043 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:43.938809 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:43.938835 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:43.938839 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:44.942442 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:44.942470 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:44.942475 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:46.469759 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:46.469782 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:46.469787 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:46.470006 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:46.470021 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:47.539974 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:47.539999 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:47.540004 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:48.542895 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:48.542920 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:48.542925 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:48.724972 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.053928
2015-04-09 15:57:48.725009 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:49.730585 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:49.730614 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:49.730619 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:50.728862 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:50.728895 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:50.730740 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:50.730766 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:50.730772 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:51.728843 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:51.728878 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:51.730910 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:51.730935 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:51.730942 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:52.731114 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:52.731149 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:52.731155 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:54.395248 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.670238
2015-04-09 15:57:54.395284 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:57:54.395328 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:54.395352 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:54.395474 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:54.395485 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:54.395487 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:57.066958 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:57.066982 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:57.066988 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:58.356922 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:58.356947 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:58.356960 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:59.356639 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:59.356664 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:57:59.357036 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:57:59.357063 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:57:59.357072 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:57:59.425430 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.030144
2015-04-09 15:57:59.425478 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:00.537777 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:00.537803 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:00.537809 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:00.537821 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:00.537846 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:01.338629 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:01.338661 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:02.339292 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:02.339301 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:02.339305 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:03.669935 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:03.669958 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:03.669963 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:04.672450 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.246970
2015-04-09 15:58:04.672487 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:04.673425 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:04.673452 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:04.673457 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:05.560768 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:05.560808 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:05.676675 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:05.676707 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:05.676711 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:06.677041 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:06.677080 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:06.677085 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:07.677379 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:07.677399 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:07.677402 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:08.154589 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:08.154621 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:09.611301 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:09.611321 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:09.611326 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:10.306898 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.634409
2015-04-09 15:58:10.306946 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:10.307080 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:10.307100 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:11.311166 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:11.311199 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:11.312551 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:11.312564 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:11.312566 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:12.314731 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:12.314741 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:12.314743 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:13.318790 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:13.318809 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:13.318813 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:14.323033 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:14.323052 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:14.323057 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:15.600428 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.293482
2015-04-09 15:58:15.600469 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:15.600690 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:15.600706 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:15.600733 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:15.600746 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:15.600750 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:16.600971 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:16.600995 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:16.601001 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:16.962324 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:16.962389 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:17.606936 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:17.606965 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:17.606972 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:18.611634 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:18.611660 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:18.611664 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:20.359920 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:20.359953 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:20.359959 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:20.528371 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:20.528413 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:21.366656 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.766186
2015-04-09 15:58:21.366698 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:21.366717 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:21.366725 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:21.366730 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:22.383442 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:22.383468 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:22.383474 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:23.588196 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:23.588216 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:23.588220 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:24.338707 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:24.338741 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:25.339024 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:25.339044 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:25.339049 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:27.581140 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:27.581168 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:27.581234 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 6.214536
2015-04-09 15:58:27.581266 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:27.581287 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:27.581295 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:27.581299 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:28.581373 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:28.581394 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:28.581399 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:32.984910 7f709a200700 1 heartbeat_map reset_timeout 'OSD::recovery_tp thread 0x7f709a200700' had timed out after 4
2015-04-09 15:58:32.985072 7f709aa01700 1 heartbeat_map reset_timeout 'OSD::op_tp thread 0x7f709aa01700' had timed out after 4
2015-04-09 15:58:32.985110 7f70a515e700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f70a515e700' had timed out after 4
2015-04-09 15:58:32.985151 7f709b202700 1 heartbeat_map reset_timeout 'OSD::op_tp thread 0x7f709b202700' had timed out after 4
2015-04-09 15:58:32.985451 7f70a595f700 1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f70a595f700' had timed out after 4
2015-04-09 15:58:32.985538 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.404273
2015-04-09 15:58:32.985553 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:32.985567 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:32.985573 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:32.985576 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:32.985594 7f70999ff700 1 heartbeat_map reset_timeout 'OSD::disk_tp thread 0x7f70999ff700' had timed out after 4
2015-04-09 15:58:32.985949 7f70ad96f700 1 heartbeat_map is_healthy 'OSD::command_tp thread 0x7f70991fe700' had timed out after 4
2015-04-09 15:58:32.986008 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:32.986018 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:32.987651 7f70991fe700 1 heartbeat_map reset_timeout 'OSD::command_tp thread 0x7f70991fe700' had timed out after 4
2015-04-09 15:58:34.015811 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:34.015837 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:34.015841 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:35.027925 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:35.027963 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:35.027968 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:36.172359 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:36.172388 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:36.172394 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:37.176270 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:37.176293 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:37.176299 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:37.176590 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:37.176624 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:38.612322 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.626769
2015-04-09 15:58:38.612362 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:38.612388 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:38.612397 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:38.612401 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:40.342629 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:40.342653 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:40.342657 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:41.347137 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:41.347160 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:41.347165 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:41.347167 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:41.347188 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:42.351397 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:42.351455 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:42.351457 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:43.354520 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:43.354540 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:43.354544 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:43.679180 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.066819
2015-04-09 15:58:43.679211 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:44.357473 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:44.357498 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:44.357503 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:45.357902 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:45.357927 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:45.357933 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:45.507203 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:45.507237 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:46.358016 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:46.358046 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:46.358052 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:47.358341 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:47.358367 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:47.358369 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:48.401197 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:48.401235 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:48.401240 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:49.556667 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.877455
2015-04-09 15:58:49.556695 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:49.556713 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:49.556719 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:49.556723 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:50.560270 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:50.560294 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:50.560299 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:51.473465 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:51.473508 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:51.560380 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:51.560413 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:51.560418 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:52.560495 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:52.560517 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:52.560522 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:55.179030 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:55.179064 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:58:55.179663 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:55.179685 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:55.179689 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:55.179756 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.623061
2015-04-09 15:58:55.179779 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:58:57.018719 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:57.018746 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:57.018751 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:58:58.026875 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:58:58.026903 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:58:58.026909 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:00.007096 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:00.007121 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:00.007127 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:00.007216 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:00.007232 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:00.468534 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.288755
2015-04-09 15:59:00.468570 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:02.033093 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:02.033120 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:02.033128 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:03.032361 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:03.032406 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:03.033223 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:03.033263 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:03.033268 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:04.033424 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:04.033449 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:04.033461 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:05.296555 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:05.296578 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:05.296584 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:05.493157 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.024587
2015-04-09 15:59:05.493185 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:06.296664 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:06.296689 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:06.296694 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:07.028333 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:07.028365 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:07.296777 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:07.296796 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:07.296803 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:08.357285 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:08.357303 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:08.357307 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:09.288341 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:09.288390 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:10.251238 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:10.251261 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:10.251266 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:10.549096 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.055911
2015-04-09 15:59:10.549123 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:11.258358 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:11.258378 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:11.258384 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:12.262925 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:12.262946 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:12.262951 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:13.265989 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:13.266011 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:13.266016 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:13.266040 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:13.266057 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:14.271202 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:14.271232 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:14.271237 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:15.471088 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:15.471133 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:15.471138 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:15.549376 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000253
2015-04-09 15:59:15.549431 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:16.476810 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:16.476833 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:16.476838 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:16.477262 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:16.477289 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:17.482502 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:17.482526 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:17.482531 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:18.487755 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:18.487773 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:18.487777 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:19.640115 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:19.640133 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:19.640138 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:20.641542 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.092112
2015-04-09 15:59:20.641573 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:20.641590 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:20.641596 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:20.641599 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:21.640160 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:21.640192 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:21.641670 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:21.641690 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:21.641697 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:22.641770 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:22.641788 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:22.641792 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:23.784061 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:23.784089 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:23.784094 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:25.834094 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:25.834121 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:25.834467 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.192893
2015-04-09 15:59:25.834495 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:25.834602 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:25.834617 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:25.834621 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:27.181850 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:27.181870 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:27.181874 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:28.218407 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:28.218432 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:28.218438 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:28.734307 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:28.734351 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:29.357799 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:29.357829 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:29.357835 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:30.357913 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:30.357937 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:30.357942 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:31.367172 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:31.367190 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:31.367177 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:31.367201 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:31.367204 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:31.367461 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.532966
2015-04-09 15:59:31.367486 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:32.338455 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:32.338480 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:33.157295 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:33.157322 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:33.157331 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:34.276256 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:34.276283 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:34.276288 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:35.280340 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:35.280371 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:35.280375 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:36.284267 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:36.284297 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:36.284302 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:36.921975 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.554489
2015-04-09 15:59:36.922013 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:37.284467 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:37.284489 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:37.284833 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:37.284846 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:37.284850 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:38.306461 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:38.306489 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:38.306495 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:40.479917 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:40.479943 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:40.479948 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:41.481881 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:41.481907 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:41.481920 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:42.487311 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.565297
2015-04-09 15:59:42.487331 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:42.487346 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:42.487345 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:42.487353 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:43.490661 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:43.490720 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:43.491236 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:43.491252 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:43.491257 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:44.498776 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:44.498804 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:44.498808 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:45.780706 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:45.780732 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:45.780737 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:47.116041 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:47.116065 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:47.116070 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:47.503297 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.015951
2015-04-09 15:59:47.503344 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:48.284938 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:48.284966 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:48.284972 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:48.285024 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:48.285038 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:49.292869 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:49.292895 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:49.292901 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:50.298660 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:50.298685 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:50.298690 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:51.300523 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:51.300542 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:51.300546 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:52.300691 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:52.300730 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:52.300734 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:52.588373 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.085029
2015-04-09 15:59:52.588420 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 15:59:53.313631 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:53.313654 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:53.313659 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:54.316617 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:54.316640 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:54.316645 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:54.316708 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:54.316720 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 15:59:55.316725 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:55.316752 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:55.316758 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:56.338845 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:56.338868 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:56.338892 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:57.339357 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:57.339385 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:57.339390 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:59.512617 7f70aa969700 5 osd.15 3814 tick
2015-04-09 15:59:59.512637 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 15:59:59.512644 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 15:59:59.512734 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 6.924314
2015-04-09 15:59:59.512751 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:00.353890 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:00.353920 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:01.218816 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:01.218835 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:01.218840 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:02.219046 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:02.219084 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:02.219093 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:03.701380 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:03.701407 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:03.701413 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:03.701615 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:03.701640 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:05.109708 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.596956
2015-04-09 16:00:05.109749 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:05.109956 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:05.109967 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:05.109971 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:06.108747 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:06.108791 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:06.110057 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:06.110076 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:06.110080 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:07.112671 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:07.112689 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:07.112692 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:09.563326 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:09.563357 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:09.563341 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:09.563383 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:09.563388 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:10.255036 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.145287
2015-04-09 16:00:10.255068 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:10.684516 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:10.684537 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:10.684544 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:12.264703 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:12.264720 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:12.264723 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:13.270686 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:13.270714 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:13.271545 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:13.271573 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:13.271576 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:14.278563 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:14.278590 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:14.278595 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:15.357037 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:15.357059 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:15.357064 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:15.357084 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.102016
2015-04-09 16:00:15.357101 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:16.357147 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:16.357167 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:16.357172 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:17.357249 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:17.357272 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:17.357277 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:18.458040 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:18.458068 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:18.458074 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:19.260413 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:19.260441 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:20.139772 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:20.139787 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:20.139791 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:21.260956 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:21.260989 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.903888
2015-04-09 16:00:21.260981 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:21.261012 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:21.261030 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:21.261036 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:21.261040 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:22.260414 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:22.260441 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:22.261122 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:22.261148 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:22.261153 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:23.420149 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:23.420164 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:23.420166 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:24.869454 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:24.869481 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:24.869486 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:26.375790 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:26.375820 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:26.375959 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.114946
2015-04-09 16:00:26.375989 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:26.376011 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:26.376020 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:26.376030 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:27.453839 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:27.453859 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:27.453865 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:29.729024 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:29.729043 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:29.729049 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:32.263746 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.887755
2015-04-09 16:00:32.263766 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:32.263789 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:32.263787 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:32.264067 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:32.264084 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:32.264088 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:33.274509 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:33.274534 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:33.274538 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:34.807280 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:34.807299 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:34.807304 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:36.128416 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:36.128450 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:36.128455 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:37.132012 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:37.132033 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:37.132038 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:37.322610 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.058823
2015-04-09 16:00:37.322834 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:38.366223 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:38.366260 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:38.366265 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:38.366740 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:38.366776 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:39.366414 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:39.366434 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:39.366438 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:40.238414 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:40.238445 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:41.247004 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:41.247025 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:41.247029 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:42.250521 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:42.250540 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:42.250543 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:42.338368 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.015532
2015-04-09 16:00:42.338410 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:43.258480 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:43.258496 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:43.258499 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:44.258597 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:44.258617 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:44.258621 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:44.543809 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:44.543843 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:45.548632 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:45.548669 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:45.548673 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:46.551024 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:46.551041 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:46.551045 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:47.556292 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:47.556310 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:47.556315 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:47.556815 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.218405
2015-04-09 16:00:47.556850 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:48.558916 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:48.558941 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:48.558945 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:48.644094 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:48.644120 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:49.559141 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:49.559173 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:49.559175 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:51.312250 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:51.312275 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:51.312280 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:51.652896 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:51.652937 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:52.312359 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:52.312385 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:52.312395 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:52.900195 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.343345
2015-04-09 16:00:52.900236 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:00:53.312614 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:53.312646 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:53.312651 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:54.312735 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:54.312759 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:54.312763 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:55.313045 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:55.313088 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:55.313094 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:56.327568 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:56.327608 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:56.327613 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:57.894591 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:00:57.894625 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:00:57.894630 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:00:57.894756 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:57.894784 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:00:57.921545 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.021308
2015-04-09 16:00:57.921589 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:01.366372 7f709a200700 1 heartbeat_map reset_timeout 'OSD::recovery_tp thread 0x7f709a200700' had timed out after 4
2015-04-09 16:01:01.366405 7f70999ff700 1 heartbeat_map reset_timeout 'OSD::disk_tp thread 0x7f70999ff700' had timed out after 4
2015-04-09 16:01:01.366436 7f709aa01700 1 heartbeat_map reset_timeout 'OSD::op_tp thread 0x7f709aa01700' had timed out after 4
2015-04-09 16:01:01.366938 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:01.366960 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:01.366966 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:01.367259 7f709b202700 1 heartbeat_map reset_timeout 'OSD::op_tp thread 0x7f709b202700' had timed out after 4
2015-04-09 16:01:01.367277 7f70991fe700 1 heartbeat_map reset_timeout 'OSD::command_tp thread 0x7f70991fe700' had timed out after 4
2015-04-09 16:01:01.395135 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:01.395180 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:01.895328 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:01.895361 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:02.367052 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:02.367080 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:02.367083 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:02.921676 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000086
2015-04-09 16:01:02.921727 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:03.395402 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:03.395441 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:03.395447 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:04.395688 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:04.395723 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:04.395728 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:04.795951 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:04.796006 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:05.395812 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:05.395837 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:05.395842 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:06.396071 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:06.396105 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:06.396111 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:07.396195 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:07.396221 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:07.396226 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:07.927392 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.005660
2015-04-09 16:01:07.927443 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:08.396448 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:08.396468 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:08.396471 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:09.396636 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:09.396662 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:09.396666 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:10.096213 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:10.096262 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:10.734325 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:10.734355 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:10.734362 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:12.230610 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:12.230637 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:12.230663 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:13.238757 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:13.238780 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:13.238785 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:13.238805 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.311361
2015-04-09 16:01:13.238826 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:14.243178 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:14.243210 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:14.243217 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:14.796566 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:14.796611 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:15.243289 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:15.243306 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:15.243309 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:16.528777 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:16.528818 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:16.529133 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:16.529160 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:16.529165 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:17.534014 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:17.534033 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:17.534038 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:18.394153 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.155326
2015-04-09 16:01:18.394196 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:18.539394 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:18.539413 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:18.539418 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:20.408865 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:20.408883 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:20.408893 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:21.414033 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:21.414065 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:21.414742 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:21.414788 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:21.414797 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:22.416810 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:22.416833 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:22.416838 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:24.365899 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:24.365917 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:24.365921 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:24.366372 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:24.366402 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:24.367937 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.973740
2015-04-09 16:01:24.367964 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:25.366888 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:25.366909 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:25.366914 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:26.367184 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:26.367210 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:26.367216 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:27.367415 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:27.367445 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:27.367449 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:29.325655 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:29.325689 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:29.325695 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:29.325962 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:29.325982 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:30.352342 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.984378
2015-04-09 16:01:30.352378 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:30.353410 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:30.353430 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:30.353434 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:32.352504 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:32.352528 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:32.352534 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:32.352673 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:32.352697 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:33.352613 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:33.352632 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:33.352636 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:34.352680 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:34.352712 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:34.352716 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:34.352732 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:34.352737 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:35.456141 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.103763
2015-04-09 16:01:35.456179 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:35.456620 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:35.456639 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:35.456643 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:36.460342 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:36.460384 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:36.460391 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:37.460609 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:37.460634 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:37.460639 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:39.629407 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:39.629436 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:39.630282 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:39.630307 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:39.630312 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:41.130813 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.674631
2015-04-09 16:01:41.130866 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:41.130920 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:41.130929 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:41.130935 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:41.132399 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:41.132412 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:41.979077 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:41.979096 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:42.135335 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:42.135413 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:42.135418 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:43.456575 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:43.456599 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:43.456603 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:44.513734 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:44.513758 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:44.513765 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:45.516469 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:45.516490 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:45.516494 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:46.089379 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:46.089413 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:46.310420 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.179552
2015-04-09 16:01:46.310459 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:46.519006 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:46.519026 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:46.519035 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:47.370211 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:47.370243 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:47.524316 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:47.524340 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:47.524344 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:48.527546 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:48.527565 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:48.527569 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:50.177169 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:50.177192 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:50.177196 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:51.338530 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:51.338556 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:51.338798 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:51.338810 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:51.338813 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:51.338829 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.028370
2015-04-09 16:01:51.338846 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:52.338890 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:52.338909 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:52.338914 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:54.871389 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:54.871406 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:54.871411 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:56.254224 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:56.254392 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:56.254396 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:56.492486 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.153640
2015-04-09 16:01:56.492520 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:01:57.238705 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:57.238736 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:01:57.254484 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:57.254503 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:57.254508 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:58.288676 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:58.288713 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:58.288719 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:01:59.456928 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:01:59.456956 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:01:59.456960 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:00.145107 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:00.145137 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:00.465375 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:00.465401 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:00.465405 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:01.466995 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:01.467026 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:01.467031 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:01.493603 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.001082
2015-04-09 16:02:01.493652 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:02.467135 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:02.467172 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:02.467177 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:03.467408 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:03.467450 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:03.467456 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:04.467688 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:04.467736 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:04.467746 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:05.484227 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:05.484260 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:05.484265 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:06.145614 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:06.145665 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:06.491442 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:06.491462 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:06.491466 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:06.518276 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.024623
2015-04-09 16:02:06.518321 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:06.656318 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:06.656347 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:07.538734 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:07.538758 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:07.538764 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:08.538923 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:08.538962 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:08.538985 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:09.562067 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:09.562091 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:09.562096 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:10.662682 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:10.662706 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:10.662712 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:11.662528 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.144206
2015-04-09 16:02:11.662561 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:11.662853 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:11.662870 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:11.662875 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:12.667415 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:12.667440 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:12.667835 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:12.667863 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:12.667868 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:13.670485 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:13.670506 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:13.670510 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:14.339109 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:14.339136 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:14.674320 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:14.674337 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:14.674342 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:15.718112 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:15.718130 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:15.718134 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:16.846638 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:16.846663 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:16.846669 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:16.847046 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.184486
2015-04-09 16:02:16.847079 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:18.243251 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:18.243288 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:18.243293 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:19.560172 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:19.560196 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:19.560202 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:19.560492 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:19.560519 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:20.560536 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:20.560570 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:20.560575 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:21.560655 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:21.560695 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:21.560701 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:21.847163 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000081
2015-04-09 16:02:21.847204 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:22.560788 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:22.560826 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:22.560831 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:23.971261 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:23.971284 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:23.971289 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:24.763481 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:24.763701 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:25.087000 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:25.087027 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:25.087032 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:26.295468 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:26.295500 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:26.295502 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:26.295683 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:26.295688 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:27.453397 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:27.453424 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:27.453448 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:27.453460 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:27.453464 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:27.453503 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.606298
2015-04-09 16:02:27.453535 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:30.341553 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:30.341578 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:30.341584 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:32.066778 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:32.066810 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:32.066832 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:32.066878 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:32.066902 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:32.453612 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000076
2015-04-09 16:02:32.453652 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:33.214708 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:33.214737 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:33.214743 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:34.214834 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:34.214879 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:34.214883 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:35.406818 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:35.406849 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:35.406854 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:36.584552 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:36.584571 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:36.584575 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:37.584409 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.130756
2015-04-09 16:02:37.584445 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:37.584646 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:37.584660 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:37.584663 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:38.239624 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:38.239655 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:38.597900 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:38.597922 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:38.597927 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:39.727637 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:39.727656 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:39.727661 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:40.827253 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:40.827277 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:40.827282 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:41.834321 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:41.834346 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:41.834352 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:42.692808 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.108362
2015-04-09 16:02:42.692838 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:42.838468 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:42.838488 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:42.838493 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:42.949013 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:42.949042 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:43.838566 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:43.838586 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:43.838591 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:44.838662 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:44.838687 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:44.838691 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:45.838775 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:45.838808 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:45.838814 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:46.995566 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:46.995590 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:46.995595 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:47.713157 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:47.713189 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:47.713326 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.020487
2015-04-09 16:02:47.713357 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:48.380315 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:48.380335 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:48.380340 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:49.413952 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:49.413975 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:49.413980 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:50.414058 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:50.414081 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:50.414085 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:51.505805 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:51.505836 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:51.505843 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:52.504988 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:52.505031 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:52.505922 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:52.505939 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:52.505960 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:52.880289 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.166931
2015-04-09 16:02:52.880343 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:55.210339 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:55.210358 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:55.210363 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:56.210567 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:56.210592 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:56.210596 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:57.828870 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:57.828909 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:02:57.829013 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:57.829028 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:57.829032 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:02:57.880559 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.000215
2015-04-09 16:02:57.880599 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:02:59.074820 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:02:59.074843 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:02:59.074848 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:00.254777 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:00.254814 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:00.254818 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:00.255287 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:00.255323 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:01.348550 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:01.348578 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:01.348586 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:03.481877 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:03.481905 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:03.481913 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:03.482496 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.601896
2015-04-09 16:03:03.482530 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:03:04.472070 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:04.472101 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:04.484624 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:04.484647 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:04.484652 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:05.488306 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:05.488344 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:05.488348 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:06.492237 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:06.492261 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:06.492266 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:07.492344 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:07.492370 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:07.492375 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:08.126939 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:08.126971 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:08.562476 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:08.562496 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:08.562500 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:08.562924 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.080393
2015-04-09 16:03:08.562948 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:03:09.585127 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:09.585151 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:09.585172 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:10.664057 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:10.664072 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:10.664075 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:11.735914 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:11.735947 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:11.735952 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:12.232569 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:12.232599 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:12.736669 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:12.736687 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:12.736691 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:13.704225 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.141276
2015-04-09 16:03:13.704268 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:03:13.895761 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:13.895786 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:13.895792 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:14.629512 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:14.629547 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:15.079905 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:15.079925 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:15.079930 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:16.098157 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:16.098175 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:16.098180 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:17.128576 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:17.128593 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:17.128597 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:18.398310 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:18.398507 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:18.398517 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:18.985146 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.280877
2015-04-09 16:03:18.985182 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:03:19.404261 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:19.404288 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:19.404293 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:20.408372 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:20.408396 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:20.408401 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:21.408297 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:21.408330 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:21.408569 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:21.408587 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:21.408592 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:22.408782 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:22.408804 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:22.408808 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:23.408889 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:23.408919 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:23.408924 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:24.015406 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 5.030225
2015-04-09 16:03:24.015443 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:03:24.348469 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:24.348504 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:26.326798 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:26.326816 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:26.326820 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:27.453446 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:27.453469 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:27.453474 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:28.551863 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:28.551884 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:28.551888 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:30.127602 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:30.127621 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:30.127626 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:30.127720 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:30.127734 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:30.128781 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry woke after 6.113338
2015-04-09 16:03:30.128803 7f70a7963700 20 filestore(/var/lib/ceph/osd/osd.15) sync_entry waiting for max_interval 5.000000
2015-04-09 16:03:31.130267 7f70aa969700 5 osd.15 3814 tick
2015-04-09 16:03:31.130290 7f70aa969700 10 osd.15 3814 do_waiters -- start
2015-04-09 16:03:31.130296 7f70aa969700 10 osd.15 3814 do_waiters -- finish
2015-04-09 16:03:31.227901 7f70989fd700 20 osd.15 3814 update_osd_stat osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
2015-04-09 16:03:31.227941 7f70989fd700 5 osd.15 3814 heartbeat: osd_stat(85104 kB used, 930 GB avail, 931 GB total, peers []/[] op hist [])
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
