Hi Sage, Thank you for the prompt response. Is there anything in /dev/disk/by-partuuid/ or is it missing entirely? Nothing , it was Missing Entirely. GOOD NEWS: I worked around this issue, if I set my journal path in the /etc/ceph.conf. My udev version is udevd --version 164 I still see the segfaults, I have attached details. I put the osd debug logs(osd-output.txt) and the leveldb_bt(leveldb_bt.txt). Looks like we have an issue in leveldb.... HERE IS THE BACK TRACE: I have attached the gdb before running it. #0 0x77f68ee0 in leveldb::SkipList<char const*, leveldb::MemTable::KeyComparator>::FindGreaterOrEqual(char const* const&, leveldb::SkipList<char const*, leveldb::MemTable::KeyComparator>::Node**) const () from /usr/local/lib/libleveldb.so.1 #1 0x77f69054 in leveldb::SkipList<char const*, leveldb::MemTable::KeyComparator>::Insert(char const* const&) () from /usr/local/lib/libleveldb.so.1 #2 0x77f68618 in leveldb::MemTable::Add(unsigned long long, leveldb::ValueType, leveldb::Slice const&, leveldb::Slice const&) () from /usr/local/lib/libleveldb.so.1 #3 0x77f7e434 in leveldb::(anonymous namespace)::MemTableInserter::Put(leveldb::Slice const&, leveldb::Slice const&) () from /usr/local/lib/libleveldb.so.1 #4 0x77f7e93c in leveldb::WriteBatch::Iterate(leveldb::WriteBatch::Handler*) const () from /usr/local/lib/libleveldb.so.1 #5 0x77f7eb8c in leveldb::WriteBatchInternal::InsertInto(leveldb::WriteBatch const*, leveldb::MemTable*) () from /usr/local/lib/libleveldb.so.1 #6 0x77f59360 in leveldb::DBImpl::Write(leveldb::WriteOptions const&, leveldb::WriteBatch*) () from /usr/local/lib/libleveldb.so.1 #7 0x00a5dda0 in LevelDBStore::submit_transaction_sync (this=0x1f77d10, t=<value optimized out>) at os/LevelDBStore.cc:146 #8 0x00b0d344 in DBObjectMap::sync (this=0x1f7af28, oid=0x0, spos=0x72cfe3b8) at os/DBObjectMap.cc:1126 #9 0x009b10b8 in FileStore::_set_replay_guard (this=0x1f72450, fd=17, spos=..., hoid=0x0, in_progress=false) at os/FileStore.cc:2070 #10 0x009b1c0c in FileStore::_set_replay_guard (this=0x1f72450, cid=DWARF-2 expression error: DW_OP_reg operations must be used either alone or in conjuction with DW_OP_piece. ) at os/FileStore.cc:2047 #11 0x009b2138 in FileStore::_create_collection (this=0x1f72450, c=DWARF-2 expression error: DW_OP_reg operations must be used either alone or in conjuction with DW_OP_piece. ) at os/FileStore.cc:4753 #12 0x009e42a8 in FileStore::_do_transaction (this=0x1f72450, t=..., op_seq=<value optimized out>, trans_num=0, handle=0x72cfec3c) at os/FileStore.cc:2413 #13 0x009eb47c in FileStore::_do_transactions (this=0x1f72450, tls=..., op_seq=2, handle=0x72cfec3c) at os/FileStore.cc:1952 #14 0x009eb858 in FileStore::_do_op (this=0x1f72450, osr=0x1f801b8, handle=...) at os/FileStore.cc:1761 #15 0x00c8f0bc in ThreadPool::worker (this=0x1f72cf0, wt=0x1f7ea90) at common/WorkQueue.cc:128 #16 0x00c91b94 in ThreadPool::WorkThread::entry() () #17 0x77f1c0a8 in start_thread () from /lib/libpthread.so.0 #18 0x777c1738 in ?? () from /lib/libc.so.6 Do I need to set any variable to set the cache size etcetc in ceph.conf. I only have osd_leveldb_cache_size=5242880 for now. Thanks Prashanth -----Original Message----- From: Sage Weil [mailto:sage@xxxxxxxxxxxx] Sent: Thursday, October 23, 2014 5:54 PM To: Prashanth Nednoor Cc: ceph-devel@xxxxxxxxxxxxxxx Subject: Re: Having issues trying to get the OSD up on a MIPS64!!! Hi Prashanth, On Thu, 23 Oct 2014, Prashanth Nednoor wrote: > Hello Everyone, > > We are using ceph-0.86, good news is we were able to compile and load > all the libraries and binaries needed to configure a CEPH-OSD on MIPS > 64 platform. The CEPH monitor is also able to detect the OSD, but not > up yet, as the osd activate failed. > Since we don?t have the required CEPH deploy utility for MIPS64, we > are following the manual procedure to create and activate an OSD. > We have disabled authentication between the clients and the OSD?s for > now. > > Has any body tried CEPH on a MIPS64? > /dev/sda is a 2TB local hard drive. > > This is how my partition looks after ceph-disk-prepare > /home/prashan/ceph-0.86/src# parted GNU Parted 2.3 Using /dev/sda > Welcome to GNU Parted! Type 'help' to view a list of commands. > (parted) p > Model: ATA TOSHIBA MQ01ABB2 (scsi) > Disk /dev/sda: 2000GB > Sector size (logical/physical): 512B/4096B Partition Table: gpt > > Number Start End Size File system Name Flags > 2 1049kB 5369MB 5368MB ceph journal > 1 5370MB 2000GB 1995GB xfs ceph data > > > > The following are the steps to create an OSD > 1) ceph-disk zap /dev/sda > 2) ceph-disk-prepare --cluster f615496c-b40a-4905-bbcd- > 2d3e181ff21a --fs-type xfs /dev/sda > 3) mount /dev/sda1 /var/lib/ceph/osd/ceph-0/ > 4) ceph-osd -i 0 ?mkfs is giving an error , > filestore(/var/lib/ceph/osd/ceph-0) could not find > 23c2fcde/osd_superblock/0//-1 in index: (2) No such file. > After this it segfaults. We have analyzed this further with the help > of strace and root caused this as objectmap file reading issue. > open("/var/lib/ceph/osd/ceph-0/current/omap/000005.log", O_RDONLY) = > 11, the first time it reads 32k, the read succeeds with 63 bytes and > it tries to read again with 27k and the read returns 0 bytes and the > CEPH osd segfaults. Can you generate a full log with --debug-osd 20 --debug-filestore 20 --debug-jouranl 20 passed to ceph-osd --mkfs and post that somewhere? It should tell us where things are going wrong. In particular, we want to see if that file/object is being written properly. It will also have a backtrace showing exactly where it crashed. > Please note that ceph-disk prepare creates a journal in a path which > is not valid(dev/disk/by-partuuid/cbd4a5d1-012f-4863-b492-080ad2a505cb). > So after step3 above I remove this journal below and manually create a > journal file before doing step4 above. > > > ls -l /var/lib/ceph/osd/ceph-0/ > total 16 > -rw-r--r-- 1 root root 37 Oct 22 21:40 ceph_fsid > -rw-r--r-- 1 root root 37 Oct 22 21:40 fsid lrwxrwxrwx 1 root root 58 > Oct 22 21:40 journal -> /dev/disk/by- > partuuid/cbd4a5d1-012f-4863-b492-080ad2a505cb Is there anything in /dev/disk/by-partuuid/ or is it missing entirely? Maybe you have an old udev. What distro is this? sage > -rw-r--r-- 1 root root 37 Oct 22 21:40 journal_uuid > -rw-r--r-- 1 root root 21 Oct 22 21:40 magic > > Any pointers to move ahead will be greatly appreciated?? > > thanks > Prashanth > > > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" > in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo > info at http://vger.kernel.org/majordomo-info.html > >
#0 0x77f68ee0 in leveldb::SkipList<char const*, leveldb::MemTable::KeyComparator>::FindGreaterOrEqual(char const* const&, leveldb::SkipList<char const*, leveldb::MemTable::KeyComparator>::Node**) const () from /usr/local/lib/libleveldb.so.1 #1 0x77f69054 in leveldb::SkipList<char const*, leveldb::MemTable::KeyComparator>::Insert(char const* const&) () from /usr/local/lib/libleveldb.so.1 #2 0x77f68618 in leveldb::MemTable::Add(unsigned long long, leveldb::ValueType, leveldb::Slice const&, leveldb::Slice const&) () from /usr/local/lib/libleveldb.so.1 #3 0x77f7e434 in leveldb::(anonymous namespace)::MemTableInserter::Put(leveldb::Slice const&, leveldb::Slice const&) () from /usr/local/lib/libleveldb.so.1 #4 0x77f7e93c in leveldb::WriteBatch::Iterate(leveldb::WriteBatch::Handler*) const () from /usr/local/lib/libleveldb.so.1 #5 0x77f7eb8c in leveldb::WriteBatchInternal::InsertInto(leveldb::WriteBatch const*, leveldb::MemTable*) () from /usr/local/lib/libleveldb.so.1 #6 0x77f59360 in leveldb::DBImpl::Write(leveldb::WriteOptions const&, leveldb::WriteBatch*) () from /usr/local/lib/libleveldb.so.1 #7 0x00a5dda0 in LevelDBStore::submit_transaction_sync (this=0x1f77d10, t=<value optimized out>) at os/LevelDBStore.cc:146 #8 0x00b0d344 in DBObjectMap::sync (this=0x1f7af28, oid=0x0, spos=0x72cfe3b8) at os/DBObjectMap.cc:1126 #9 0x009b10b8 in FileStore::_set_replay_guard (this=0x1f72450, fd=17, spos=..., hoid=0x0, in_progress=false) at os/FileStore.cc:2070 #10 0x009b1c0c in FileStore::_set_replay_guard (this=0x1f72450, cid=DWARF-2 expression error: DW_OP_reg operations must be used either alone or in conjuction with DW_OP_piece. ) at os/FileStore.cc:2047 #11 0x009b2138 in FileStore::_create_collection (this=0x1f72450, c=DWARF-2 expression error: DW_OP_reg operations must be used either alone or in conjuction with DW_OP_piece. ) at os/FileStore.cc:4753 #12 0x009e42a8 in FileStore::_do_transaction (this=0x1f72450, t=..., op_seq=<value optimized out>, trans_num=0, handle=0x72cfec3c) at os/FileStore.cc:2413 #13 0x009eb47c in FileStore::_do_transactions (this=0x1f72450, tls=..., op_seq=2, handle=0x72cfec3c) at os/FileStore.cc:1952 #14 0x009eb858 in FileStore::_do_op (this=0x1f72450, osr=0x1f801b8, handle=...) at os/FileStore.cc:1761 #15 0x00c8f0bc in ThreadPool::worker (this=0x1f72cf0, wt=0x1f7ea90) at common/WorkQueue.cc:128 #16 0x00c91b94 in ThreadPool::WorkThread::entry() () #17 0x77f1c0a8 in start_thread () from /lib/libpthread.so.0 #18 0x777c1738 in ?? () from /lib/libc.so.6
This is were my journal points to /var/lib/ceph/osd/ceph-0/journal -> /dev/disk/by-partuuid/9a868ac8-5ec8-4047-aa4c-8e9e6ce0 In the above output, for my system The path "/dev/disk/by-partuuid/" is missing. udevd --version 164 gdb --args ceph-osd -i=0 --mkfs --debug-osd 20 --debug-filestore 20 --debug-journal 20 --mkfs -d root@octeon:/home/prashan/ceph-0.86/src# ./ceph-osd -i=0 --mkfs --debug-osd 20 --debug-filestore 20 --debug-journal 20 --mkfs -d ./ceph-osd -i=0 --mkfs --debug-osd 20 --debug-filestore 20 --debug-journal 20 --mkfs -d 2014-10-24 19:26:59.981239 7760b000 0 ceph version 0.86 (97dcc0539dfa7dac3de74852305d51580b7b1f82), process ceph-osd, pid 3563 2014-10-24 19:26:59.982021 7760b000 10 filestore(/var/lib/ceph/osd/ceph-0) dump_stop 2014-10-24 19:26:59.983418 7760b000 1 filestore(/var/lib/ceph/osd/ceph-0) mkfs in /var/lib/ceph/osd/ceph-0 2014-10-24 19:27:00.001999 7760b000 1 filestore(/var/lib/ceph/osd/ceph-0) mkfs fsid is already set to 536991b9-cc6d-4023-860d-53ad361dce7e 2014-10-24 19:27:00.126542 7760b000 0 filestore(/var/lib/ceph/osd/ceph-0) backend generic (magic 0x58465342) 2014-10-24 19:27:00.126590 7760b000 1 filestore(/var/lib/ceph/osd/ceph-0) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-10-24 19:27:00.338490 7760b000 1 filestore(/var/lib/ceph/osd/ceph-0) leveldb db exists/created 2014-10-24 19:27:00.338577 7760b000 10 filestore(/var/lib/ceph/osd/ceph-0) open_journal at /dev/sda2 2014-10-24 19:27:00.338752 7760b000 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 5367661056) 2014-10-24 19:27:00.345391 7760b000 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sda2) 2014-10-24 19:27:00.345793 7760b000 1 journal _open /dev/sda2 fd 8: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-10-24 19:27:00.345857 7760b000 10 journal read_header 2014-10-24 19:27:00.360413 7760b000 10 journal header: block_size 4096 alignment 4096 max_size 5367660544 2014-10-24 19:27:00.360442 7760b000 10 journal header: start 4096 2014-10-24 19:27:00.360448 7760b000 10 journal write_pos 0 2014-10-24 19:27:00.360485 7760b000 -1 journal check: ondisk fsid 8b1f947b-0a47-4f4e-bcd7-57ed33d5d300 doesn't match expected 536991b9-cc6d-4023-860d-53ad361dce7e, invalid (someone else's?) journal 2014-10-24 19:27:00.360636 7760b000 2 journal create /dev/sda2 fsid 536991b9-cc6d-4023-860d-53ad361dce7e 2014-10-24 19:27:00.360726 7760b000 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 5367661056) 2014-10-24 19:27:00.372395 7760b000 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sda2) 2014-10-24 19:27:00.372554 7760b000 1 journal _open /dev/sda2 fd 8: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-10-24 19:27:00.372588 7760b000 10 journal header: block_size 4096 alignment 4096 max_size 5367660544 2014-10-24 19:27:00.372596 7760b000 10 journal header: start 4096 2014-10-24 19:27:00.372601 7760b000 10 journal write_pos 0 2014-10-24 19:27:00.393775 7760b000 2 journal create done 2014-10-24 19:27:00.394579 7760b000 0 filestore(/var/lib/ceph/osd/ceph-0) mkjournal created journal on /dev/sda2 2014-10-24 19:27:00.394712 7760b000 1 filestore(/var/lib/ceph/osd/ceph-0) mkfs done in /var/lib/ceph/osd/ceph-0 2014-10-24 19:27:00.394774 7760b000 5 filestore(/var/lib/ceph/osd/ceph-0) basedir /var/lib/ceph/osd/ceph-0 journal /dev/sda2 2014-10-24 19:27:00.394897 7760b000 10 filestore(/var/lib/ceph/osd/ceph-0) mount fsid is 536991b9-cc6d-4023-860d-53ad361dce7e 2014-10-24 19:27:00.395107 7760b000 0 filestore(/var/lib/ceph/osd/ceph-0) backend generic (magic 0x58465342) 2014-10-24 19:27:00.526956 7760b000 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features: FIEMAP ioctl is supported and appears to work 2014-10-24 19:27:00.526999 7760b000 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-10-24 19:27:00.527553 7760b000 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features: syncfs(2) syscall not supported 2014-10-24 19:27:00.527595 7760b000 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features: no syncfs(2), must use sync(2). 2014-10-24 19:27:00.527600 7760b000 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-0) detect_features: WARNING: multiple ceph-osd daemons on the same host will be slow 2014-10-24 19:27:00.528461 7760b000 5 filestore(/var/lib/ceph/osd/ceph-0) mount op_seq is 1 2014-10-24 19:27:00.660719 7760b000 20 filestore (init)dbobjectmap: seq is 1 2014-10-24 19:27:00.660791 7760b000 10 filestore(/var/lib/ceph/osd/ceph-0) open_journal at /dev/sda2 2014-10-24 19:27:00.660883 7760b000 0 filestore(/var/lib/ceph/osd/ceph-0) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-10-24 19:27:00.660901 7760b000 10 filestore(/var/lib/ceph/osd/ceph-0) list_collections 2014-10-24 19:27:00.661284 7760b000 10 journal journal_replay fs op_seq 1 2014-10-24 19:27:00.661293 752ff460 20 filestore(/var/lib/ceph/osd/ceph-0) sync_entry waiting for max_interval 5.000000 2014-10-24 19:27:00.661318 7760b000 2 journal open /dev/sda2 fsid 536991b9-cc6d-4023-860d-53ad361dce7e fs_op_seq 1 2014-10-24 19:27:00.661377 7760b000 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 5367661056) 2014-10-24 19:27:00.668052 7760b000 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sda2) 2014-10-24 19:27:00.668387 7760b000 1 journal _open /dev/sda2 fd 14: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-10-24 19:27:00.668426 7760b000 10 journal read_header 2014-10-24 19:27:00.682659 7760b000 10 journal header: block_size 4096 alignment 4096 max_size 5367660544 2014-10-24 19:27:00.682686 7760b000 10 journal header: start 4096 2014-10-24 19:27:00.682692 7760b000 10 journal write_pos 4096 2014-10-24 19:27:00.682705 7760b000 10 journal open header.fsid = 536991b9-cc6d-4023-860d-53ad361dce7e 2014-10-24 19:27:00.683010 7760b000 2 journal No further valid entries found, journal is most likely valid 2014-10-24 19:27:00.683024 7760b000 10 journal open reached end of journal. 2014-10-24 19:27:00.683070 7760b000 2 journal No further valid entries found, journal is most likely valid 2014-10-24 19:27:00.683078 7760b000 3 journal journal_replay: end of journal, done. 2014-10-24 19:27:00.683084 7760b000 10 journal make_writeable 2014-10-24 19:27:00.683372 7760b000 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 5367661056) 2014-10-24 19:27:00.694396 7760b000 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sda2) 2014-10-24 19:27:00.694542 7760b000 1 journal _open /dev/sda2 fd 14: 5367660544 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-10-24 19:27:00.694677 74aff460 10 journal write_thread_entry start 2014-10-24 19:27:00.694719 74aff460 20 journal prepare_multi_write queue_pos now 4096 2014-10-24 19:27:00.694753 73aff460 10 journal write_finish_thread_entry enter 2014-10-24 19:27:00.694759 7760b000 10 journal journal_start 2014-10-24 19:27:00.694773 73aff460 20 journal write_finish_thread_entry sleeping 2014-10-24 19:27:00.694867 74aff460 15 journal do_aio_write writing 4096~0 + header 2014-10-24 19:27:00.694890 74aff460 20 journal write_aio_bl 0~4096 seq 0 2014-10-24 19:27:00.694958 74aff460 20 journal write_aio_bl .. 0~4096 in 1 2014-10-24 19:27:00.695129 74aff460 20 journal write_aio_bl 4096~0 seq 0 2014-10-24 19:27:00.695306 73aff460 20 journal write_finish_thread_entry waiting for aio(s) 2014-10-24 19:27:00.695306 74aff460 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes 2014-10-24 19:27:00.695333 74aff460 20 journal write_thread_entry going to sleep 2014-10-24 19:27:00.695357 73aff460 10 journal write_finish_thread_entry aio 0~4096 done 2014-10-24 19:27:00.695373 73aff460 20 journal check_aio_completion 2014-10-24 19:27:00.695377 73aff460 20 journal check_aio_completion completed seq 0 0~4096 2014-10-24 19:27:00.695392 73aff460 20 journal write_finish_thread_entry sleeping 2014-10-24 19:27:00.695822 7760b000 15 filestore(/var/lib/ceph/osd/ceph-0) read meta/23c2fcde/osd_superblock/0//-1 0~0 2014-10-24 19:27:00.696088 7760b000 -1 filestore(/var/lib/ceph/osd/ceph-0) could not find 23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory 2014-10-24 19:27:00.696207 7760b000 10 filestore(/var/lib/ceph/osd/ceph-0) FileStore::read(meta/23c2fcde/osd_superblock/0//-1) open error: (2) No such file or directory 2014-10-24 19:27:00.696479 7760b000 5 filestore(/var/lib/ceph/osd/ceph-0) queue_transactions new osr(default 0x1f72ac8)/0x1f72ac8 2014-10-24 19:27:00.696612 7760b000 10 journal op_submit_start 2 2014-10-24 19:27:00.696692 7760b000 5 filestore(/var/lib/ceph/osd/ceph-0) queue_transactions (writeahead) 2 0x7f73ea18 2014-10-24 19:27:00.696766 7760b000 10 journal op_journal_transactions 2 0x7f73ea18 2014-10-24 19:27:00.696908 7760b000 5 journal submit_entry seq 2 len 534 (0x7630c3f8) 2014-10-24 19:27:00.697037 7760b000 10 journal op_submit_finish 2 2014-10-24 19:27:00.697046 74aff460 20 journal write_thread_entry woke up 2014-10-24 19:27:00.697088 74aff460 10 journal room 5367656447 max_size 5367660544 pos 4096 header.start 4096 top 4096 2014-10-24 19:27:00.697104 74aff460 10 journal check_for_full at 4096 : 4096 < 5367656447 2014-10-24 19:27:00.697115 74aff460 15 journal prepare_single_write 1 will write 4096 : seq 2 len 534 -> 4096 (head 40 pre_pad 0 ebl 534 post_pad 3482 tail 40) (ebl alignment -1) 2014-10-24 19:27:00.697166 74aff460 20 journal prepare_multi_write queue_pos now 8192 2014-10-24 19:27:00.697176 74aff460 15 journal do_aio_write writing 4096~4096 2014-10-24 19:27:00.697226 74aff460 20 journal write_aio_bl 4096~4096 seq 2 2014-10-24 19:27:00.697244 74aff460 20 journal write_aio_bl .. 4096~4096 in 1 2014-10-24 19:27:00.704964 73aff460 20 journal write_finish_thread_entry waiting for aio(s) 2014-10-24 19:27:00.704958 74aff460 5 journal put_throttle finished 1 ops and 534 bytes, now 0 ops and 0 bytes 2014-10-24 19:27:00.704995 74aff460 20 journal write_thread_entry going to sleep 2014-10-24 19:27:00.705062 73aff460 10 journal write_finish_thread_entry aio 4096~4096 done 2014-10-24 19:27:00.705078 73aff460 20 journal check_aio_completion 2014-10-24 19:27:00.705082 73aff460 20 journal check_aio_completion completed seq 2 4096~4096 2014-10-24 19:27:00.705094 73aff460 20 journal check_aio_completion queueing finishers through seq 2 2014-10-24 19:27:00.705104 73aff460 10 journal queue_completions_thru seq 2 queueing seq 2 0x7630c3f8 lat 0.008102 2014-10-24 19:27:00.705137 73aff460 20 journal write_finish_thread_entry sleeping 2014-10-24 19:27:00.705154 732ff460 5 filestore(/var/lib/ceph/osd/ceph-0) _journaled_ahead 0x7630c0e0 seq 2 osr(default 0x1f72ac8) 0x7f73ea18 2014-10-24 19:27:00.705174 732ff460 5 filestore(/var/lib/ceph/osd/ceph-0) queue_op 0x7630c0e0 seq 2 osr(default 0x1f72ac8) 528 bytes (queue has 1 ops and 528 bytes) 2014-10-24 19:27:00.705220 72aff460 10 journal op_apply_start 2 open_ops 0 -> 1 2014-10-24 19:27:00.705229 72aff460 5 filestore(/var/lib/ceph/osd/ceph-0) _do_op 0x7630c0e0 seq 2 osr(default 0x1f72ac8)/0x1f72ac8 start 2014-10-24 19:27:00.705242 72aff460 10 filestore(/var/lib/ceph/osd/ceph-0) _do_transaction on 0x7f73ea18 2014-10-24 19:27:00.705283 72aff460 15 filestore(/var/lib/ceph/osd/ceph-0) create_collection /var/lib/ceph/osd/ceph-0/current/meta 2014-10-24 19:27:00.733384 72aff460 10 filestore(/var/lib/ceph/osd/ceph-0) create_collection /var/lib/ceph/osd/ceph-0/current/meta = 0 2014-10-24 19:27:00.733695 72aff460 10 filestore(/var/lib/ceph/osd/ceph-0) _set_replay_guard 2.0.0 2014-10-24 19:27:00.771441 72aff460 20 filestore dbobjectmap: seq is 1 *** Caught signal (Segmentation fault) ** in thread 72aff460 Segmentation fault
