hello, My many osds can't not start , now I lost lots of data , how can I solve this issue? the failed osd got a "Caught signal (Aborted)". what happen ? help me !!!!! the osd log are list below: --------------------------- 2014-05-01 15:30:22.291380 7f3467c387a0 0 ceph version 0.80-rc1 (6769f4dc88425396921f94e1a37a1c90758aa3ea), process ceph-osd, pid 15289 2014-05-01 15:30:22.301109 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount detected xfs (libxfs) 2014-05-01 15:30:22.301133 7f3467c387a0 1 filestore(/var/lib/ceph/osd.38) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs 2014-05-01 15:30:22.305344 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is supported and appears to work 2014-05-01 15:30:22.305371 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-05-01 15:30:22.306550 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-05-01 15:30:22.306752 7f3467c387a0 0 xfsfilestorebackend(/var/lib/ceph/osd.38) detect_feature: extsize is supported 2014-05-01 15:30:22.310354 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-05-01 15:30:22.311465 7f3467c387a0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2014-05-01 15:30:22.311541 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 20: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-05-01 15:30:22.311624 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 20: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-05-01 15:30:22.312238 7f3467c387a0 1 journal close /var/lib/ceph/osd.38/journal 2014-05-01 15:30:22.315076 7f3467c387a0 2 osd.38 0 mounting /var/lib/ceph/osd.38 /var/lib/ceph/osd.38/journal 2014-05-01 15:30:22.315139 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount detected xfs (libxfs) 2014-05-01 15:30:22.317250 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is supported and appears to work 2014-05-01 15:30:22.317268 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-05-01 15:30:22.318229 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: syscall(SYS_syncfs, fd) fully supported 2014-05-01 15:30:22.318346 7f3467c387a0 0 xfsfilestorebackend(/var/lib/ceph/osd.38) detect_feature: extsize is supported 2014-05-01 15:30:22.320891 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-05-01 15:30:22.321878 7f3467c387a0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2014-05-01 15:30:22.321893 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 21: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-05-01 15:30:22.321946 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 21: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-05-01 15:30:22.322235 7f3467c387a0 2 osd.38 0 boot 2014-05-01 15:30:22.322356 7f3467c387a0 10 osd.38 0 read_superblock sb(d1c527f4-3d28-4e3d-a9c7-c253279b6179 osd.38 5eb444bd-06a0-40fc-bf3a-023b9f7952da e30899 [26307,30899] lci=[30751,30899]) 2014-05-01 15:30:22.322456 7f3467c387a0 10 open_all_classes 2014-05-01 15:30:22.322515 7f3467c387a0 10 open_all_classes found kvs 2014-05-01 15:30:22.322525 7f3467c387a0 10 _get_class adding new class name kvs 0x29a8108 2014-05-01 15:30:22.322529 7f3467c387a0 10 _load_class kvs from /usr/lib64/rados-classes/libcls_kvs.so 2014-05-01 15:30:22.323110 7f3467c387a0 10 register_class kvs status 3 2014-05-01 15:30:22.323121 7f3467c387a0 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7f345fbf2b40 2014-05-01 15:30:22.323126 7f3467c387a0 10 register_cxx_method kvs.get_next_idata flags 1 0x7f345fbf1ba0 2014-05-01 15:30:22.323128 7f3467c387a0 10 register_cxx_method kvs.get_prev_idata flags 1 0x7f345fbf3590 2014-05-01 15:30:22.323130 7f3467c387a0 10 register_cxx_method kvs.read_many flags 1 0x7f345fbf2130 2014-05-01 15:30:22.323132 7f3467c387a0 10 register_cxx_method kvs.check_writable flags 3 0x7f345fbefac0 2014-05-01 15:30:22.323137 7f3467c387a0 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7f345fbefed0 2014-05-01 15:30:22.323140 7f3467c387a0 10 register_cxx_method kvs.omap_insert flags 2 0x7f345fbf13b0 2014-05-01 15:30:22.323143 7f3467c387a0 10 register_cxx_method kvs.create_with_omap flags 2 0x7f345fbf0ff0 2014-05-01 15:30:22.323145 7f3467c387a0 10 register_cxx_method kvs.omap_remove flags 2 0x7f345fbf0100 2014-05-01 15:30:22.323147 7f3467c387a0 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7f345fbf0810 2014-05-01 15:30:22.323149 7f3467c387a0 10 _load_class kvs success 2014-05-01 15:30:22.323152 7f3467c387a0 10 open_all_classes found rgw 2014-05-01 15:30:22.323154 7f3467c387a0 10 _get_class adding new class name rgw 0x29a8648 2014-05-01 15:30:22.323157 7f3467c387a0 10 _load_class rgw from /usr/lib64/rados-classes/libcls_rgw.so 2014-05-01 15:30:22.327033 7f3467c387a0 10 register_class rgw status 3 2014-05-01 15:30:22.327036 7f3467c387a0 10 register_cxx_method rgw.bucket_init_index flags 3 0x7f345f8ab2b0 2014-05-01 15:30:22.327040 7f3467c387a0 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7f345f8a9620 2014-05-01 15:30:22.327047 7f3467c387a0 10 register_cxx_method rgw.bucket_list flags 1 0x7f345f8b04e0 2014-05-01 15:30:22.327050 7f3467c387a0 10 register_cxx_method rgw.bucket_check_index flags 1 0x7f345f8b0090 2014-05-01 15:30:22.327052 7f3467c387a0 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7f345f8b0330 2014-05-01 15:30:22.327054 7f3467c387a0 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7f345f8b2150 2014-05-01 15:30:22.327057 7f3467c387a0 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7f345f8b2ba0 2014-05-01 15:30:22.327059 7f3467c387a0 10 register_cxx_method rgw.bi_log_list flags 1 0x7f345f8aa930 2014-05-01 15:30:22.327061 7f3467c387a0 10 register_cxx_method rgw.bi_log_trim flags 3 0x7f345f8abac0 2014-05-01 15:30:22.327063 7f3467c387a0 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7f345f8b1640 2014-05-01 15:30:22.327065 7f3467c387a0 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7f345f8af300 2014-05-01 15:30:22.327067 7f3467c387a0 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7f345f8ac9f0 2014-05-01 15:30:22.327072 7f3467c387a0 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7f345f8ac650 2014-05-01 15:30:22.327074 7f3467c387a0 10 register_cxx_method rgw.gc_set_entry flags 3 0x7f345f8ae4d0 2014-05-01 15:30:22.327077 7f3467c387a0 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7f345f8ae1a0 2014-05-01 15:30:22.327079 7f3467c387a0 10 register_cxx_method rgw.gc_list flags 1 0x7f345f8ad2e0 2014-05-01 15:30:22.327084 7f3467c387a0 10 register_cxx_method rgw.gc_remove flags 3 0x7f345f8ae7b0 2014-05-01 15:30:22.327086 7f3467c387a0 10 _load_class rgw success 2014-05-01 15:30:22.327089 7f3467c387a0 10 open_all_classes found refcount 2014-05-01 15:30:22.327093 7f3467c387a0 10 _get_class adding new class name refcount 0x29a8488 2014-05-01 15:30:22.327096 7f3467c387a0 10 _load_class refcount from /usr/lib64/rados-classes/libcls_refcount.so 2014-05-01 15:30:22.330572 7f3467c387a0 10 register_class refcount status 3 2014-05-01 15:30:22.330574 7f3467c387a0 10 register_cxx_method refcount.get flags 3 0x7f3457ad5fe0 2014-05-01 15:30:22.330578 7f3467c387a0 10 register_cxx_method refcount.put flags 3 0x7f3457ad6580 2014-05-01 15:30:22.330581 7f3467c387a0 10 register_cxx_method refcount.set flags 3 0x7f3457ad4c20 2014-05-01 15:30:22.330583 7f3467c387a0 10 register_cxx_method refcount.read flags 1 0x7f3457ad5a40 2014-05-01 15:30:22.330585 7f3467c387a0 10 _load_class refcount success 2014-05-01 15:30:22.330588 7f3467c387a0 10 open_all_classes found hello 2014-05-01 15:30:22.330593 7f3467c387a0 10 _get_class adding new class name hello 0x29a8aa8 2014-05-01 15:30:22.330596 7f3467c387a0 10 _load_class hello from /usr/lib64/rados-classes/libcls_hello.so 2014-05-01 15:30:22.330824 7f3467c387a0 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello 2014-05-01 15:30:22.330884 7f3467c387a0 10 register_class hello status 3 2014-05-01 15:30:22.330912 7f3467c387a0 10 register_cxx_method hello.say_hello flags 1 0x7f345782c1d0 2014-05-01 15:30:22.330939 7f3467c387a0 10 register_cxx_method hello.record_hello flags 2 0x7f345782c3c0 2014-05-01 15:30:22.330966 7f3467c387a0 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7f345782c920 2014-05-01 15:30:22.330992 7f3467c387a0 10 register_cxx_method hello.replay flags 1 0x7f345782c1a0 2014-05-01 15:30:22.331018 7f3467c387a0 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7f345782cc00 2014-05-01 15:30:22.331044 7f3467c387a0 10 register_cxx_method hello.bad_reader flags 2 0x7f345782c190 2014-05-01 15:30:22.331071 7f3467c387a0 10 register_cxx_method hello.bad_writer flags 1 0x7f345782c180 2014-05-01 15:30:22.331097 7f3467c387a0 10 _load_class hello success 2014-05-01 15:30:22.331124 7f3467c387a0 10 open_all_classes found rbd 2014-05-01 15:30:22.331149 7f3467c387a0 10 _get_class adding new class name rbd 0x29a88e8 2014-05-01 15:30:22.331175 7f3467c387a0 10 _load_class rbd from /usr/lib64/rados-classes/libcls_rbd.so 2014-05-01 15:30:22.331451 7f3467c387a0 10 register_class rbd status 3 2014-05-01 15:30:22.331489 7f3467c387a0 10 register_cxx_method rbd.create flags 3 0x7f345761a8e0 2014-05-01 15:30:22.331496 7f3467c387a0 10 register_cxx_method rbd.get_features flags 1 0x7f3457622360 2014-05-01 15:30:22.331500 7f3467c387a0 10 register_cxx_method rbd.get_size flags 1 0x7f34576229f0 2014-05-01 15:30:22.331502 7f3467c387a0 10 register_cxx_method rbd.set_size flags 3 0x7f3457621070 2014-05-01 15:30:22.331504 7f3467c387a0 10 register_cxx_method rbd.get_snapcontext flags 1 0x7f345761d540 2014-05-01 15:30:22.331506 7f3467c387a0 10 register_cxx_method rbd.get_object_prefix flags 1 0x7f345761a730 2014-05-01 15:30:22.331509 7f3467c387a0 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7f3457621ea0 2014-05-01 15:30:22.331511 7f3467c387a0 10 register_cxx_method rbd.snapshot_add flags 3 0x7f3457620560 2014-05-01 15:30:22.331513 7f3467c387a0 10 register_cxx_method rbd.snapshot_remove flags 3 0x7f3457621c50 2014-05-01 15:30:22.331515 7f3467c387a0 10 register_cxx_method rbd.get_all_features flags 1 0x7f34576192f0 2014-05-01 15:30:22.331517 7f3467c387a0 10 register_cxx_method rbd.copyup flags 3 0x7f3457619270 2014-05-01 15:30:22.331519 7f3467c387a0 10 register_cxx_method rbd.get_parent flags 1 0x7f3457622e30 2014-05-01 15:30:22.331521 7f3467c387a0 10 register_cxx_method rbd.set_parent flags 3 0x7f345761ff50 2014-05-01 15:30:22.331523 7f3467c387a0 10 register_cxx_method rbd.remove_parent flags 3 0x7f345761fd40 2014-05-01 15:30:22.331548 7f3467c387a0 10 register_cxx_method rbd.set_protection_status flags 3 0x7f34576226c0 2014-05-01 15:30:22.331551 7f3467c387a0 10 register_cxx_method rbd.get_protection_status flags 1 0x7f34576220f0 2014-05-01 15:30:22.331553 7f3467c387a0 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7f34576215c0 2014-05-01 15:30:22.331555 7f3467c387a0 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7f345761c130 2014-05-01 15:30:22.331557 7f3467c387a0 10 register_cxx_method rbd.add_child flags 3 0x7f345761efa0 2014-05-01 15:30:22.331560 7f3467c387a0 10 register_cxx_method rbd.remove_child flags 3 0x7f345761e9f0 2014-05-01 15:30:22.331581 7f3467c387a0 10 register_cxx_method rbd.get_children flags 1 0x7f345761e680 2014-05-01 15:30:22.331583 7f3467c387a0 10 register_cxx_method rbd.get_id flags 1 0x7f3457619820 2014-05-01 15:30:22.331585 7f3467c387a0 10 register_cxx_method rbd.set_id flags 3 0x7f345761d950 2014-05-01 15:30:22.331587 7f3467c387a0 10 register_cxx_method rbd.dir_get_id flags 1 0x7f345761bd30 2014-05-01 15:30:22.331589 7f3467c387a0 10 register_cxx_method rbd.dir_get_name flags 1 0x7f345761af00 2014-05-01 15:30:22.331591 7f3467c387a0 10 register_cxx_method rbd.dir_list flags 1 0x7f345761b1f0 2014-05-01 15:30:22.331593 7f3467c387a0 10 register_cxx_method rbd.dir_add_image flags 3 0x7f345761d300 2014-05-01 15:30:22.331596 7f3467c387a0 10 register_cxx_method rbd.dir_remove_image flags 3 0x7f345761c940 2014-05-01 15:30:22.331598 7f3467c387a0 10 register_cxx_method rbd.dir_rename_image flags 3 0x7f345761d020 2014-05-01 15:30:22.331600 7f3467c387a0 10 register_cxx_method rbd.snap_list flags 1 0x7f345761fa90 2014-05-01 15:30:22.331602 7f3467c387a0 10 register_cxx_method rbd.snap_add flags 3 0x7f345761e160 2014-05-01 15:30:22.331604 7f3467c387a0 10 register_cxx_method rbd.snap_remove flags 3 0x7f345761dbe0 2014-05-01 15:30:22.331606 7f3467c387a0 10 _load_class rbd success 2014-05-01 15:30:22.331609 7f3467c387a0 10 open_all_classes found user 2014-05-01 15:30:22.331612 7f3467c387a0 10 _get_class adding new class name user 0x29a8728 2014-05-01 15:30:22.331614 7f3467c387a0 10 _load_class user from /usr/lib64/rados-classes/libcls_user.so 2014-05-01 15:30:22.331773 7f3467c387a0 10 register_class user status 3 2014-05-01 15:30:22.331775 7f3467c387a0 10 register_cxx_method user.set_buckets_info flags 3 0x7f345740e190 2014-05-01 15:30:22.331779 7f3467c387a0 10 register_cxx_method user.complete_stats_sync flags 3 0x7f345740cd40 2014-05-01 15:30:22.331781 7f3467c387a0 10 register_cxx_method user.remove_bucket flags 3 0x7f345740f170 2014-05-01 15:30:22.331783 7f3467c387a0 10 register_cxx_method user.list_buckets flags 1 0x7f345740d350 2014-05-01 15:30:22.331785 7f3467c387a0 10 register_cxx_method user.get_header flags 1 0x7f345740c9b0 2014-05-01 15:30:22.331787 7f3467c387a0 10 _load_class user success 2014-05-01 15:30:22.331789 7f3467c387a0 10 open_all_classes found replica_log 2014-05-01 15:30:22.331792 7f3467c387a0 10 _get_class adding new class name replica_log 0x29a8f08 2014-05-01 15:30:22.331794 7f3467c387a0 10 _load_class replica_log from /usr/lib64/rados-classes/libcls_replica_log.so 2014-05-01 15:30:22.331939 7f3467c387a0 10 register_class replica_log status 3 2014-05-01 15:30:22.331941 7f3467c387a0 10 register_cxx_method replica_log.set flags 3 0x7f3457206b30 2014-05-01 15:30:22.331943 7f3467c387a0 10 register_cxx_method replica_log.get flags 1 0x7f3457205ae0 2014-05-01 15:30:22.331950 7f3467c387a0 10 register_cxx_method replica_log.delete flags 3 0x7f34572065c0 2014-05-01 15:30:22.331952 7f3467c387a0 10 _load_class replica_log success 2014-05-01 15:30:22.331954 7f3467c387a0 10 open_all_classes found log 2014-05-01 15:30:22.331957 7f3467c387a0 10 _get_class adding new class name log 0x29a8d48 2014-05-01 15:30:22.331959 7f3467c387a0 10 _load_class log from /usr/lib64/rados-classes/libcls_log.so 2014-05-01 15:30:22.332092 7f3467c387a0 10 register_class log status 3 2014-05-01 15:30:22.332094 7f3467c387a0 10 register_cxx_method log.add flags 3 0x7f3456ffdad0 2014-05-01 15:30:22.332097 7f3467c387a0 10 register_cxx_method log.list flags 1 0x7f3456fff150 2014-05-01 15:30:22.332100 7f3467c387a0 10 register_cxx_method log.trim flags 3 0x7f3456ffe930 2014-05-01 15:30:22.332102 7f3467c387a0 10 register_cxx_method log.info flags 1 0x7f3457000510 2014-05-01 15:30:22.332104 7f3467c387a0 10 _load_class log success 2014-05-01 15:30:22.332106 7f3467c387a0 10 open_all_classes found version 2014-05-01 15:30:22.332109 7f3467c387a0 10 _get_class adding new class name version 0x29a8b88 2014-05-01 15:30:22.332111 7f3467c387a0 10 _load_class version from /usr/lib64/rados-classes/libcls_version.so 2014-05-01 15:30:22.332246 7f3467c387a0 10 register_class version status 3 2014-05-01 15:30:22.332275 7f3467c387a0 10 register_cxx_method version.set flags 3 0x7f3456df86b0 2014-05-01 15:30:22.332280 7f3467c387a0 10 register_cxx_method version.inc flags 3 0x7f3456df80c0 2014-05-01 15:30:22.332282 7f3467c387a0 10 register_cxx_method version.inc_conds flags 3 0x7f3456df80c0 2014-05-01 15:30:22.332284 7f3467c387a0 10 register_cxx_method version.read flags 1 0x7f3456df7d40 2014-05-01 15:30:22.332286 7f3467c387a0 10 register_cxx_method version.check_conds flags 1 0x7f3456df7730 2014-05-01 15:30:22.332288 7f3467c387a0 10 _load_class version success 2014-05-01 15:30:22.332291 7f3467c387a0 10 open_all_classes found statelog 2014-05-01 15:30:22.332294 7f3467c387a0 10 _get_class adding new class name statelog 0x29a9528 2014-05-01 15:30:22.332296 7f3467c387a0 10 _load_class statelog from /usr/lib64/rados-classes/libcls_statelog.so 2014-05-01 15:30:22.332646 7f3467c387a0 10 register_class statelog status 3 2014-05-01 15:30:22.332679 7f3467c387a0 10 register_cxx_method statelog.add flags 3 0x7f3456bf0060 2014-05-01 15:30:22.332707 7f3467c387a0 10 register_cxx_method statelog.list flags 1 0x7f3456bf1010 2014-05-01 15:30:22.332734 7f3467c387a0 10 register_cxx_method statelog.remove flags 3 0x7f3456bf23b0 2014-05-01 15:30:22.332760 7f3467c387a0 10 register_cxx_method statelog.check_state flags 1 0x7f3456bf0790 2014-05-01 15:30:22.332786 7f3467c387a0 10 _load_class statelog success 2014-05-01 15:30:22.332811 7f3467c387a0 10 open_all_classes found lock 2014-05-01 15:30:22.332836 7f3467c387a0 10 _get_class adding new class name lock 0x29a9368 2014-05-01 15:30:22.332887 7f3467c387a0 10 _load_class lock from /usr/lib64/rados-classes/libcls_lock.so 2014-05-01 15:30:22.333476 7f3467c387a0 10 register_class lock status 3 2014-05-01 15:30:22.333515 7f3467c387a0 10 register_cxx_method lock.lock flags 3 0x7f34569e8120 2014-05-01 15:30:22.333542 7f3467c387a0 10 register_cxx_method lock.unlock flags 3 0x7f34569e7bf0 2014-05-01 15:30:22.333569 7f3467c387a0 10 register_cxx_method lock.break_lock flags 3 0x7f34569e7750 2014-05-01 15:30:22.333595 7f3467c387a0 10 register_cxx_method lock.get_info flags 1 0x7f34569e6300 2014-05-01 15:30:22.333621 7f3467c387a0 10 register_cxx_method lock.list_locks flags 1 0x7f34569e4e40 2014-05-01 15:30:22.333657 7f3467c387a0 10 _load_class lock success 2014-05-01 15:30:22.333721 7f3467c387a0 20 osd.38 0 get_map 30899 - loading and decoding 0x2958c00 2014-05-01 15:30:22.333936 7f3467c387a0 10 osd.38 0 add_map_bl 30899 0 bytes 2014-05-01 15:30:22.336845 7f3467c387a0 -1 *** Caught signal (Aborted) ** in thread 7f3467c387a0 ceph version 0.80-rc1 (6769f4dc88425396921f94e1a37a1c90758aa3ea) 1: /root/ceph-0.80-rc1/src/ceph-osd() [0x9b2da9] 2: /lib64/libpthread.so.0() [0x3318a0f710] 3: (gsignal()+0x35) [0x3318232925] 4: (abort()+0x175) [0x3318234105] 5: (__gnu_cxx::__verbose_terminate_handler()+0x12d) [0x3323ebea5d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- begin dump of recent events --- -234> 2014-05-01 15:30:22.287434 7f3467c387a0 5 asok(0x2930230) register_command perfcounters_dump hook 0x28e0010 -233> 2014-05-01 15:30:22.287497 7f3467c387a0 5 asok(0x2930230) register_command 1 hook 0x28e0010 -232> 2014-05-01 15:30:22.287503 7f3467c387a0 5 asok(0x2930230) register_command perf dump hook 0x28e0010 -231> 2014-05-01 15:30:22.287514 7f3467c387a0 5 asok(0x2930230) register_command perfcounters_schema hook 0x28e0010 -230> 2014-05-01 15:30:22.287522 7f3467c387a0 5 asok(0x2930230) register_command 2 hook 0x28e0010 -229> 2014-05-01 15:30:22.287528 7f3467c387a0 5 asok(0x2930230) register_command perf schema hook 0x28e0010 -228> 2014-05-01 15:30:22.287534 7f3467c387a0 5 asok(0x2930230) register_command config show hook 0x28e0010 -227> 2014-05-01 15:30:22.287541 7f3467c387a0 5 asok(0x2930230) register_command config set hook 0x28e0010 -226> 2014-05-01 15:30:22.287547 7f3467c387a0 5 asok(0x2930230) register_command config get hook 0x28e0010 -225> 2014-05-01 15:30:22.287553 7f3467c387a0 5 asok(0x2930230) register_command log flush hook 0x28e0010 -224> 2014-05-01 15:30:22.287559 7f3467c387a0 5 asok(0x2930230) register_command log dump hook 0x28e0010 -223> 2014-05-01 15:30:22.287565 7f3467c387a0 5 asok(0x2930230) register_command log reopen hook 0x28e0010 -222> 2014-05-01 15:30:22.291380 7f3467c387a0 0 ceph version 0.80-rc1 (6769f4dc88425396921f94e1a37a1c90758aa3ea), process ceph-osd, pid 15289 -221> 2014-05-01 15:30:22.298948 7f3467c387a0 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6850/15289 need_addr=1 -220> 2014-05-01 15:30:22.299036 7f3467c387a0 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6851/15289 need_addr=1 -219> 2014-05-01 15:30:22.299107 7f3467c387a0 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6852/15289 need_addr=1 -218> 2014-05-01 15:30:22.299173 7f3467c387a0 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6853/15289 need_addr=1 -217> 2014-05-01 15:30:22.299245 7f3467c387a0 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6854/15289 need_addr=1 -216> 2014-05-01 15:30:22.299896 7f3467c387a0 1 finished global_init_daemonize -215> 2014-05-01 15:30:22.300293 7f3467c387a0 5 asok(0x2930230) init /var/run/ceph/ceph-osd.38.asok -214> 2014-05-01 15:30:22.300328 7f3467c387a0 5 asok(0x2930230) bind_and_listen /var/run/ceph/ceph-osd.38.asok -213> 2014-05-01 15:30:22.300701 7f3467c387a0 5 asok(0x2930230) register_command 0 hook 0x28d8030 -212> 2014-05-01 15:30:22.300724 7f3467c387a0 5 asok(0x2930230) register_command version hook 0x28d8030 -211> 2014-05-01 15:30:22.300732 7f3467c387a0 5 asok(0x2930230) register_command git_version hook 0x28d8030 -210> 2014-05-01 15:30:22.300742 7f3467c387a0 5 asok(0x2930230) register_command help hook 0x28e0030 -209> 2014-05-01 15:30:22.300749 7f3467c387a0 5 asok(0x2930230) register_command get_command_descriptions hook 0x28e00a0 -208> 2014-05-01 15:30:22.300800 7f346682e700 5 asok(0x2930230) entry start -207> 2014-05-01 15:30:22.301109 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount detected xfs (libxfs) -206> 2014-05-01 15:30:22.301133 7f3467c387a0 1 filestore(/var/lib/ceph/osd.38) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs -205> 2014-05-01 15:30:22.305344 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is supported and appears to work -204> 2014-05-01 15:30:22.305371 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option -203> 2014-05-01 15:30:22.306550 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: syscall(SYS_syncfs, fd) fully supported -202> 2014-05-01 15:30:22.306752 7f3467c387a0 0 xfsfilestorebackend(/var/lib/ceph/osd.38) detect_feature: extsize is supported -201> 2014-05-01 15:30:22.310354 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled -200> 2014-05-01 15:30:22.311414 7f3467c387a0 2 journal open /var/lib/ceph/osd.38/journal fsid 5eb444bd-06a0-40fc-bf3a-023b9f7952da fs_op_seq 108668453 -199> 2014-05-01 15:30:22.311465 7f3467c387a0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway -198> 2014-05-01 15:30:22.311541 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 20: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 -197> 2014-05-01 15:30:22.311591 7f3467c387a0 2 journal No further valid entries found, journal is most likely valid -196> 2014-05-01 15:30:22.311599 7f3467c387a0 2 journal No further valid entries found, journal is most likely valid -195> 2014-05-01 15:30:22.311602 7f3467c387a0 3 journal journal_replay: end of journal, done. -194> 2014-05-01 15:30:22.311624 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 20: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 -193> 2014-05-01 15:30:22.312095 7f3460008700 1 FileStore::op_tp worker finish -192> 2014-05-01 15:30:22.312120 7f3460a09700 1 FileStore::op_tp worker finish -191> 2014-05-01 15:30:22.312130 7f3461e0b700 1 FileStore::op_tp worker finish -190> 2014-05-01 15:30:22.312141 7f346280c700 1 FileStore::op_tp worker finish -189> 2014-05-01 15:30:22.312165 7f346140a700 1 FileStore::op_tp worker finish -188> 2014-05-01 15:30:22.312238 7f3467c387a0 1 journal close /var/lib/ceph/osd.38/journal -187> 2014-05-01 15:30:22.314595 7f3467c387a0 10 monclient(hunting): build_initial_monmap -186> 2014-05-01 15:30:22.314742 7f3467c387a0 5 adding auth protocol: cephx -185> 2014-05-01 15:30:22.314751 7f3467c387a0 5 adding auth protocol: cephx -184> 2014-05-01 15:30:22.314864 7f3467c387a0 1 -- 0.0.0.0:6850/15289 messenger.start -183> 2014-05-01 15:30:22.314898 7f3467c387a0 1 -- :/0 messenger.start -182> 2014-05-01 15:30:22.314919 7f3467c387a0 1 -- 0.0.0.0:6853/15289 messenger.start -181> 2014-05-01 15:30:22.314936 7f3467c387a0 1 -- 0.0.0.0:6852/15289 messenger.start -180> 2014-05-01 15:30:22.314968 7f3467c387a0 1 -- 0.0.0.0:6851/15289 messenger.start -179> 2014-05-01 15:30:22.314999 7f3467c387a0 1 -- 0.0.0.0:6854/15289 messenger.start -178> 2014-05-01 15:30:22.315076 7f3467c387a0 2 osd.38 0 mounting /var/lib/ceph/osd.38 /var/lib/ceph/osd.38/journal -177> 2014-05-01 15:30:22.315139 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount detected xfs (libxfs) -176> 2014-05-01 15:30:22.317250 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is supported and appears to work -175> 2014-05-01 15:30:22.317268 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option -174> 2014-05-01 15:30:22.318229 7f3467c387a0 0 genericfilestorebackend(/var/lib/ceph/osd.38) detect_features: syscall(SYS_syncfs, fd) fully supported -173> 2014-05-01 15:30:22.318346 7f3467c387a0 0 xfsfilestorebackend(/var/lib/ceph/osd.38) detect_feature: extsize is supported -172> 2014-05-01 15:30:22.320891 7f3467c387a0 0 filestore(/var/lib/ceph/osd.38) mount: WRITEAHEAD journal mode explicitly enabled in conf -171> 2014-05-01 15:30:22.321849 7f3467c387a0 2 journal open /var/lib/ceph/osd.38/journal fsid 5eb444bd-06a0-40fc-bf3a-023b9f7952da fs_op_seq 108668453 -170> 2014-05-01 15:30:22.321878 7f3467c387a0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway -169> 2014-05-01 15:30:22.321893 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 21: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 -168> 2014-05-01 15:30:22.321918 7f3467c387a0 2 journal No further valid entries found, journal is most likely valid -167> 2014-05-01 15:30:22.321927 7f3467c387a0 2 journal No further valid entries found, journal is most likely valid -166> 2014-05-01 15:30:22.321930 7f3467c387a0 3 journal journal_replay: end of journal, done. -165> 2014-05-01 15:30:22.321946 7f3467c387a0 1 journal _open /var/lib/ceph/osd.38/journal fd 21: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0 -164> 2014-05-01 15:30:22.322235 7f3467c387a0 2 osd.38 0 boot -163> 2014-05-01 15:30:22.322356 7f3467c387a0 10 osd.38 0 read_superblock sb(d1c527f4-3d28-4e3d-a9c7-c253279b6179 osd.38 5eb444bd-06a0-40fc-bf3a-023b9f7952da e30899 [26307,30899] lci=[30751,30899]) -162> 2014-05-01 15:30:22.322456 7f3467c387a0 10 open_all_classes -161> 2014-05-01 15:30:22.322515 7f3467c387a0 10 open_all_classes found kvs -160> 2014-05-01 15:30:22.322525 7f3467c387a0 10 _get_class adding new class name kvs 0x29a8108 -159> 2014-05-01 15:30:22.322529 7f3467c387a0 10 _load_class kvs from /usr/lib64/rados-classes/libcls_kvs.so -158> 2014-05-01 15:30:22.323110 7f3467c387a0 10 register_class kvs status 3 -157> 2014-05-01 15:30:22.323121 7f3467c387a0 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7f345fbf2b40 -156> 2014-05-01 15:30:22.323126 7f3467c387a0 10 register_cxx_method kvs.get_next_idata flags 1 0x7f345fbf1ba0 -155> 2014-05-01 15:30:22.323128 7f3467c387a0 10 register_cxx_method kvs.get_prev_idata flags 1 0x7f345fbf3590 -154> 2014-05-01 15:30:22.323130 7f3467c387a0 10 register_cxx_method kvs.read_many flags 1 0x7f345fbf2130 -153> 2014-05-01 15:30:22.323132 7f3467c387a0 10 register_cxx_method kvs.check_writable flags 3 0x7f345fbefac0 -152> 2014-05-01 15:30:22.323137 7f3467c387a0 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7f345fbefed0 -151> 2014-05-01 15:30:22.323140 7f3467c387a0 10 register_cxx_method kvs.omap_insert flags 2 0x7f345fbf13b0 -150> 2014-05-01 15:30:22.323143 7f3467c387a0 10 register_cxx_method kvs.create_with_omap flags 2 0x7f345fbf0ff0 -149> 2014-05-01 15:30:22.323145 7f3467c387a0 10 register_cxx_method kvs.omap_remove flags 2 0x7f345fbf0100 -148> 2014-05-01 15:30:22.323147 7f3467c387a0 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7f345fbf0810 -147> 2014-05-01 15:30:22.323149 7f3467c387a0 10 _load_class kvs success -146> 2014-05-01 15:30:22.323152 7f3467c387a0 10 open_all_classes found rgw -145> 2014-05-01 15:30:22.323154 7f3467c387a0 10 _get_class adding new class name rgw 0x29a8648 -144> 2014-05-01 15:30:22.323157 7f3467c387a0 10 _load_class rgw from /usr/lib64/rados-classes/libcls_rgw.so -143> 2014-05-01 15:30:22.327020 7f3467c387a0 1 <cls> cls/rgw/cls_rgw.cc:1599: Loaded rgw class! -142> 2014-05-01 15:30:22.327033 7f3467c387a0 10 register_class rgw status 3 -141> 2014-05-01 15:30:22.327036 7f3467c387a0 10 register_cxx_method rgw.bucket_init_index flags 3 0x7f345f8ab2b0 -140> 2014-05-01 15:30:22.327040 7f3467c387a0 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7f345f8a9620 -139> 2014-05-01 15:30:22.327047 7f3467c387a0 10 register_cxx_method rgw.bucket_list flags 1 0x7f345f8b04e0 -138> 2014-05-01 15:30:22.327050 7f3467c387a0 10 register_cxx_method rgw.bucket_check_index flags 1 0x7f345f8b0090 -137> 2014-05-01 15:30:22.327052 7f3467c387a0 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7f345f8b0330 -136> 2014-05-01 15:30:22.327054 7f3467c387a0 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7f345f8b2150 -135> 2014-05-01 15:30:22.327057 7f3467c387a0 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7f345f8b2ba0 -134> 2014-05-01 15:30:22.327059 7f3467c387a0 10 register_cxx_method rgw.bi_log_list flags 1 0x7f345f8aa930 -133> 2014-05-01 15:30:22.327061 7f3467c387a0 10 register_cxx_method rgw.bi_log_trim flags 3 0x7f345f8abac0 -132> 2014-05-01 15:30:22.327063 7f3467c387a0 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7f345f8b1640 -131> 2014-05-01 15:30:22.327065 7f3467c387a0 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7f345f8af300 -130> 2014-05-01 15:30:22.327067 7f3467c387a0 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7f345f8ac9f0 -129> 2014-05-01 15:30:22.327072 7f3467c387a0 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7f345f8ac650 -128> 2014-05-01 15:30:22.327074 7f3467c387a0 10 register_cxx_method rgw.gc_set_entry flags 3 0x7f345f8ae4d0 -127> 2014-05-01 15:30:22.327077 7f3467c387a0 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7f345f8ae1a0 -126> 2014-05-01 15:30:22.327079 7f3467c387a0 10 register_cxx_method rgw.gc_list flags 1 0x7f345f8ad2e0 -125> 2014-05-01 15:30:22.327084 7f3467c387a0 10 register_cxx_method rgw.gc_remove flags 3 0x7f345f8ae7b0 -124> 2014-05-01 15:30:22.327086 7f3467c387a0 10 _load_class rgw success -123> 2014-05-01 15:30:22.327089 7f3467c387a0 10 open_all_classes found refcount -122> 2014-05-01 15:30:22.327093 7f3467c387a0 10 _get_class adding new class name refcount 0x29a8488 -121> 2014-05-01 15:30:22.327096 7f3467c387a0 10 _load_class refcount from /usr/lib64/rados-classes/libcls_refcount.so -120> 2014-05-01 15:30:22.330561 7f3467c387a0 1 <cls> cls/refcount/cls_refcount.cc:231: Loaded refcount class! -119> 2014-05-01 15:30:22.330572 7f3467c387a0 10 register_class refcount status 3 -118> 2014-05-01 15:30:22.330574 7f3467c387a0 10 register_cxx_method refcount.get flags 3 0x7f3457ad5fe0 -117> 2014-05-01 15:30:22.330578 7f3467c387a0 10 register_cxx_method refcount.put flags 3 0x7f3457ad6580 -116> 2014-05-01 15:30:22.330581 7f3467c387a0 10 register_cxx_method refcount.set flags 3 0x7f3457ad4c20 -115> 2014-05-01 15:30:22.330583 7f3467c387a0 10 register_cxx_method refcount.read flags 1 0x7f3457ad5a40 -114> 2014-05-01 15:30:22.330585 7f3467c387a0 10 _load_class refcount success -113> 2014-05-01 15:30:22.330588 7f3467c387a0 10 open_all_classes found hello -112> 2014-05-01 15:30:22.330593 7f3467c387a0 10 _get_class adding new class name hello 0x29a8aa8 -111> 2014-05-01 15:30:22.330596 7f3467c387a0 10 _load_class hello from /usr/lib64/rados-classes/libcls_hello.so -110> 2014-05-01 15:30:22.330824 7f3467c387a0 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello -109> 2014-05-01 15:30:22.330884 7f3467c387a0 10 register_class hello status 3 -108> 2014-05-01 15:30:22.330912 7f3467c387a0 10 register_cxx_method hello.say_hello flags 1 0x7f345782c1d0 -107> 2014-05-01 15:30:22.330939 7f3467c387a0 10 register_cxx_method hello.record_hello flags 2 0x7f345782c3c0 -106> 2014-05-01 15:30:22.330966 7f3467c387a0 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7f345782c920 -105> 2014-05-01 15:30:22.330992 7f3467c387a0 10 register_cxx_method hello.replay flags 1 0x7f345782c1a0 -104> 2014-05-01 15:30:22.331018 7f3467c387a0 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7f345782cc00 -103> 2014-05-01 15:30:22.331044 7f3467c387a0 10 register_cxx_method hello.bad_reader flags 2 0x7f345782c190 -102> 2014-05-01 15:30:22.331071 7f3467c387a0 10 register_cxx_method hello.bad_writer flags 1 0x7f345782c180 -101> 2014-05-01 15:30:22.331097 7f3467c387a0 10 _load_class hello success -100> 2014-05-01 15:30:22.331124 7f3467c387a0 10 open_all_classes found rbd -99> 2014-05-01 15:30:22.331149 7f3467c387a0 10 _get_class adding new class name rbd 0x29a88e8 -98> 2014-05-01 15:30:22.331175 7f3467c387a0 10 _load_class rbd from /usr/lib64/rados-classes/libcls_rbd.so -97> 2014-05-01 15:30:22.331451 7f3467c387a0 10 register_class rbd status 3 -96> 2014-05-01 15:30:22.331489 7f3467c387a0 10 register_cxx_method rbd.create flags 3 0x7f345761a8e0 -95> 2014-05-01 15:30:22.331496 7f3467c387a0 10 register_cxx_method rbd.get_features flags 1 0x7f3457622360 -94> 2014-05-01 15:30:22.331500 7f3467c387a0 10 register_cxx_method rbd.get_size flags 1 0x7f34576229f0 -93> 2014-05-01 15:30:22.331502 7f3467c387a0 10 register_cxx_method rbd.set_size flags 3 0x7f3457621070 -92> 2014-05-01 15:30:22.331504 7f3467c387a0 10 register_cxx_method rbd.get_snapcontext flags 1 0x7f345761d540 -91> 2014-05-01 15:30:22.331506 7f3467c387a0 10 register_cxx_method rbd.get_object_prefix flags 1 0x7f345761a730 -90> 2014-05-01 15:30:22.331509 7f3467c387a0 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7f3457621ea0 -89> 2014-05-01 15:30:22.331511 7f3467c387a0 10 register_cxx_method rbd.snapshot_add flags 3 0x7f3457620560 -88> 2014-05-01 15:30:22.331513 7f3467c387a0 10 register_cxx_method rbd.snapshot_remove flags 3 0x7f3457621c50 -87> 2014-05-01 15:30:22.331515 7f3467c387a0 10 register_cxx_method rbd.get_all_features flags 1 0x7f34576192f0 -86> 2014-05-01 15:30:22.331517 7f3467c387a0 10 register_cxx_method rbd.copyup flags 3 0x7f3457619270 -85> 2014-05-01 15:30:22.331519 7f3467c387a0 10 register_cxx_method rbd.get_parent flags 1 0x7f3457622e30 -84> 2014-05-01 15:30:22.331521 7f3467c387a0 10 register_cxx_method rbd.set_parent flags 3 0x7f345761ff50 -83> 2014-05-01 15:30:22.331523 7f3467c387a0 10 register_cxx_method rbd.remove_parent flags 3 0x7f345761fd40 -82> 2014-05-01 15:30:22.331548 7f3467c387a0 10 register_cxx_method rbd.set_protection_status flags 3 0x7f34576226c0 -81> 2014-05-01 15:30:22.331551 7f3467c387a0 10 register_cxx_method rbd.get_protection_status flags 1 0x7f34576220f0 -80> 2014-05-01 15:30:22.331553 7f3467c387a0 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7f34576215c0 -79> 2014-05-01 15:30:22.331555 7f3467c387a0 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7f345761c130 -78> 2014-05-01 15:30:22.331557 7f3467c387a0 10 register_cxx_method rbd.add_child flags 3 0x7f345761efa0 -77> 2014-05-01 15:30:22.331560 7f3467c387a0 10 register_cxx_method rbd.remove_child flags 3 0x7f345761e9f0 -76> 2014-05-01 15:30:22.331581 7f3467c387a0 10 register_cxx_method rbd.get_children flags 1 0x7f345761e680 -75> 2014-05-01 15:30:22.331583 7f3467c387a0 10 register_cxx_method rbd.get_id flags 1 0x7f3457619820 -74> 2014-05-01 15:30:22.331585 7f3467c387a0 10 register_cxx_method rbd.set_id flags 3 0x7f345761d950 -73> 2014-05-01 15:30:22.331587 7f3467c387a0 10 register_cxx_method rbd.dir_get_id flags 1 0x7f345761bd30 -72> 2014-05-01 15:30:22.331589 7f3467c387a0 10 register_cxx_method rbd.dir_get_name flags 1 0x7f345761af00 -71> 2014-05-01 15:30:22.331591 7f3467c387a0 10 register_cxx_method rbd.dir_list flags 1 0x7f345761b1f0 -70> 2014-05-01 15:30:22.331593 7f3467c387a0 10 register_cxx_method rbd.dir_add_image flags 3 0x7f345761d300 -69> 2014-05-01 15:30:22.331596 7f3467c387a0 10 register_cxx_method rbd.dir_remove_image flags 3 0x7f345761c940 -68> 2014-05-01 15:30:22.331598 7f3467c387a0 10 register_cxx_method rbd.dir_rename_image flags 3 0x7f345761d020 -67> 2014-05-01 15:30:22.331600 7f3467c387a0 10 register_cxx_method rbd.snap_list flags 1 0x7f345761fa90 -66> 2014-05-01 15:30:22.331602 7f3467c387a0 10 register_cxx_method rbd.snap_add flags 3 0x7f345761e160 -65> 2014-05-01 15:30:22.331604 7f3467c387a0 10 register_cxx_method rbd.snap_remove flags 3 0x7f345761dbe0 -64> 2014-05-01 15:30:22.331606 7f3467c387a0 10 _load_class rbd success -63> 2014-05-01 15:30:22.331609 7f3467c387a0 10 open_all_classes found user -62> 2014-05-01 15:30:22.331612 7f3467c387a0 10 _get_class adding new class name user 0x29a8728 -61> 2014-05-01 15:30:22.331614 7f3467c387a0 10 _load_class user from /usr/lib64/rados-classes/libcls_user.so -60> 2014-05-01 15:30:22.331768 7f3467c387a0 1 <cls> cls/user/cls_user.cc:367: Loaded user class! -59> 2014-05-01 15:30:22.331773 7f3467c387a0 10 register_class user status 3 -58> 2014-05-01 15:30:22.331775 7f3467c387a0 10 register_cxx_method user.set_buckets_info flags 3 0x7f345740e190 -57> 2014-05-01 15:30:22.331779 7f3467c387a0 10 register_cxx_method user.complete_stats_sync flags 3 0x7f345740cd40 -56> 2014-05-01 15:30:22.331781 7f3467c387a0 10 register_cxx_method user.remove_bucket flags 3 0x7f345740f170 -55> 2014-05-01 15:30:22.331783 7f3467c387a0 10 register_cxx_method user.list_buckets flags 1 0x7f345740d350 -54> 2014-05-01 15:30:22.331785 7f3467c387a0 10 register_cxx_method user.get_header flags 1 0x7f345740c9b0 -53> 2014-05-01 15:30:22.331787 7f3467c387a0 10 _load_class user success -52> 2014-05-01 15:30:22.331789 7f3467c387a0 10 open_all_classes found replica_log -51> 2014-05-01 15:30:22.331792 7f3467c387a0 10 _get_class adding new class name replica_log 0x29a8f08 -50> 2014-05-01 15:30:22.331794 7f3467c387a0 10 _load_class replica_log from /usr/lib64/rados-classes/libcls_replica_log.so -49> 2014-05-01 15:30:22.331934 7f3467c387a0 1 <cls> cls/replica_log/cls_replica_log.cc:141: Loaded replica log class! -48> 2014-05-01 15:30:22.331939 7f3467c387a0 10 register_class replica_log status 3 -47> 2014-05-01 15:30:22.331941 7f3467c387a0 10 register_cxx_method replica_log.set flags 3 0x7f3457206b30 -46> 2014-05-01 15:30:22.331943 7f3467c387a0 10 register_cxx_method replica_log.get flags 1 0x7f3457205ae0 -45> 2014-05-01 15:30:22.331950 7f3467c387a0 10 register_cxx_method replica_log.delete flags 3 0x7f34572065c0 -44> 2014-05-01 15:30:22.331952 7f3467c387a0 10 _load_class replica_log success -43> 2014-05-01 15:30:22.331954 7f3467c387a0 10 open_all_classes found log -42> 2014-05-01 15:30:22.331957 7f3467c387a0 10 _get_class adding new class name log 0x29a8d48 -41> 2014-05-01 15:30:22.331959 7f3467c387a0 10 _load_class log from /usr/lib64/rados-classes/libcls_log.so -40> 2014-05-01 15:30:22.332088 7f3467c387a0 1 <cls> cls/log/cls_log.cc:312: Loaded log class! -39> 2014-05-01 15:30:22.332092 7f3467c387a0 10 register_class log status 3 -38> 2014-05-01 15:30:22.332094 7f3467c387a0 10 register_cxx_method log.add flags 3 0x7f3456ffdad0 -37> 2014-05-01 15:30:22.332097 7f3467c387a0 10 register_cxx_method log.list flags 1 0x7f3456fff150 -36> 2014-05-01 15:30:22.332100 7f3467c387a0 10 register_cxx_method log.trim flags 3 0x7f3456ffe930 -35> 2014-05-01 15:30:22.332102 7f3467c387a0 10 register_cxx_method log.info flags 1 0x7f3457000510 -34> 2014-05-01 15:30:22.332104 7f3467c387a0 10 _load_class log success -33> 2014-05-01 15:30:22.332106 7f3467c387a0 10 open_all_classes found version -32> 2014-05-01 15:30:22.332109 7f3467c387a0 10 _get_class adding new class name version 0x29a8b88 -31> 2014-05-01 15:30:22.332111 7f3467c387a0 10 _load_class version from /usr/lib64/rados-classes/libcls_version.so -30> 2014-05-01 15:30:22.332227 7f3467c387a0 1 <cls> cls/version/cls_version.cc:227: Loaded version class! -29> 2014-05-01 15:30:22.332246 7f3467c387a0 10 register_class version status 3 -28> 2014-05-01 15:30:22.332275 7f3467c387a0 10 register_cxx_method version.set flags 3 0x7f3456df86b0 -27> 2014-05-01 15:30:22.332280 7f3467c387a0 10 register_cxx_method version.inc flags 3 0x7f3456df80c0 -26> 2014-05-01 15:30:22.332282 7f3467c387a0 10 register_cxx_method version.inc_conds flags 3 0x7f3456df80c0 -25> 2014-05-01 15:30:22.332284 7f3467c387a0 10 register_cxx_method version.read flags 1 0x7f3456df7d40 -24> 2014-05-01 15:30:22.332286 7f3467c387a0 10 register_cxx_method version.check_conds flags 1 0x7f3456df7730 -23> 2014-05-01 15:30:22.332288 7f3467c387a0 10 _load_class version success -22> 2014-05-01 15:30:22.332291 7f3467c387a0 10 open_all_classes found statelog -21> 2014-05-01 15:30:22.332294 7f3467c387a0 10 _get_class adding new class name statelog 0x29a9528 -20> 2014-05-01 15:30:22.332296 7f3467c387a0 10 _load_class statelog from /usr/lib64/rados-classes/libcls_statelog.so -19> 2014-05-01 15:30:22.332630 7f3467c387a0 1 <cls> cls/statelog/cls_statelog.cc:306: Loaded log class! -18> 2014-05-01 15:30:22.332646 7f3467c387a0 10 register_class statelog status 3 -17> 2014-05-01 15:30:22.332679 7f3467c387a0 10 register_cxx_method statelog.add flags 3 0x7f3456bf0060 -16> 2014-05-01 15:30:22.332707 7f3467c387a0 10 register_cxx_method statelog.list flags 1 0x7f3456bf1010 -15> 2014-05-01 15:30:22.332734 7f3467c387a0 10 register_cxx_method statelog.remove flags 3 0x7f3456bf23b0 -14> 2014-05-01 15:30:22.332760 7f3467c387a0 10 register_cxx_method statelog.check_state flags 1 0x7f3456bf0790 -13> 2014-05-01 15:30:22.332786 7f3467c387a0 10 _load_class statelog success -12> 2014-05-01 15:30:22.332811 7f3467c387a0 10 open_all_classes found lock -11> 2014-05-01 15:30:22.332836 7f3467c387a0 10 _get_class adding new class name lock 0x29a9368 -10> 2014-05-01 15:30:22.332887 7f3467c387a0 10 _load_class lock from /usr/lib64/rados-classes/libcls_lock.so -9> 2014-05-01 15:30:22.333476 7f3467c387a0 10 register_class lock status 3 -8> 2014-05-01 15:30:22.333515 7f3467c387a0 10 register_cxx_method lock.lock flags 3 0x7f34569e8120 -7> 2014-05-01 15:30:22.333542 7f3467c387a0 10 register_cxx_method lock.unlock flags 3 0x7f34569e7bf0 -6> 2014-05-01 15:30:22.333569 7f3467c387a0 10 register_cxx_method lock.break_lock flags 3 0x7f34569e7750 -5> 2014-05-01 15:30:22.333595 7f3467c387a0 10 register_cxx_method lock.get_info flags 1 0x7f34569e6300 -4> 2014-05-01 15:30:22.333621 7f3467c387a0 10 register_cxx_method lock.list_locks flags 1 0x7f34569e4e40 -3> 2014-05-01 15:30:22.333657 7f3467c387a0 10 _load_class lock success -2> 2014-05-01 15:30:22.333721 7f3467c387a0 20 osd.38 0 get_map 30899 - loading and decoding 0x2958c00 -1> 2014-05-01 15:30:22.333936 7f3467c387a0 10 osd.38 0 add_map_bl 30899 0 bytes 0> 2014-05-01 15:30:22.336845 7f3467c387a0 -1 *** Caught signal (Aborted) ** in thread 7f3467c387a0 ceph version 0.80-rc1 (6769f4dc88425396921f94e1a37a1c90758aa3ea) 1: /root/ceph-0.80-rc1/src/ceph-osd() [0x9b2da9] 2: /lib64/libpthread.so.0() [0x3318a0f710] 3: (gsignal()+0x35) [0x3318232925] 4: (abort()+0x175) [0x3318234105] 5: (__gnu_cxx::__verbose_terminate_handler()+0x12d) [0x3323ebea5d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 30/30 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.38.log --- end dump of recent events --- -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140501/2d9fa301/attachment.htm>