Hello, just testing the latest luminous rc packages on debian stretch with bluestore OSDs. OSDs without a separate block.db partition do fine. But when I try to create an OSD with a separate block.db partition: ( e.g., ceph-deploy osd create --bluestore --block-db=/dev/nvme0bnp1 node1:/dev/sdi ) then all MON daemons fail and the cluster stops running (cf. appended journalctl logs) Do you have any idea how to narrow down the problem? (objdump?) (please note that I faked /etc/debian_version to jessie, since ceph-deploy 1.5.38 from https://download.ceph.com/debian-luminous/dists/stretch/ does not yet support stretch - but I suppose that's not related to my problem). Kind regards, Thomas
Jul 07 09:58:54 node1 systemd[1]: Started Ceph cluster monitor daemon. Jul 07 09:58:54 node1 ceph-mon[550]: starting mon.node1 rank 0 at x.x.x.x:6789/0 mon_data /var/lib/ceph/mon/ceph-node1 fsid 1e50b861-c10f-4356-9af6-3a90441ee694 Jul 07 16:38:44 node1 ceph-mon[550]: /build/ceph-12.1.0/src/mon/OSDMonitor.cc: In function 'void OSDMonitor::check_pg_creates_subs()' thread 7f678fe61700 time 2017-07-07 16:38:44.576052 Jul 07 16:38:44 node1 ceph-mon[550]: /build/ceph-12.1.0/src/mon/OSDMonitor.cc: 2977: FAILED assert(osdmap.get_up_osd_features() & CEPH_FEATURE_MON_STATEFUL_SUB) Jul 07 16:38:44 node1 ceph-mon[550]: ceph version 12.1.0 (262617c9f16c55e863693258061c5b25dea5b086) luminous (dev) Jul 07 16:38:44 node1 ceph-mon[550]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55fc324c8802] Jul 07 16:38:44 node1 ceph-mon[550]: 2: (()+0x474ed0) [0x55fc323c8ed0] Jul 07 16:38:44 node1 ceph-mon[550]: 3: (OSDMonitor::update_from_paxos(bool*)+0x1a4d) [0x55fc323f168d] Jul 07 16:38:44 node1 ceph-mon[550]: 4: (PaxosService::refresh(bool*)+0x3ff) [0x55fc323b673f] Jul 07 16:38:44 node1 ceph-mon[550]: 5: (Monitor::refresh_from_paxos(bool*)+0x1a3) [0x55fc322767e3] Jul 07 16:38:44 node1 ceph-mon[550]: 6: (Paxos::do_refresh()+0x47) [0x55fc323a0227] Jul 07 16:38:44 node1 ceph-mon[550]: 7: (Paxos::commit_finish()+0x703) [0x55fc323b1ad3] Jul 07 16:38:44 node1 ceph-mon[550]: 8: (C_Committed::finish(int)+0x2b) [0x55fc323b55bb] Jul 07 16:38:44 node1 ceph-mon[550]: 9: (Context::complete(int)+0x9) [0x55fc322b2c19] Jul 07 16:38:44 node1 ceph-mon[550]: 10: (MonitorDBStore::C_DoTransaction::finish(int)+0xa0) [0x55fc323b33c0] Jul 07 16:38:44 node1 ceph-mon[550]: 11: (Context::complete(int)+0x9) [0x55fc322b2c19] Jul 07 16:38:44 node1 ceph-mon[550]: 12: (Finisher::finisher_thread_entry()+0x4c0) [0x55fc324c7950] Jul 07 16:38:44 node1 ceph-mon[550]: 13: (()+0x7494) [0x7f679dd95494] Jul 07 16:38:44 node1 ceph-mon[550]: 14: (clone()+0x3f) [0x7f679b27baff] Jul 07 16:38:44 node1 ceph-mon[550]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. Jul 07 16:38:44 node1 ceph-mon[550]: 2017-07-07 16:38:44.581790 7f678fe61700 -1 /build/ceph-12.1.0/src/mon/OSDMonitor.cc: In function 'void OSDMonitor::check_pg_creates_subs()' thread 7f678f Jul 07 16:38:44 node1 ceph-mon[550]: /build/ceph-12.1.0/src/mon/OSDMonitor.cc: 2977: FAILED assert(osdmap.get_up_osd_features() & CEPH_FEATURE_MON_STATEFUL_SUB) Jul 07 16:38:44 node1 ceph-mon[550]: ceph version 12.1.0 (262617c9f16c55e863693258061c5b25dea5b086) luminous (dev) Jul 07 16:38:44 node1 ceph-mon[550]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55fc324c8802] Jul 07 16:38:44 node1 ceph-mon[550]: 2: (()+0x474ed0) [0x55fc323c8ed0] Jul 07 16:38:44 node1 ceph-mon[550]: 3: (OSDMonitor::update_from_paxos(bool*)+0x1a4d) [0x55fc323f168d] Jul 07 16:38:44 node1 ceph-mon[550]: 4: (PaxosService::refresh(bool*)+0x3ff) [0x55fc323b673f] Jul 07 16:38:44 node1 ceph-mon[550]: 5: (Monitor::refresh_from_paxos(bool*)+0x1a3) [0x55fc322767e3] Jul 07 16:38:44 node1 ceph-mon[550]: 6: (Paxos::do_refresh()+0x47) [0x55fc323a0227] Jul 07 16:38:44 node1 ceph-mon[550]: 7: (Paxos::commit_finish()+0x703) [0x55fc323b1ad3] Jul 07 16:38:44 node1 ceph-mon[550]: 8: (C_Committed::finish(int)+0x2b) [0x55fc323b55bb] Jul 07 16:38:44 node1 ceph-mon[550]: 9: (Context::complete(int)+0x9) [0x55fc322b2c19] Jul 07 16:38:44 node1 ceph-mon[550]: 10: (MonitorDBStore::C_DoTransaction::finish(int)+0xa0) [0x55fc323b33c0] Jul 07 16:38:44 node1 ceph-mon[550]: 11: (Context::complete(int)+0x9) [0x55fc322b2c19] Jul 07 16:38:44 node1 ceph-mon[550]: 12: (Finisher::finisher_thread_entry()+0x4c0) [0x55fc324c7950] Jul 07 16:38:44 node1 ceph-mon[550]: 13: (()+0x7494) [0x7f679dd95494] Jul 07 16:38:44 node1 ceph-mon[550]: 14: (clone()+0x3f) [0x7f679b27baff] Jul 07 16:38:44 node1 ceph-mon[550]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. Jul 07 16:38:44 node1 ceph-mon[550]: 0> 2017-07-07 16:38:44.581790 7f678fe61700 -1 /build/ceph-12.1.0/src/mon/OSDMonitor.cc: In function 'void OSDMonitor::check_pg_creates_subs()' threa Jul 07 16:38:44 node1 ceph-mon[550]: /build/ceph-12.1.0/src/mon/OSDMonitor.cc: 2977: FAILED assert(osdmap.get_up_osd_features() & CEPH_FEATURE_MON_STATEFUL_SUB) Jul 07 16:38:44 node1 ceph-mon[550]: ceph version 12.1.0 (262617c9f16c55e863693258061c5b25dea5b086) luminous (dev) ...
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
