Sorry, forgot to point out this: 2013-07-29 20:46:26.366916 7f4f28c6e700 0 -- 2.4.1.7:6801/13319 >> 2.4.1.8:6802/18344 pipe(0x284378 0 sd=30 :53729 s=1 pgs=0 cs=0 l=0).connect claims to be 2.4.1.8:6802/17272 not 2.4.1.8:6802/18344 - wrong node! Not sure what that means? > -----Original Message----- > From: ceph-users-bounces@xxxxxxxxxxxxxx [mailto:ceph-users- > bounces@xxxxxxxxxxxxxx] On Behalf Of Don Talton (dotalton) > Sent: Monday, July 29, 2013 1:49 PM > To: Gregory Farnum > Cc: ceph-users@xxxxxxxxxxxxxx > Subject: Re: OSD never gets marked up? > > The entirety of the osd log file is below. I tried this on both bobtail and > cuttlefish. Bobtail I noticed errors about all of the xfs features not being > supported, which has gone away in cuttlefish. So I am assuming that issue is > resolved. I don't see any other errors > > 2013-07-29 20:46:25.813383 7f4f3e909780 0 ceph version 0.61.7 > (8f010aff684e820ecc837c25ac77c7a05d71 91ff), process ceph- > osd, pid 13319 > 2013-07-29 20:46:25.814051 7f4f3e909780 1 -- 2.4.1.7:0/0 learned my addr > 2.4.1.7:0/0 > 2013-07-29 20:46:25.814068 7f4f3e909780 1 accepter.accepter.bind > my_inst.addr is 2.4.1.7:6800/13319 need_addr=0 > 2013-07-29 20:46:25.814085 7f4f3e909780 1 -- 2.4.1.7:0/0 learned my addr > 2.4.1.7:0/0 > 2013-07-29 20:46:25.814090 7f4f3e909780 1 accepter.accepter.bind > my_inst.addr is 2.4.1.7:6801/13319 need_addr=0 > 2013-07-29 20:46:25.814103 7f4f3e909780 1 -- 2.4.1.7:0/0 learned my addr > 2.4.1.7:0/0 > 2013-07-29 20:46:25.814108 7f4f3e909780 1 accepter.accepter.bind > my_inst.addr is 2.4.1.7:6802/13319 need_addr=0 > 2013-07-29 20:46:25.893876 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > supported and appears to work > 2013-07-29 20:46:25.893892 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > disabled via 'filestore fiemap' config option > 2013-07-29 20:46:25.894237 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount did NOT detect b > trfs > 2013-07-29 20:46:25.943640 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount syncfs(2) syscal l > fully supported (by glibc and kernel) > 2013-07-29 20:46:25.943784 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount found snaps <> > 2013-07-29 20:46:26.027401 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount: enabling WRITEA > HEAD journal mode: btrfs not detected > 2013-07-29 20:46:26.034732 7f4f3e909780 -1 journal FileJournal::_open: > disabling aio for non-block j ournal. Use journal_force_aio to > force use of aio anyway > 2013-07-29 20:46:26.034828 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 20: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.035140 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 20: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.036189 7f4f3e909780 1 journal close > /var/lib/ceph/osd/osd.0/journal > 2013-07-29 20:46:26.036905 7f4f3e909780 1 -- 2.4.1.7:6800/13319 > messenger.start > 2013-07-29 20:46:26.036958 7f4f3e909780 1 -- :/0 messenger.start > 2013-07-29 20:46:26.036975 7f4f3e909780 1 -- 2.4.1.7:6802/13319 > messenger.start > 2013-07-29 20:46:26.036993 7f4f3e909780 1 -- 2.4.1.7:6801/13319 > messenger.start > 2013-07-29 20:46:26.102317 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > supported and appears to work > 2013-07-29 20:46:26.102329 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > disabled via 'filestore fiemap' config option > 2013-07-29 20:46:26.102643 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount did NOT detect b > trfs > 2013-07-29 20:46:26.201852 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount syncfs(2) syscal l > fully supported (by glibc and kernel) > 2013-07-29 20:46:26.201928 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount found snaps <> > 2013-07-29 20:46:26.268809 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount: enabling WRITEA > HEAD journal mode: btrfs not detected > 2013-07-29 20:46:26.272557 7f4f3e909780 -1 journal FileJournal::_open: > disabling aio for non-block j ournal. Use journal_force_aio to > force use of aio anyway > 2013-07-29 20:46:26.272577 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 28: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.272819 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 28: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.359004 7f4f3e909780 1 accepter.accepter.start > 2013-07-29 20:46:26.359104 7f4f3e909780 1 accepter.accepter.start > 2013-07-29 20:46:26.359226 7f4f3e909780 1 accepter.accepter.start > 2013-07-29 20:46:26.360175 7f4f3e909780 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- ?+0 > 0x28086c0 con 0x285a160 > 2013-07-29 20:46:26.362067 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 1 ==== mon _map v1 ==== 191+0+0 > (4283215978 0 0) 0x2d68000 con 0x285a160 > 2013-07-29 20:46:26.362175 7f4f31580700 1 monclient(hunting): found mon.0 > 2013-07-29 20:46:26.362183 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 2 ==== aut h_reply(proto 2 0 Success) v1 ==== > 33+0+0 (658115354 0 0) 0x2d68400 con 0x285a160 > 2013-07-29 20:46:26.362381 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 > 0x2d66b40 con 0x285a160 > 2013-07-29 20:46:26.363316 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 3 ==== aut h_reply(proto 2 0 Success) v1 ==== > 206+0+0 (453976051 0 0) 0x2d68200 con 0x285a160 > 2013-07-29 20:46:26.363470 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 > 0x2d66d80 con 0x285a160 > 2013-07-29 20:46:26.364569 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 4 ==== aut h_reply(proto 2 0 Success) v1 ==== > 393+0+0 (184374621 0 0) 0x2d68600 con 0x285a160 > 2013-07-29 20:46:26.364662 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- mon_subscribe ({monmap=0+}) v2 -- ?+0 > 0x2829700 con 0x285a160 > 2013-07-29 20:46:26.364690 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- mon_subscribe > ({monmap=0+,osd_pg_creates=0}) v2 -- ?+0 0x2829a80 con 0x285a160 > 2013-07-29 20:46:26.364759 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- ?+0 > 0x2d66b40 con 0x285a160 > 2013-07-29 20:46:26.364866 7f4f3e909780 5 monclient: authenticate success, > global_id 4267 > 2013-07-29 20:46:26.365466 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 5 ==== mon _map v1 ==== 191+0+0 > (4283215978 0 0) 0x2d68400 con 0x285a160 > 2013-07-29 20:46:26.365518 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 6 ==== mon _subscribe_ack(300s) v1 ==== > 20+0+0 (1055034598 0 0) 0x2829a80 con 0x285a160 > 2013-07-29 20:46:26.365555 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 7 ==== mon _map v1 ==== 191+0+0 > (4283215978 0 0) 0x2d68000 con 0x285a160 > 2013-07-29 20:46:26.365581 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 8 ==== mon _subscribe_ack(300s) v1 ==== > 20+0+0 (1055034598 0 0) 0x2829700 con 0x285a160 > 2013-07-29 20:46:26.365732 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 9 ==== aut h_reply(proto 2 0 Success) v1 ==== > 194+0+0 (3275840853 0 0) 0x2d68800 con 0x285a160 > 2013-07-29 20:46:26.366916 7f4f28c6e700 0 -- 2.4.1.7:6801/13319 >> > 2.4.1.8:6802/18344 pipe(0x284378 0 sd=30 :53729 s=1 pgs=0 > cs=0 l=0).connect claims to be 2.4.1.8:6802/17272 not 2.4.1.8:6802/18344 - > wrong node! > 2013-07-29 20:46:26.367028 7f4f28c6e700 0 -- 2.4.1.7:6801/13319 >> > 2.4.1.8:6802/18344 pipe(0x284378 0 sd=30 :53729 s=1 pgs=0 > cs=0 l=0).fault with nothing to send, going to standby > 2013-07-29 20:46:26.369488 7f4f3e909780 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- mon_get_versi on(what=osdmap handle=1) > v1 -- ?+0 0x2829540 con 0x285a160 > 2013-07-29 20:46:26.370168 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 10 ==== mo n_check_map_ack(handle=1 > version=5) v2 ==== 24+0+0 (3346734780 0 0) 0x2829e00 con 0x285a160 > 2013-07-29 20:46:26.370303 7f4f2d578700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- osd_boot(osd. 0 booted 0 v64) v3 -- ?+0 > 0x2833400 con 0x285a160 > root@ceph-osd0:~# cat /var/log/ceph/ceph-osd.0.log > 2013-07-29 20:46:25.813383 7f4f3e909780 0 ceph version 0.61.7 > (8f010aff684e820ecc837c25ac77c7a05d71 91ff), process ceph- > osd, pid 13319 > 2013-07-29 20:46:25.814051 7f4f3e909780 1 -- 2.4.1.7:0/0 learned my addr > 2.4.1.7:0/0 > 2013-07-29 20:46:25.814068 7f4f3e909780 1 accepter.accepter.bind > my_inst.addr is 2.4.1.7:6800/13319 need_addr=0 > 2013-07-29 20:46:25.814085 7f4f3e909780 1 -- 2.4.1.7:0/0 learned my addr > 2.4.1.7:0/0 > 2013-07-29 20:46:25.814090 7f4f3e909780 1 accepter.accepter.bind > my_inst.addr is 2.4.1.7:6801/13319 need_addr=0 > 2013-07-29 20:46:25.814103 7f4f3e909780 1 -- 2.4.1.7:0/0 learned my addr > 2.4.1.7:0/0 > 2013-07-29 20:46:25.814108 7f4f3e909780 1 accepter.accepter.bind > my_inst.addr is 2.4.1.7:6802/13319 need_addr=0 > 2013-07-29 20:46:25.893876 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > supported and appears to work > 2013-07-29 20:46:25.893892 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > disabled via 'filestore fiemap' config option > 2013-07-29 20:46:25.894237 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount did NOT detect b > trfs > 2013-07-29 20:46:25.943640 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount syncfs(2) syscal l > fully supported (by glibc and kernel) > 2013-07-29 20:46:25.943784 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount found snaps <> > 2013-07-29 20:46:26.027401 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount: enabling WRITEA > HEAD journal mode: btrfs not detected > 2013-07-29 20:46:26.034732 7f4f3e909780 -1 journal FileJournal::_open: > disabling aio for non-block j ournal. Use journal_force_aio to > force use of aio anyway > 2013-07-29 20:46:26.034828 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 20: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.035140 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 20: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.036189 7f4f3e909780 1 journal close > /var/lib/ceph/osd/osd.0/journal > 2013-07-29 20:46:26.036905 7f4f3e909780 1 -- 2.4.1.7:6800/13319 > messenger.start > 2013-07-29 20:46:26.036958 7f4f3e909780 1 -- :/0 messenger.start > 2013-07-29 20:46:26.036975 7f4f3e909780 1 -- 2.4.1.7:6802/13319 > messenger.start > 2013-07-29 20:46:26.036993 7f4f3e909780 1 -- 2.4.1.7:6801/13319 > messenger.start > 2013-07-29 20:46:26.102317 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > supported and appears to work > 2013-07-29 20:46:26.102329 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount FIEMAP ioctl is > disabled via 'filestore fiemap' config option > 2013-07-29 20:46:26.102643 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount did NOT detect b > trfs > 2013-07-29 20:46:26.201852 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount syncfs(2) syscal l > fully supported (by glibc and kernel) > 2013-07-29 20:46:26.201928 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount found snaps <> > 2013-07-29 20:46:26.268809 7f4f3e909780 0 > filestore(/var/lib/ceph/osd/osd.0) mount: enabling WRITEA > HEAD journal mode: btrfs not detected > 2013-07-29 20:46:26.272557 7f4f3e909780 -1 journal FileJournal::_open: > disabling aio for non-block j ournal. Use journal_force_aio to > force use of aio anyway > 2013-07-29 20:46:26.272577 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 28: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.272819 7f4f3e909780 1 journal _open > /var/lib/ceph/osd/osd.0/journal fd 28: 4294 967296 bytes, > block size 4096 bytes, directio = 1, aio = 0 > 2013-07-29 20:46:26.359004 7f4f3e909780 1 accepter.accepter.start > 2013-07-29 20:46:26.359104 7f4f3e909780 1 accepter.accepter.start > 2013-07-29 20:46:26.359226 7f4f3e909780 1 accepter.accepter.start > 2013-07-29 20:46:26.360175 7f4f3e909780 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- ?+0 > 0x28086c0 con 0x285a160 > 2013-07-29 20:46:26.362067 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 1 ==== mon _map v1 ==== 191+0+0 > (4283215978 0 0) 0x2d68000 con 0x285a160 > 2013-07-29 20:46:26.362175 7f4f31580700 1 monclient(hunting): found mon.0 > 2013-07-29 20:46:26.362183 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 2 ==== aut h_reply(proto 2 0 Success) v1 ==== > 33+0+0 (658115354 0 0) 0x2d68400 con 0x285a160 > 2013-07-29 20:46:26.362381 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 > 0x2d66b40 con 0x285a160 > 2013-07-29 20:46:26.363316 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 3 ==== aut h_reply(proto 2 0 Success) v1 ==== > 206+0+0 (453976051 0 0) 0x2d68200 con 0x285a160 > 2013-07-29 20:46:26.363470 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 > 0x2d66d80 con 0x285a160 > 2013-07-29 20:46:26.364569 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 4 ==== aut h_reply(proto 2 0 Success) v1 ==== > 393+0+0 (184374621 0 0) 0x2d68600 con 0x285a160 > 2013-07-29 20:46:26.364662 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- mon_subscribe ({monmap=0+}) v2 -- ?+0 > 0x2829700 con 0x285a160 > 2013-07-29 20:46:26.364690 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- mon_subscribe > ({monmap=0+,osd_pg_creates=0}) v2 -- ?+0 0x2829a80 con 0x285a160 > 2013-07-29 20:46:26.364759 7f4f31580700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- ?+0 > 0x2d66b40 con 0x285a160 > 2013-07-29 20:46:26.364866 7f4f3e909780 5 monclient: authenticate success, > global_id 4267 > 2013-07-29 20:46:26.365466 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 5 ==== mon _map v1 ==== 191+0+0 > (4283215978 0 0) 0x2d68400 con 0x285a160 > 2013-07-29 20:46:26.365518 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 6 ==== mon _subscribe_ack(300s) v1 ==== > 20+0+0 (1055034598 0 0) 0x2829a80 con 0x285a160 > 2013-07-29 20:46:26.365555 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 7 ==== mon _map v1 ==== 191+0+0 > (4283215978 0 0) 0x2d68000 con 0x285a160 > 2013-07-29 20:46:26.365581 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 8 ==== mon _subscribe_ack(300s) v1 ==== > 20+0+0 (1055034598 0 0) 0x2829700 con 0x285a160 > 2013-07-29 20:46:26.365732 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 9 ==== aut h_reply(proto 2 0 Success) v1 ==== > 194+0+0 (3275840853 0 0) 0x2d68800 con 0x285a160 > 2013-07-29 20:46:26.366916 7f4f28c6e700 0 -- 2.4.1.7:6801/13319 >> > 2.4.1.8:6802/18344 pipe(0x284378 0 sd=30 :53729 s=1 pgs=0 > cs=0 l=0).connect claims to be 2.4.1.8:6802/17272 not 2.4.1.8:6802/18344 - > wrong node! > 2013-07-29 20:46:26.367028 7f4f28c6e700 0 -- 2.4.1.7:6801/13319 >> > 2.4.1.8:6802/18344 pipe(0x284378 0 sd=30 :53729 s=1 pgs=0 > cs=0 l=0).fault with nothing to send, going to standby > 2013-07-29 20:46:26.369488 7f4f3e909780 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- mon_get_versi on(what=osdmap handle=1) > v1 -- ?+0 0x2829540 con 0x285a160 > 2013-07-29 20:46:26.370168 7f4f31580700 1 -- 2.4.1.7:6800/13319 <== mon.0 > 2.4.1.4:6789/0 10 ==== mo n_check_map_ack(handle=1 > version=5) v2 ==== 24+0+0 (3346734780 0 0) 0x2829e00 con 0x285a160 > 2013-07-29 20:46:26.370303 7f4f2d578700 1 -- 2.4.1.7:6800/13319 --> > 2.4.1.4:6789/0 -- osd_boot(osd. 0 booted 0 v64) v3 -- ?+0 > 0x2833400 con 0x285a160 > > > > > > > > > -----Original Message----- > > From: Gregory Farnum [mailto:greg@xxxxxxxxxxx] > > Sent: Monday, July 29, 2013 12:17 PM > > To: Don Talton (dotalton) > > Cc: ceph-users@xxxxxxxxxxxxxx > > Subject: Re: OSD never gets marked up? > > > > On Mon, Jul 29, 2013 at 11:36 AM, Don Talton (dotalton) > > <dotalton@xxxxxxxxx> wrote: > > > Hello, > > > > > > I have a small test cluster that I deploy using puppet-ceph. Both > > > the MON > > and the OSDs deploy properly, and appear to have all of the correct > > configurations. However, the OSDs are never marked as up. Any input is > > appreciated. The daemons are running on each OSD server, the OSDs are > > listed in the crushmap, and I can see them successfully authenticate > > with the MON eg. > > > > > > 2013-07-29 18:34:19.231905 7fbfeaaa5700 1 -- 2.4.1.7:0/14269 <== > > > mon.0 2.4.1.4:6789/0 8 ==== > > > mon_command_ack([osd,crush,create-or- > > move,0,0.91,root=default,host=cep > > > h-osd0]=0 create-or-move updated item id 0 name 'osd.0' weight 0.91 > > > at location {host=ceph-osd0,root=default} to crush map v8) v1 ==== > > > 223+0+0 (1170550300 0 0) 0x7fbfe00016d0 con 0x16b1770create-or-move > > > updated item id 0 name 'osd.0' weight 0.91 at location > > > {host=ceph-osd0,root=default} to crush map > > > > This is actually not the OSDs doing this, but the ceph admin tool > > using the bootstrap-osd key. Do you have anything that's from the > > OSDs? There might be something helpful in the OSD logs; if not you can > > add "debug ms = 1" and restart them and there certainly will be. > > -Greg > > Software Engineer #42 @ http://inktank.com | http://ceph.com > > > > > > > > > > > My ceph.conf > > > > > > [global] > > > auth cluster required = cephx > > > auth service required = cephx > > > auth client required = cephx > > > keyring = /etc/ceph/keyring > > > > > > fsid = e80afa94-a64c-486c-9e34-d55e85f26406 > > > debug ms = 1/5 > > > > > > [mon] > > > mon data = /var/lib/ceph/mon/mon.$id > > > debug mon = 20 > > > debug paxos = 1/5 > > > debug auth = 2 > > > > > > [osd] > > > osd journal size = 4096 > > > cluster network = 2.4.1.0/24 > > > public network = 2.4.1.0/24 > > > filestore flusher = false > > > osd data = /var/lib/ceph/osd/osd.$id > > > osd journal = /var/lib/ceph/osd/osd.$id/journal > > > osd mkfs type = xfs > > > keyring = /var/lib/ceph/osd/osd.$id/keyring > > > debug osd = 1/5 > > > debug filestore = 1/5 > > > debug journal = 1 > > > debug monc = 5/20 > > > > > > [mds] > > > mds data = /var/lib/ceph/mds/mds.$id > > > keyring = /var/lib/ceph/mds/mds.$id/keyring > > > > > > [mon.0] > > > host = ceph-mon0 > > > mon addr = 2.4.1.4:6789 > > > _______________________________________________ > > > ceph-users mailing list > > > ceph-users@xxxxxxxxxxxxxx > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |