Hi, Can you run ls -lah /var/lib/ceph/osd/ceph-10/journal It?s saying it can?t find the journal Regards, Matt On 9 Aug 2014, at 12:51 am, O'Reilly, Dan <Daniel.OReilly at dish.com> wrote: > I?m afraid I don?t know exactly how to interpret this, but after a reboot: > > 2014-08-08 08:48:44.616005 7f0c3b1447a0 0 ceph version 0.80.1 (a38fe1169b6d2ac98b427334c12d7cf81f809b74), process ceph-osd, pid 2978 > 2014-08-08 08:48:44.635680 7f0c3b1447a0 0 filestore(/var/lib/ceph/osd/ceph-10) mount detected xfs (libxfs) > 2014-08-08 08:48:44.635730 7f0c3b1447a0 1 filestore(/var/lib/ceph/osd/ceph-10) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs > 2014-08-08 08:48:44.681911 7f0c3b1447a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_features: FIEMAP ioctl is supported and appears to work > 2014-08-08 08:48:44.681959 7f0c3b1447a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option > 2014-08-08 08:48:44.748483 7f0c3b1447a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_features: syscall(SYS_syncfs, fd) fully supported > 2014-08-08 08:48:44.748605 7f0c3b1447a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_feature: extsize is supported > 2014-08-08 08:48:44.889826 7f0c3b1447a0 0 filestore(/var/lib/ceph/osd/ceph-10) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled > 2014-08-08 08:48:45.064198 7f0c3b1447a0 -1 filestore(/var/lib/ceph/osd/ceph-10) mount failed to open journal /var/lib/ceph/osd/ceph-10/journal: (2) No such file or directory > 2014-08-08 08:48:45.074220 7f0c3b1447a0 -1 ** ERROR: error converting store /var/lib/ceph/osd/ceph-10: (2) No such file or directory > 2014-08-08 08:49:19.957725 7f2c40c1a7a0 0 ceph version 0.80.1 (a38fe1169b6d2ac98b427334c12d7cf81f809b74), process ceph-osd, pid 4707 > 2014-08-08 08:49:19.973896 7f2c40c1a7a0 0 filestore(/var/lib/ceph/osd/ceph-10) mount detected xfs (libxfs) > 2014-08-08 08:49:19.973931 7f2c40c1a7a0 1 filestore(/var/lib/ceph/osd/ceph-10) disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs > 2014-08-08 08:49:20.016413 7f2c40c1a7a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_features: FIEMAP ioctl is supported and appears to work > 2014-08-08 08:49:20.016444 7f2c40c1a7a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option > 2014-08-08 08:49:20.083052 7f2c40c1a7a0 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_features: syscall(SYS_syncfs, fd) fully supported > 2014-08-08 08:49:20.083179 7f2c40c1a7a0 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-10) detect_feature: extsize is supported > 2014-08-08 08:49:20.134213 7f2c40c1a7a0 0 filestore(/var/lib/ceph/osd/ceph-10) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled > 2014-08-08 08:49:20.136710 7f2c40c1a7a0 -1 filestore(/var/lib/ceph/osd/ceph-10) mount failed to open journal /var/lib/ceph/osd/ceph-10/journal: (2) No such file or directory > 2014-08-08 08:49:20.146797 7f2c40c1a7a0 -1 ** ERROR: error converting store /var/lib/ceph/osd/ceph-10: (2) No such file or directory > > From: German Anders [mailto:ganders at despegar.com] > Sent: Friday, August 08, 2014 8:23 AM > To: O'Reilly, Dan > Cc: Karan Singh; ceph-users at lists.ceph.com > Subject: Re: [ceph-users] Can't start OSD > > How about the logs? Is something there? > > ls /var/log/ceph/ > > German Anders > > > > > > > > > > > > > > > > > --- Original message --- > Asunto: Re: [ceph-users] Can't start OSD > De: "O'Reilly, Dan" <Daniel.OReilly at dish.com> > Para: Karan Singh <karan.singh at csc.fi> > Cc: ceph-users at lists.ceph.com <ceph-users at lists.ceph.com> > Fecha: Friday, 08/08/2014 10:53 > > > Nope. Nothing works. This is VERY frustrating. > > What happened: > > - I rebooted the box, simulating a system failure. > - When the system came back up, ceph wasn?t started, and the osd volumes weren?t mounted. > - I did a ?service ceph start osd? and the ceph processes don?t start > - I did a ?ceph-deploy activate? on the devices, so they?re mounted. ?service ceph start? still doesn?t start anything. > > Right now: > > # service ceph restart > === osd.18 === > === osd.18 === > Stopping Ceph osd.18 on tm1cldosdl04...done > === osd.18 === > create-or-move updated item name 'osd.18' weight 0.45 at location {host=tm1cldosdl04,root=default} to crush map > Starting Ceph osd.18 on tm1cldosdl04... > starting osd.18 at :/0 osd_data /var/lib/ceph/osd/ceph-18 /var/lib/ceph/osd/ceph-18/journal > === osd.17 === > === osd.17 === > Stopping Ceph osd.17 on tm1cldosdl04...done > === osd.17 === > create-or-move updated item name 'osd.17' weight 0.45 at location {host=tm1cldosdl04,root=default} to crush map > Starting Ceph osd.17 on tm1cldosdl04... > starting osd.17 at :/0 osd_data /var/lib/ceph/osd/ceph-17 /var/lib/ceph/osd/ceph-17/journal > === osd.19 === > === osd.19 === > Stopping Ceph osd.19 on tm1cldosdl04...done > === osd.19 === > create-or-move updated item name 'osd.19' weight 0.45 at location {host=tm1cldosdl04,root=default} to crush map > Starting Ceph osd.19 on tm1cldosdl04... > starting osd.19 at :/0 osd_data /var/lib/ceph/osd/ceph-19 /var/lib/ceph/osd/ceph-19/journal > === osd.16 === > === osd.16 === > Stopping Ceph osd.16 on tm1cldosdl04...done > === osd.16 === > create-or-move updated item name 'osd.16' weight 0.45 at location {host=tm1cldosdl04,root=default} to crush map > Starting Ceph osd.16 on tm1cldosdl04... > starting osd.16 at :/0 osd_data /var/lib/ceph/osd/ceph-16 /var/lib/ceph/osd/ceph-16/journal > [NEW:note: root at tm1cldosdl04 on parent: /root] > # ps -eaf|grep ceph > root 7528 6124 0 07:32 pts/0 00:00:00 grep ceph > [NEW:note: root at tm1cldosdl04 on parent: /root] > # ceph osd tree > # id weight type name up/down reweight > -1 9 root default > -2 1.8 host tm1cldosdl01 > 0 0.45 osd.0 up 1 > 1 0.45 osd.1 up 1 > 2 0.45 osd.2 up 1 > 3 0.45 osd.3 up 1 > -3 1.8 host tm1cldosdl02 > 4 0.45 osd.4 up 1 > 5 0.45 osd.5 up 1 > 6 0.45 osd.6 up 1 > 7 0.45 osd.7 up 1 > -4 1.8 host tm1cldosdl03 > 8 0.45 osd.8 up 1 > 9 0.45 osd.9 up 1 > 10 0.45 osd.10 up 1 > 11 0.45 osd.11 up 1 > -5 3.6 host tm1cldosdl04 > 12 0.45 osd.12 DNE > 13 0.45 osd.13 DNE > 14 0.45 osd.14 DNE > 15 0.45 osd.15 DNE > 16 0.45 osd.16 down 0 > 17 0.45 osd.17 down 0 > 18 0.45 osd.18 down 0 > 19 0.45 osd.19 down 0 > > I?m missing something here. I don?t know if it?s a config issue or what. But the docs haven?t helped me. > > From: Karan Singh [mailto:karan.singh at csc.fi] > Sent: Friday, August 08, 2014 1:11 AM > To: O'Reilly, Dan > Cc: ceph-users at lists.ceph.com > Subject: Re: [ceph-users] Can't start OSD > > Try to make these OSD IN > > ceph osd in osd.12 osd.13 osd.14 osd.15 > > Then restart osd services > > > - Karan Singh - > > On 08 Aug 2014, at 00:55, O'Reilly, Dan <Daniel.OReilly at dish.com> wrote: > > > > # id weight type name up/down reweight > -1 7.2 root default > -2 1.8 host tm1cldosdl01 > 0 0.45 osd.0 up 1 > 1 0.45 osd.1 up 1 > 2 0.45 osd.2 up 1 > 3 0.45 osd.3 up 1 > -3 1.8 host tm1cldosdl02 > 4 0.45 osd.4 up 1 > 5 0.45 osd.5 up 1 > 6 0.45 osd.6 up 1 > 7 0.45 osd.7 up 1 > -4 1.8 host tm1cldosdl03 > 8 0.45 osd.8 up 1 > 9 0.45 osd.9 up 1 > 10 0.45 osd.10 up 1 > 11 0.45 osd.11 up 1 > -5 1.8 host tm1cldosdl04 > 12 0.45 osd.12 down 0 > 13 0.45 osd.13 down 0 > 14 0.45 osd.14 down 0 > 15 0.45 osd.15 down 0 > [ceph at tm1cldosdl04 ~]$ sudo /etc/init.d/ceph start osd.12 > /etc/init.d/ceph: osd.12 not found (/etc/ceph/ceph.conf defines , /var/lib/ceph defines ) > > What am I missing? Specifically, what would need to be in ceph.conf or /var/lib/ceph? > > Dan O'Reilly > UNIX Systems Administration > <image001.jpg> > 9601 S. Meridian Blvd. > Englewood, CO 80112 > 720-514-6293 > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140809/e86f882a/attachment.htm>
 |