On Tue, 13 Oct 2015, Luis Periquito wrote: > Hi Sage, > > awesome help. > > Sorry for not telling before, but I'm running 2xMON in precise and > 1xMON in trusty. Looking at the status page > (http://ceph.com/gitbuilder.cgi) it seems the precise build is > failing... Can you have a look? I've repushed the branch, this time cherry-picking the master fix. Let me know if you run into other problems! Thanks- sage > > thanks, > > > On Tue, Oct 13, 2015 at 2:59 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > On Tue, 13 Oct 2015, Loic Dachary wrote: > >> https://github.com/ceph/ceph/compare/hammer...wip-ecpool-hammer > >> > >> In order to bypass the crush verification, you could: > >> > >> ceph tell mon.* injectargs --crushtool /bin/true > > > > Ah, good trick! > > > > http://tracker.ceph.com/issues/13477 > > > > is the ticket, and my fix for master is > > > > https://github.com/ceph/ceph/pull/6246 > > > > sage > > > >> > >> Cheers > >> > >> On 13/10/2015 15:41, Sage Weil wrote: > >> > On Tue, 13 Oct 2015, Luis Periquito wrote: > >> >> the store.db dir is 3.4GB big :( > >> >> > >> >> can I do it on my side? > >> > > >> > Nevermind, I was able to reproduce it from the bugzilla. I've pushed a > >> > branch wip-ecpool-hammer. Not sure which distro you're on, but packages > >> > will appear at gitbuilder.ceph.com in 30-45 minutes. This fixes the mon > >> > crash, which will let you delete the pool. I suggest stopping the OSDs > >> > before starting the mon with this or else they might get pg create > >> > messages and crash too. Once the pool is removed you can start them > >> > again. They shouldn't need to be upgraded. > >> > > >> > Note that the latest hammer doesn't let you create the pool at all because > >> > it fails the crush safety check (I had to disable the check to reproduce > >> > this), so that's good at least! > >> > > >> > sage > >> > > >> >> > >> >> On Tue, Oct 13, 2015 at 2:25 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > >> >>> On Tue, 13 Oct 2015, Luis Periquito wrote: > >> >>>> Any ideas? I'm growing desperate :( > >> >>>> > >> >>>> I've tried compiling from source, and including > >> >>>> https://github.com/ceph/ceph/pull/5276, but it still crashes on boot > >> >>>> of the ceph-mon > >> >>> > >> >>> If you can email a (link to a) tarball of your mon data directory I'd love > >> >>> to extract the osdmap and see why crush is crashing.. it's obviously not > >> >>> supposed to do that (even with a bad rule). You can also use > >> >>> the ceph-post-file utility. > >> >>> > >> >>> Thanks! > >> >>> sage > >> >>> > >> >>> > >> >>>> > >> >>>> ---------- Forwarded message ---------- > >> >>>> From: Luis Periquito <periquito@xxxxxxxxx> > >> >>>> Date: Tue, Oct 13, 2015 at 12:26 PM > >> >>>> Subject: Re: monitor crashing > >> >>>> To: Ceph Users <ceph-users@xxxxxxxxxxxxxx> > >> >>>> > >> >>>> > >> >>>> I'm currently running Hammer (0.94.3), created an invalid LRC profile > >> >>>> (typo in the l=, should have been l=4 but was l=3, and now I don't > >> >>>> have enough different ruleset-locality) and created a pool. Is there > >> >>>> any way to delete this pool? remember I can't start the ceph-mon... > >> >>>> > >> >>>> On Tue, Oct 13, 2015 at 11:56 AM, Luis Periquito <periquito@xxxxxxxxx> wrote: > >> >>>>> It seems I've hit this bug: > >> >>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1231630 > >> >>>>> > >> >>>>> is there any way I can recover this cluster? It worked in our test > >> >>>>> cluster, but crashed the production one... > >> >>>> -- > >> >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >> >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx > >> >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> >>>> > >> >>>> > >> >> -- > >> >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >> >> the body of a message to majordomo@xxxxxxxxxxxxxxx > >> >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> >> > >> >> > >> > -- > >> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > >> > the body of a message to majordomo@xxxxxxxxxxxxxxx > >> > More majordomo info at http://vger.kernel.org/majordomo-info.html > >> > > >> > >> -- > >> Loïc Dachary, Artisan Logiciel Libre > >> > >> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > >
