Le mercredi 12 septembre 2018 à 12:35 +0200, Michael Scherer a écrit :
> Le mardi 11 septembre 2018 à 19:54 +0530, Nigel Babu a écrit :
> > On Tue, Sep 11, 2018 at 7:06 PM Michael Scherer <misc@xxxxxxxxxx>
> > wrote:
> >
> > > And... rescue mode is not working. So the server is down until
> > > Rackspace fix it.
> > >
> > > Can someone disable the freebsd smoke test, as I think our 2nd
> > > builder
> > > is not yet building fine ?
> > >
> >
> >
> > Disabled. Please do not merge any JJB review requests until this is
> > fixed.
>
> So, just to keep people updated on this adventure:
>
> - after 3 or 4h, the rescue mode managed to appear. Turn out that it
> take time to copy the rescue image to the cloud, and I guess no one
> recently did that for freebsd. Rackspace say "40 minutes", which is
> still a lot.
>
> This morning, I was greeted by a welcoming prompt saying "zfs, can't
> mount the root" or something, because the rescue mode of rackspace
> was
> a bit broken. Trying to fix it, I did reboot out of rescue mode.
>
> So I went back to my initial plan: "boot in rescue mode".
>
> Thanks to surhuman reflexes I acquired dodging nerfs guns in the
> office, I manage to hit the "s" key at the right time. While it seems
> easy, the remote console of rackspace is a bit slow to show the boot
> loader, the latency from France over the atlantic is a noticable, and
> the interface disconnect itself every minutes of idling, and every
> time
> the video mode is changed (like when it go from POST to bootloader
> menu). So that was a race between the machine and me, and I won it,
> with a "#_ " prompt waiting for me.
>
> Of course, things would be too simple if that was just that, so / was
> readonly. And that's freebsd, so "mount -o rw,remount /" didn't work.
> It didn't work for 2 reasons:
>
> - zfs do not work like this ("zfs set readonly=off zroot", for people
> asking how to do later)
>
> - the keyboard was kinda broken. Not broke like "that's a us layout
> and
> misc is using a french layout" broken, cause that, I know how to deal
> with it. More broken with "only a-z keys are working, and the rest
> are
> randomly placed somewhere else". It turn out that without '.' and
> '/',
> thing can be complicated in the shell. But I am full of ressources,
> and
> judicious use of <tab> + <backspace> did let me get what I needed.
>
> So I did manage to change the root password, reboot again.
>
> Password didn't work the first time (did I mention latency), I retry,
> it worked.
>
> And so, network is fine. But sshd didn't start. Why ? because it
> check
> the config, and the config did show a warning on "duplicate line".
>
> The upgrade did change sshd config file, adding a ton of comment, and
> at the end, a duplicated line:
>
> Subsystem sftp /usr/libexec/sftp-server
>
> And upon removal, things were working.
>
> However, I did reboot to test and ... still broken. So back to me
> racing against the bootloader I guess. (cause the temp password I set
> is not working again, wonder if there was some fallback due to zfs or
> something).
So for a reason I do not understand, the reboot seems to have reverted
the server to the initial broken state. I would blame either the cloud,
or zfs. Anyway, now the server is working, I am upgrading userspace and
cleaning a bit.
But... I of course hit this bug:
https://github.com/freebsd/pkg/issues/902
But the -f option do seems to work fine.
So now, after a reboot to verify, a few upgrade, the system is up to
date. I will reenable as a voting member, if there is a issue, please
tell us by opening a bug (with link to the failure).
--
Michael Scherer
Sysadmin, Community Infrastructure and Platform, OSAS
Attachment:
signature.asc
Description: This is a digitally signed message part
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-devel
