> Op 13 mei 2016 om 14:56 schreef Jan Schermer <jan@xxxxxxxxxxx>: > > > Can you check that the dependencies have started? Anything about those in the logs? > > network-online.target local-fs.target ceph-create-keys@%i.service > May 13 16:59:15 alpha systemd[1]: Reached target Local File Systems (Pre). May 13 16:59:15 alpha systemd[1]: Reached target Local File Systems. .. .. May 13 16:59:18 alpha systemd[1]: Reached target Network. May 13 16:59:18 alpha systemd[1]: Reached target Network is Online. In the systemd logs there is no trace of 'ceph-create-keys@%i.service' starting up, so that seems to be the cullprit. Wido > Jan > > > > On 13 May 2016, at 14:30, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > > > This is starting to sound like a xenial systemd issue to me. Maybe poke > > the canonical folks? > > > > You might edit the unit file and make it touch something in /tmp instead > > of starting Ceph just to rule out ceph... > > > > sage > > > > > > On Fri, 13 May 2016, Wido den Hollander wrote: > > > >> No luck either. After a reboot only the Ceph OSD starts, but the monitor not. > >> > >> I have checked: > >> - service is enabled > >> - tried to re-enable the service > >> - check the MON logs to see if it was started, it wasn't > >> - systemd log to see if it wants to start the MON, it doesn't > >> > >> My systemd-foo isn't that good either, so I don't know what is happening here. > >> > >> Wido > >> > >>> Op 12 mei 2016 om 15:31 schreef Jan Schermer <jan@xxxxxxxxxxx>: > >>> > >>> > >>> Btw try replacing > >>> > >>> WantedBy=ceph-mon.target > >>> > >>> With: WantedBy=default.target > >>> then systemctl daemon-reload. > >>> > >>> See if that does the trick > >>> > >>> I only messed with systemctl to have my own services start, I still hope it goes away eventually... :P > >>> > >>> Jan > >>> > >>>> On 12 May 2016, at 15:01, Wido den Hollander <wido@xxxxxxxx> wrote: > >>>> > >>>> > >>>> To also answer Sage's question: No, this is a fresh Jewel install in a few test VMs. This system was not upgraded. > >>>> > >>>> It was installed 2 hours ago. > >>>> > >>>>> Op 12 mei 2016 om 14:51 schreef Jan Schermer <jan@xxxxxxxxxxx>: > >>>>> > >>>>> > >>>>> Can you post the contents of ceph-mon@.service file? > >>>>> > >>>> > >>>> Yes, here you go: > >>>> > >>>> root@charlie:~# cat /lib/systemd/system/ceph-mon@.service > >>>> [Unit] > >>>> Description=Ceph cluster monitor daemon > >>>> > >>>> # According to: > >>>> # http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget > >>>> # these can be removed once ceph-mon will dynamically change network > >>>> # configuration. > >>>> After=network-online.target local-fs.target ceph-create-keys@%i.service > >>>> Wants=network-online.target local-fs.target ceph-create-keys@%i.service > >>>> > >>>> PartOf=ceph-mon.target > >>>> > >>>> [Service] > >>>> LimitNOFILE=1048576 > >>>> LimitNPROC=1048576 > >>>> EnvironmentFile=-/etc/default/ceph > >>>> Environment=CLUSTER=ceph > >>>> ExecStart=/usr/bin/ceph-mon -f --cluster ${CLUSTER} --id %i --setuser ceph --setgroup ceph > >>>> ExecReload=/bin/kill -HUP $MAINPID > >>>> PrivateDevices=yes > >>>> ProtectHome=true > >>>> ProtectSystem=full > >>>> PrivateTmp=true > >>>> TasksMax=infinity > >>>> Restart=on-failure > >>>> StartLimitInterval=30min > >>>> StartLimitBurst=3 > >>>> > >>>> [Install] > >>>> WantedBy=ceph-mon.target > >>>> root@charlie:~# > >>>> > >>>>> what does > >>>>> systemctl is-enabled ceph-mon@charlie > >>>>> say? > >>>>> > >>>> > >>>> root@charlie:~# systemctl is-enabled ceph-mon@charlie > >>>> enabled > >>>> root@charlie:~# > >>>> > >>>>> However, this looks like it was just started at a bad moment and died - nothing in logs? > >>>>> > >>>> > >>>> No, I checked the ceph-mon logs in /var/log/ceph. No sign of it even trying to start after boot. In /var/log/syslog there also is not a trace of ceph-mon. > >>>> > >>>> Only the OSD starts. > >>>> > >>>> Wido > >>>> > >>>>> Jan > >>>>> > >>>>> > >>>>>> On 12 May 2016, at 14:44, Sage Weil <sage@xxxxxxxxxxxx> wrote: > >>>>>> > >>>>>> On Thu, 12 May 2016, Wido den Hollander wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> I am setting up a Jewel cluster in VMs with Ubuntu 16.04. > >>>>>>> > >>>>>>> ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9) > >>>>>>> > >>>>>>> After a reboot the Ceph Monitors don't start and I have to do so manually. > >>>>>>> > >>>>>>> Three machines, alpha, bravo and charlie all have the same problem. > >>>>>>> > >>>>>>> root@charlie:~# systemctl status ceph-mon@charlie > >>>>>>> ● ceph-mon@charlie.service - Ceph cluster monitor daemon > >>>>>>> Loaded: loaded (/lib/systemd/system/ceph-mon@.service; enabled; vendor preset: enabled) > >>>>>>> Active: inactive (dead) > >>>>>>> root@charlie:~# > >>>>>>> > >>>>>>> I can start it and it works > >>>>>> > >>>>>> Hmm.. my systemd-fu is weak, but if it's enabled it seems like it shoud > >>>>>> come up. > >>>>>> > >>>>>> Was this an upgraded package? What if you do 'systemctl reenable > >>>>>> ceph-mon@charlie'? > >>>>>> > >>>>>> sage > >>>>>> > >>>>>> > >>>>>> > >>>>>>> > >>>>>>> root@charlie:~# systemctl start ceph-mon@charlie > >>>>>>> root@charlie:~# systemctl status ceph-mon@charlie > >>>>>>> ● ceph-mon@charlie.service - Ceph cluster monitor daemon > >>>>>>> Loaded: loaded (/lib/systemd/system/ceph-mon@.service; enabled; vendor preset: enabled) > >>>>>>> Active: active (running) since Thu 2016-05-12 16:08:56 CEST; 1s ago > >>>>>>> Main PID: 1368 (ceph-mon) > >>>>>>> > >>>>>>> I tried removing the /var/log/ceph/ceph-mon.charlie.log file and reboot to see if the mon was actually invoked, but it wasn't. > >>>>>>> > >>>>>>> ceph.target has been started and so is the OSD on the machine. It is just the monitor which hasn't been started. > >>>>>>> > >>>>>>> In the syslog I see: > >>>>>>> > >>>>>>> May 12 16:11:19 charlie systemd[1]: Starting Ceph object storage daemon... > >>>>>>> May 12 16:11:19 charlie systemd[1]: Starting LSB: Start Ceph distributed file system daemons at boot time... > >>>>>>> May 12 16:11:19 charlie systemd[1]: Started LSB: Start Ceph distributed file system daemons at boot time. > >>>>>>> May 12 16:11:20 charlie systemd[1]: Started Ceph object storage daemon. > >>>>>>> May 12 16:11:20 charlie systemd[1]: Started Ceph disk activation: /dev/sdb2. > >>>>>>> May 12 16:11:21 charlie systemd[1]: Started Ceph object storage daemon. > >>>>>>> May 12 16:11:21 charlie systemd[1]: Started Ceph disk activation: /dev/sdb1. > >>>>>>> > >>>>>>> Am I missing something or is this a bug? > >>>>>>> > >>>>>>> Wido > >>>>>>> _______________________________________________ > >>>>>>> ceph-users mailing list > >>>>>>> ceph-users@xxxxxxxxxxxxxx > >>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>>>>> _______________________________________________ > >>>>>> ceph-users mailing list > >>>>>> ceph-users@xxxxxxxxxxxxxx > >>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>>>> > >>> > >> > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |