This is starting to sound like a xenial systemd issue to me. Maybe poke the canonical folks? You might edit the unit file and make it touch something in /tmp instead of starting Ceph just to rule out ceph... sage On Fri, 13 May 2016, Wido den Hollander wrote: > No luck either. After a reboot only the Ceph OSD starts, but the monitor not. > > I have checked: > - service is enabled > - tried to re-enable the service > - check the MON logs to see if it was started, it wasn't > - systemd log to see if it wants to start the MON, it doesn't > > My systemd-foo isn't that good either, so I don't know what is happening here. > > Wido > > > Op 12 mei 2016 om 15:31 schreef Jan Schermer <jan@xxxxxxxxxxx>: > > > > > > Btw try replacing > > > > WantedBy=ceph-mon.target > > > > With: WantedBy=default.target > > then systemctl daemon-reload. > > > > See if that does the trick > > > > I only messed with systemctl to have my own services start, I still hope it goes away eventually... :P > > > > Jan > > > > > On 12 May 2016, at 15:01, Wido den Hollander <wido@xxxxxxxx> wrote: > > > > > > > > > To also answer Sage's question: No, this is a fresh Jewel install in a few test VMs. This system was not upgraded. > > > > > > It was installed 2 hours ago. > > > > > >> Op 12 mei 2016 om 14:51 schreef Jan Schermer <jan@xxxxxxxxxxx>: > > >> > > >> > > >> Can you post the contents of ceph-mon@.service file? > > >> > > > > > > Yes, here you go: > > > > > > root@charlie:~# cat /lib/systemd/system/ceph-mon@.service > > > [Unit] > > > Description=Ceph cluster monitor daemon > > > > > > # According to: > > > # http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget > > > # these can be removed once ceph-mon will dynamically change network > > > # configuration. > > > After=network-online.target local-fs.target ceph-create-keys@%i.service > > > Wants=network-online.target local-fs.target ceph-create-keys@%i.service > > > > > > PartOf=ceph-mon.target > > > > > > [Service] > > > LimitNOFILE=1048576 > > > LimitNPROC=1048576 > > > EnvironmentFile=-/etc/default/ceph > > > Environment=CLUSTER=ceph > > > ExecStart=/usr/bin/ceph-mon -f --cluster ${CLUSTER} --id %i --setuser ceph --setgroup ceph > > > ExecReload=/bin/kill -HUP $MAINPID > > > PrivateDevices=yes > > > ProtectHome=true > > > ProtectSystem=full > > > PrivateTmp=true > > > TasksMax=infinity > > > Restart=on-failure > > > StartLimitInterval=30min > > > StartLimitBurst=3 > > > > > > [Install] > > > WantedBy=ceph-mon.target > > > root@charlie:~# > > > > > >> what does > > >> systemctl is-enabled ceph-mon@charlie > > >> say? > > >> > > > > > > root@charlie:~# systemctl is-enabled ceph-mon@charlie > > > enabled > > > root@charlie:~# > > > > > >> However, this looks like it was just started at a bad moment and died - nothing in logs? > > >> > > > > > > No, I checked the ceph-mon logs in /var/log/ceph. No sign of it even trying to start after boot. In /var/log/syslog there also is not a trace of ceph-mon. > > > > > > Only the OSD starts. > > > > > > Wido > > > > > >> Jan > > >> > > >> > > >>> On 12 May 2016, at 14:44, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > >>> > > >>> On Thu, 12 May 2016, Wido den Hollander wrote: > > >>>> Hi, > > >>>> > > >>>> I am setting up a Jewel cluster in VMs with Ubuntu 16.04. > > >>>> > > >>>> ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9) > > >>>> > > >>>> After a reboot the Ceph Monitors don't start and I have to do so manually. > > >>>> > > >>>> Three machines, alpha, bravo and charlie all have the same problem. > > >>>> > > >>>> root@charlie:~# systemctl status ceph-mon@charlie > > >>>> ● ceph-mon@charlie.service - Ceph cluster monitor daemon > > >>>> Loaded: loaded (/lib/systemd/system/ceph-mon@.service; enabled; vendor preset: enabled) > > >>>> Active: inactive (dead) > > >>>> root@charlie:~# > > >>>> > > >>>> I can start it and it works > > >>> > > >>> Hmm.. my systemd-fu is weak, but if it's enabled it seems like it shoud > > >>> come up. > > >>> > > >>> Was this an upgraded package? What if you do 'systemctl reenable > > >>> ceph-mon@charlie'? > > >>> > > >>> sage > > >>> > > >>> > > >>> > > >>>> > > >>>> root@charlie:~# systemctl start ceph-mon@charlie > > >>>> root@charlie:~# systemctl status ceph-mon@charlie > > >>>> ● ceph-mon@charlie.service - Ceph cluster monitor daemon > > >>>> Loaded: loaded (/lib/systemd/system/ceph-mon@.service; enabled; vendor preset: enabled) > > >>>> Active: active (running) since Thu 2016-05-12 16:08:56 CEST; 1s ago > > >>>> Main PID: 1368 (ceph-mon) > > >>>> > > >>>> I tried removing the /var/log/ceph/ceph-mon.charlie.log file and reboot to see if the mon was actually invoked, but it wasn't. > > >>>> > > >>>> ceph.target has been started and so is the OSD on the machine. It is just the monitor which hasn't been started. > > >>>> > > >>>> In the syslog I see: > > >>>> > > >>>> May 12 16:11:19 charlie systemd[1]: Starting Ceph object storage daemon... > > >>>> May 12 16:11:19 charlie systemd[1]: Starting LSB: Start Ceph distributed file system daemons at boot time... > > >>>> May 12 16:11:19 charlie systemd[1]: Started LSB: Start Ceph distributed file system daemons at boot time. > > >>>> May 12 16:11:20 charlie systemd[1]: Started Ceph object storage daemon. > > >>>> May 12 16:11:20 charlie systemd[1]: Started Ceph disk activation: /dev/sdb2. > > >>>> May 12 16:11:21 charlie systemd[1]: Started Ceph object storage daemon. > > >>>> May 12 16:11:21 charlie systemd[1]: Started Ceph disk activation: /dev/sdb1. > > >>>> > > >>>> Am I missing something or is this a bug? > > >>>> > > >>>> Wido > > >>>> _______________________________________________ > > >>>> ceph-users mailing list > > >>>> ceph-users@xxxxxxxxxxxxxx > > >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >>> _______________________________________________ > > >>> ceph-users mailing list > > >>> ceph-users@xxxxxxxxxxxxxx > > >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >> > > > >
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
