Can you check that the dependencies have started? Anything about those in the logs? network-online.target local-fs.target ceph-create-keys@%i.service Jan > On 13 May 2016, at 14:30, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > This is starting to sound like a xenial systemd issue to me. Maybe poke > the canonical folks? > > You might edit the unit file and make it touch something in /tmp instead > of starting Ceph just to rule out ceph... > > sage > > > On Fri, 13 May 2016, Wido den Hollander wrote: > >> No luck either. After a reboot only the Ceph OSD starts, but the monitor not. >> >> I have checked: >> - service is enabled >> - tried to re-enable the service >> - check the MON logs to see if it was started, it wasn't >> - systemd log to see if it wants to start the MON, it doesn't >> >> My systemd-foo isn't that good either, so I don't know what is happening here. >> >> Wido >> >>> Op 12 mei 2016 om 15:31 schreef Jan Schermer <jan@xxxxxxxxxxx>: >>> >>> >>> Btw try replacing >>> >>> WantedBy=ceph-mon.target >>> >>> With: WantedBy=default.target >>> then systemctl daemon-reload. >>> >>> See if that does the trick >>> >>> I only messed with systemctl to have my own services start, I still hope it goes away eventually... :P >>> >>> Jan >>> >>>> On 12 May 2016, at 15:01, Wido den Hollander <wido@xxxxxxxx> wrote: >>>> >>>> >>>> To also answer Sage's question: No, this is a fresh Jewel install in a few test VMs. This system was not upgraded. >>>> >>>> It was installed 2 hours ago. >>>> >>>>> Op 12 mei 2016 om 14:51 schreef Jan Schermer <jan@xxxxxxxxxxx>: >>>>> >>>>> >>>>> Can you post the contents of ceph-mon@.service file? >>>>> >>>> >>>> Yes, here you go: >>>> >>>> root@charlie:~# cat /lib/systemd/system/ceph-mon@.service >>>> [Unit] >>>> Description=Ceph cluster monitor daemon >>>> >>>> # According to: >>>> # http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget >>>> # these can be removed once ceph-mon will dynamically change network >>>> # configuration. >>>> After=network-online.target local-fs.target ceph-create-keys@%i.service >>>> Wants=network-online.target local-fs.target ceph-create-keys@%i.service >>>> >>>> PartOf=ceph-mon.target >>>> >>>> [Service] >>>> LimitNOFILE=1048576 >>>> LimitNPROC=1048576 >>>> EnvironmentFile=-/etc/default/ceph >>>> Environment=CLUSTER=ceph >>>> ExecStart=/usr/bin/ceph-mon -f --cluster ${CLUSTER} --id %i --setuser ceph --setgroup ceph >>>> ExecReload=/bin/kill -HUP $MAINPID >>>> PrivateDevices=yes >>>> ProtectHome=true >>>> ProtectSystem=full >>>> PrivateTmp=true >>>> TasksMax=infinity >>>> Restart=on-failure >>>> StartLimitInterval=30min >>>> StartLimitBurst=3 >>>> >>>> [Install] >>>> WantedBy=ceph-mon.target >>>> root@charlie:~# >>>> >>>>> what does >>>>> systemctl is-enabled ceph-mon@charlie >>>>> say? >>>>> >>>> >>>> root@charlie:~# systemctl is-enabled ceph-mon@charlie >>>> enabled >>>> root@charlie:~# >>>> >>>>> However, this looks like it was just started at a bad moment and died - nothing in logs? >>>>> >>>> >>>> No, I checked the ceph-mon logs in /var/log/ceph. No sign of it even trying to start after boot. In /var/log/syslog there also is not a trace of ceph-mon. >>>> >>>> Only the OSD starts. >>>> >>>> Wido >>>> >>>>> Jan >>>>> >>>>> >>>>>> On 12 May 2016, at 14:44, Sage Weil <sage@xxxxxxxxxxxx> wrote: >>>>>> >>>>>> On Thu, 12 May 2016, Wido den Hollander wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I am setting up a Jewel cluster in VMs with Ubuntu 16.04. >>>>>>> >>>>>>> ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9) >>>>>>> >>>>>>> After a reboot the Ceph Monitors don't start and I have to do so manually. >>>>>>> >>>>>>> Three machines, alpha, bravo and charlie all have the same problem. >>>>>>> >>>>>>> root@charlie:~# systemctl status ceph-mon@charlie >>>>>>> ● ceph-mon@charlie.service - Ceph cluster monitor daemon >>>>>>> Loaded: loaded (/lib/systemd/system/ceph-mon@.service; enabled; vendor preset: enabled) >>>>>>> Active: inactive (dead) >>>>>>> root@charlie:~# >>>>>>> >>>>>>> I can start it and it works >>>>>> >>>>>> Hmm.. my systemd-fu is weak, but if it's enabled it seems like it shoud >>>>>> come up. >>>>>> >>>>>> Was this an upgraded package? What if you do 'systemctl reenable >>>>>> ceph-mon@charlie'? >>>>>> >>>>>> sage >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> root@charlie:~# systemctl start ceph-mon@charlie >>>>>>> root@charlie:~# systemctl status ceph-mon@charlie >>>>>>> ● ceph-mon@charlie.service - Ceph cluster monitor daemon >>>>>>> Loaded: loaded (/lib/systemd/system/ceph-mon@.service; enabled; vendor preset: enabled) >>>>>>> Active: active (running) since Thu 2016-05-12 16:08:56 CEST; 1s ago >>>>>>> Main PID: 1368 (ceph-mon) >>>>>>> >>>>>>> I tried removing the /var/log/ceph/ceph-mon.charlie.log file and reboot to see if the mon was actually invoked, but it wasn't. >>>>>>> >>>>>>> ceph.target has been started and so is the OSD on the machine. It is just the monitor which hasn't been started. >>>>>>> >>>>>>> In the syslog I see: >>>>>>> >>>>>>> May 12 16:11:19 charlie systemd[1]: Starting Ceph object storage daemon... >>>>>>> May 12 16:11:19 charlie systemd[1]: Starting LSB: Start Ceph distributed file system daemons at boot time... >>>>>>> May 12 16:11:19 charlie systemd[1]: Started LSB: Start Ceph distributed file system daemons at boot time. >>>>>>> May 12 16:11:20 charlie systemd[1]: Started Ceph object storage daemon. >>>>>>> May 12 16:11:20 charlie systemd[1]: Started Ceph disk activation: /dev/sdb2. >>>>>>> May 12 16:11:21 charlie systemd[1]: Started Ceph object storage daemon. >>>>>>> May 12 16:11:21 charlie systemd[1]: Started Ceph disk activation: /dev/sdb1. >>>>>>> >>>>>>> Am I missing something or is this a bug? >>>>>>> >>>>>>> Wido >>>>>>> _______________________________________________ >>>>>>> ceph-users mailing list >>>>>>> ceph-users@xxxxxxxxxxxxxx >>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>>> _______________________________________________ >>>>>> ceph-users mailing list >>>>>> ceph-users@xxxxxxxxxxxxxx >>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>> >>> >> _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |