Found it. The target was not enabled:
root@0cc47a6df14e:~# systemctl status
ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853.target
● ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853.target - Ceph cluster
03977a23-f00f-4bb0-b9a7-de57f40ba853
Loaded: loaded
(/etc/systemd/system/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853.target;
enabled; vendor preset: enabled)
Active: inactive (dead)
Am Sa., 16. Sept. 2023 um 13:29 Uhr schrieb Boris <bb@xxxxxxxxx>:
> The other hosts are still online and the cluster only lost 1/3 of its
> services.
>
>
>
> > Am 16.09.2023 um 12:53 schrieb Eugen Block <eblock@xxxxxx>:
> >
> > I don’t have time to look into all the details, but I’m wondering how
> you seem to be able to start mgr services with the orchestrator if all mgr
> daemons are down. The orchestrator is a mgr module, so that’s a bit weird,
> isn’t it?
> >
> > Zitat von Boris Behrens <bb@xxxxxxxxx>:
> >
> >> Hi Eugen,
> >> the test-test cluster where we started with simple ceph and the adoption
> >> when straight forward are working fine.
> >>
> >> But this test cluster was all over the place.
> >> We had an old running update via orchestrator which was still in the
> >> pipeline, the adoption process was stopped a year ago and now got
> picked up
> >> again, and so on and so forth.
> >>
> >> But now we have it clean, at least we think it's clean.
> >>
> >> After a reboot, the services are not available. I have to start the via
> >> ceph orch.
> >> root@0cc47a6df14e:~# systemctl list-units | grep ceph
> >> ceph-crash.service
> >> loaded active running Ceph crash dump collector
> >> ceph-fuse.target
> >> loaded active active ceph target allowing to
> start/stop
> >> all ceph-fuse@.service instances at once
> >> ceph-mds.target
> >> loaded active active ceph target allowing to start/stop
> >> all ceph-mds@.service instances at once
> >> ceph-mgr.target
> >> loaded active active ceph target allowing to start/stop
> >> all ceph-mgr@.service instances at once
> >> ceph-mon.target
> >> loaded active active ceph target allowing to start/stop
> >> all ceph-mon@.service instances at once
> >> ceph-osd.target
> >> loaded active active ceph target allowing to start/stop
> >> all ceph-osd@.service instances at once
> >> ceph-radosgw.target
> >> loaded active active ceph target allowing to start/stop
> >> all ceph-radosgw@.service instances at once
> >> ceph.target
> >> loaded active active All Ceph clusters and services
> >> root@0cc47a6df14e:~# ceph orch start mgr
> >> Scheduled to start mgr.0cc47a6df14e.nvjlcx on host '0cc47a6df14e'
> >> Scheduled to start mgr.0cc47a6df330.aznjao on host '0cc47a6df330'
> >> Scheduled to start mgr.0cc47aad8ce8.ifiydp on host '0cc47aad8ce8'
> >> root@0cc47a6df14e:~# ceph orch start mon
> >> Scheduled to start mon.0cc47a6df14e on host '0cc47a6df14e'
> >> Scheduled to start mon.0cc47a6df330 on host '0cc47a6df330'
> >> Scheduled to start mon.0cc47aad8ce8 on host '0cc47aad8ce8'
> >> root@0cc47a6df14e:~# ceph orch start osd.all-flash-over-1tb
> >> Scheduled to start osd.2 on host '0cc47a6df14e'
> >> Scheduled to start osd.5 on host '0cc47a6df14e'
> >> Scheduled to start osd.3 on host '0cc47a6df330'
> >> Scheduled to start osd.0 on host '0cc47a6df330'
> >> Scheduled to start osd.4 on host '0cc47aad8ce8'
> >> Scheduled to start osd.1 on host '0cc47aad8ce8'
> >> root@0cc47a6df14e:~# systemctl list-units | grep ceph
> >>
> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@mgr.0cc47a6df14e.nvjlcx.service
> >> loaded active running Ceph
> >> mgr.0cc47a6df14e.nvjlcx for 03977a23-f00f-4bb0-b9a7-de57f40ba853
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@mon.0cc47a6df14e.service
> >> loaded active running Ceph
> >> mon.0cc47a6df14e for 03977a23-f00f-4bb0-b9a7-de57f40ba853
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@osd.2.service
> >> loaded active running Ceph osd.2
> >> for 03977a23-f00f-4bb0-b9a7-de57f40ba853
> >> ceph-crash.service
> >> loaded active running Ceph
> crash
> >> dump collector
> >> system-ceph\x2d03977a23\x2df00f\x2d4bb0\x2db9a7\x2dde57f40ba853.slice
> >> loaded active active
> >> system-ceph\x2d03977a23\x2df00f\x2d4bb0\x2db9a7\x2dde57f40ba853.slice
> >> ceph-fuse.target
> >> loaded active active ceph
> target
> >> allowing to start/stop all ceph-fuse@.service instances at once
> >> ceph-mds.target
> >> loaded active active ceph
> target
> >> allowing to start/stop all ceph-mds@.service instances at once
> >> ceph-mgr.target
> >> loaded active active ceph
> target
> >> allowing to start/stop all ceph-mgr@.service instances at once
> >> ceph-mon.target
> >> loaded active active ceph
> target
> >> allowing to start/stop all ceph-mon@.service instances at once
> >> ceph-osd.target
> >> loaded active active ceph
> target
> >> allowing to start/stop all ceph-osd@.service instances at once
> >> ceph-radosgw.target
> >> loaded active active ceph
> target
> >> allowing to start/stop all ceph-radosgw@.service instances at once
> >> ceph.target
> >> loaded active active All Ceph
> >> clusters and services
> >> root@0cc47a6df14e:~# systemctl status
> >>
> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@mgr.0cc47a6df14e.nvjlcx.service
> >> ●
> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@mgr.0cc47a6df14e.nvjlcx.service
> >> - Ceph mgr.0cc47a6df14e.nvjlcx for 03977a23-f00f-4bb0-b9a7-de57f40ba853
> >> Loaded: loaded
> >> (/etc/systemd/system/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@
> .service;
> >> enabled; vendor preset: enabled)
> >> Active: active (running) since Sat 2023-09-16 09:18:53 UTC; 51s ago
> >> Process: 4828 ExecStartPre=/bin/rm -f
> >>
> /run/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@mgr.0cc47a6df14e.nvjlcx.service-pid
> >> /run/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@mgr.0cc47a6df1>
> >> Process: 4829 ExecStart=/bin/bash
> >>
> /var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/mgr.0cc47a6df14e.nvjlcx/unit.run
> >> (code=exited, status=0/SUCCESS)
> >> Main PID: 5132 (conmon)
> >> Tasks: 36 (limit: 309227)
> >> Memory: 512.0M
> >> CGroup:
> >>
> /system.slice/system-ceph\x2d03977a23\x2df00f\x2d4bb0\x2db9a7\x2dde57f40ba853.slice/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@mgr.0cc47a6df14e.nvjlcx.service
> >> ├─container
> >> │ ├─5136 /dev/init -- /usr/bin/ceph-mgr -n
> >> mgr.0cc47a6df14e.nvjlcx -f --setuser ceph --setgroup ceph
> >> --default-log-to-file=false --default-log-to-journald=true
> --default-log>
> >> │ └─5139 /usr/bin/ceph-mgr -n mgr.0cc47a6df14e.nvjlcx -f
> >> --setuser ceph --setgroup ceph --default-log-to-file=false
> >> --default-log-to-journald=true --default-log-to-stderr=fa>
> >> └─supervisor
> >> └─5132 /usr/libexec/podman/conmon --api-version 1 -c
> >> 0165b4f78867ad284cc65fbece46013e6547a2f3ecf99cc7ffb8b720f705ee66 -u
> >> 0165b4f78867ad284cc65fbece46013e6547a2f3ecf99cc7ff>
> >>
> >> Sep 16 09:19:04 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> 2023-09-16T09:19:04.333+0000 7f4fcc0a91c0 -1 mgr[py] Module alert>
> >> Sep 16 09:19:04 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> 2023-09-16T09:19:04.501+0000 7f4fcc0a91c0 -1 mgr[py] Module iosta>
> >> Sep 16 09:19:05 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> 2023-09-16T09:19:05.249+0000 7f4fcc0a91c0 -1 mgr[py] Module orche>
> >> Sep 16 09:19:05 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> 2023-09-16T09:19:05.481+0000 7f4fcc0a91c0 -1 mgr[py] Module rbd_s>
> >> Sep 16 09:19:06 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> [16/Sep/2023:09:19:06] ENGINE Bus STARTING
> >> Sep 16 09:19:06 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> CherryPy Checker:
> >> Sep 16 09:19:06 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> The Application mounted at '' has an empty config.
> >> Sep 16 09:19:06 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> Sep 16 09:19:06 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> [16/Sep/2023:09:19:06] ENGINE Serving on http://:::9283
> >> Sep 16 09:19:06 0cc47a6df14e.f00f.gridscale.dev
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-mgr-0cc47a6df14e-nvjlcx[5132]:
> >> [16/Sep/2023:09:19:06] ENGINE Bus STARTED
> >>
> >> This seems to be the cephadm log:
> >>
> >> cephadm ['adopt', '--style', 'legacy', '--name', 'osd.3']
> >> 2023-09-15 11:32:44,290 7fef7b041740 INFO Pulling container image
> >> quay.io/ceph/ceph:v17...
> >> 2023-09-15 11:32:47,128 7fef7b041740 INFO Found online OSD at
> >> //var/lib/ceph/osd/ceph-3/fsid
> >> 2023-09-15 11:32:47,129 7fef7b041740 INFO objectstore_type is bluestore
> >> 2023-09-15 11:32:47,150 7fef7b041740 INFO Stopping old systemd unit
> >> ceph-osd@3...
> >> 2023-09-15 11:32:48,560 7fef7b041740 INFO Disabling old systemd unit
> >> ceph-osd@3...
> >> 2023-09-15 11:32:49,157 7fef7b041740 INFO Moving data...
> >> 2023-09-15 11:32:49,158 7fef7b041740 DEBUG move file
> >> '//var/lib/ceph/osd/ceph-3/require_osd_release' ->
> >>
> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/require_osd_release'
> >> 2023-09-15 11:32:49,158 7fef7b041740 DEBUG chown 167:167
> >>
> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/require_osd_release`
> >> 2023-09-15 11:32:49,158 7fef7b041740 DEBUG symlink
> >> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/block' ->
> >>
> '/dev/ceph-66d3bb27-cd5c-4897-aa76-684bc46d1c8b/osd-block-4bfc2101-e9b2-468d-8f54-a05f080ebdfe'
> >> 2023-09-15 11:32:49,158 7fef7b041740 DEBUG move file
> >> '//var/lib/ceph/osd/ceph-3/ready' ->
> >> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/ready'
> >> 2023-09-15 11:32:49,159 7fef7b041740 DEBUG chown 167:167
> >> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/ready`
> >> 2023-09-15 11:32:49,159 7fef7b041740 DEBUG move file
> >> '//var/lib/ceph/osd/ceph-3/type' ->
> >> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/type'
> >> 2023-09-15 11:32:49,159 7fef7b041740 DEBUG chown 167:167
> >> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/type`
> >> 2023-09-15 11:32:49,159 7fef7b041740 DEBUG move file
> >> '//var/lib/ceph/osd/ceph-3/fsid' ->
> >> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/fsid'
> >> 2023-09-15 11:32:49,159 7fef7b041740 DEBUG chown 167:167
> >> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/fsid`
> >> 2023-09-15 11:32:49,160 7fef7b041740 DEBUG move file
> >> '//var/lib/ceph/osd/ceph-3/ceph_fsid' ->
> >> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/ceph_fsid'
> >> 2023-09-15 11:32:49,160 7fef7b041740 DEBUG chown 167:167
> >> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/ceph_fsid`
> >> 2023-09-15 11:32:49,160 7fef7b041740 DEBUG move file
> >> '//var/lib/ceph/osd/ceph-3/keyring' ->
> >> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/keyring'
> >> 2023-09-15 11:32:49,160 7fef7b041740 DEBUG chown 167:167
> >> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/keyring`
> >> 2023-09-15 11:32:49,160 7fef7b041740 DEBUG move file
> >> '//var/lib/ceph/osd/ceph-3/whoami' ->
> >> '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/whoami'
> >> 2023-09-15 11:32:49,161 7fef7b041740 DEBUG chown 167:167
> >> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/whoami`
> >> 2023-09-15 11:32:49,161 7fef7b041740 DEBUG Remove dir
> >> `//var/lib/ceph/osd/ceph-3`
> >> 2023-09-15 11:32:49,166 7fef7b041740 INFO Chowning content...
> >> 2023-09-15 11:32:49,171 7fef7b041740 DEBUG chown: stdout changed
> ownership
> >> of '/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/block' from
> >> root:root to 167:167
> >> 2023-09-15 11:32:49,172 7fef7b041740 INFO Chowning
> >> /var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/block...
> >> 2023-09-15 11:32:49,172 7fef7b041740 INFO Disabling host unit
> ceph-volume@
> >> lvm unit...
> >> 2023-09-15 11:32:49,649 7fef7b041740 DEBUG systemctl: stderr Removed
> >>
> /etc/systemd/system/multi-user.target.wants/ceph-volume@lvm-3-4bfc2101-e9b2-468d-8f54-a05f080ebdfe.service.
> >> 2023-09-15 11:32:49,650 7fef7b041740 DEBUG copy file
> `//etc/ceph/ceph.conf`
> >> -> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/config`
> >> 2023-09-15 11:32:49,650 7fef7b041740 DEBUG chown 167:167
> >> `/var/lib/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/osd.3/config`
> >> 2023-09-15 11:32:49,650 7fef7b041740 INFO Moving logs...
> >> 2023-09-15 11:32:49,651 7fef7b041740 DEBUG move file
> >> '//var/log/ceph/ceph-osd.3.log' ->
> >> '/var/log/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/ceph-osd.3.log'
> >> 2023-09-15 11:32:49,651 7fef7b041740 DEBUG chown 167:167
> >> `/var/log/ceph/03977a23-f00f-4bb0-b9a7-de57f40ba853/ceph-osd.3.log`
> >> 2023-09-15 11:32:49,651 7fef7b041740 INFO Creating new units...
> >> 2023-09-15 11:32:50,803 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-console-messages.conf ...
> >> 2023-09-15 11:32:50,803 7fef7b041740 DEBUG sysctl: stdout kernel.printk
> = 4
> >> 4 1 7
> >> 2023-09-15 11:32:50,803 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-ipv6-privacy.conf ...
> >> 2023-09-15 11:32:50,803 7fef7b041740 DEBUG sysctl: stdout
> >> net.ipv6.conf.all.use_tempaddr = 2
> >> 2023-09-15 11:32:50,803 7fef7b041740 DEBUG sysctl: stdout
> >> net.ipv6.conf.default.use_tempaddr = 2
> >> 2023-09-15 11:32:50,803 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-kernel-hardening.conf ...
> >> 2023-09-15 11:32:50,803 7fef7b041740 DEBUG sysctl: stdout
> >> kernel.kptr_restrict = 1
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-link-restrictions.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_hardlinks = 1
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_symlinks = 1
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-magic-sysrq.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout kernel.sysrq
> = 176
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-network-security.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> net.ipv4.conf.default.rp_filter = 2
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> net.ipv4.conf.all.rp_filter = 2
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-ptrace.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> kernel.yama.ptrace_scope = 1
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/10-zeropage.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> vm.mmap_min_addr
> >> = 65536
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/30-ceph-osd.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout fs.aio-max-nr
> =
> >> 1048576
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> kernel.pid_max =
> >> 4194304
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /usr/lib/sysctl.d/50-coredump.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t
> %c
> >> %h
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /usr/lib/sysctl.d/50-default.conf ...
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> net.ipv4.conf.default.promote_secondaries = 1
> >> 2023-09-15 11:32:50,804 7fef7b041740 DEBUG sysctl: stdout
> >> net.ipv4.ping_group_range = 0 2147483647
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> >> net.core.default_qdisc = fq_codel
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_regular = 1
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_fifos = 1
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /usr/lib/sysctl.d/50-pid-max.conf ...
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> kernel.pid_max =
> >> 4194304
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/90-ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853-osd.conf ...
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout fs.aio-max-nr
> =
> >> 1048576
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> kernel.pid_max =
> >> 4194304
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.d/99-sysctl.conf ...
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /usr/lib/sysctl.d/protect-links.conf ...
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_fifos = 1
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_hardlinks = 1
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_regular = 2
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout
> >> fs.protected_symlinks = 1
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stdout * Applying
> >> /etc/sysctl.conf ...
> >> 2023-09-15 11:32:50,805 7fef7b041740 DEBUG sysctl: stderr sysctl:
> setting
> >> key "net.ipv4.conf.all.promote_secondaries": Invalid argument
> >> 2023-09-15 11:32:51,469 7fef7b041740 DEBUG Non-zero exit code 1 from
> >> systemctl reset-failed ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@osd.3
> >> 2023-09-15 11:32:51,469 7fef7b041740 DEBUG systemctl: stderr Failed to
> >> reset failed state of unit
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@osd.3.service: Unit
> >> ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@osd.3.service not loaded.
> >> 2023-09-15 11:32:51,954 7fef7b041740 DEBUG systemctl: stderr Created
> >> symlink
> >>
> /etc/systemd/system/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853.target.wants/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@osd.3.service
> >> → /etc/systemd/system/ceph-03977a23-f00f-4bb0-b9a7-de57f40ba853@
> .service.
> >> 2023-09-15 11:32:54,331 7fef7b041740 DEBUG firewalld does not appear to
> be
> >> present
> >>
> >>> Am Sa., 16. Sept. 2023 um 10:25 Uhr schrieb Eugen Block <eblock@xxxxxx
> >:
> >>>
> >>> That sounds a bit strange to me, because all clusters we adopted so
> >>> far successfully converted the previous systemd-units into systemd
> >>> units targeting the pods. This process also should have been logged
> >>> (stdout, probably in the cephadm.log as well), resulting in "enabled"
> >>> systemd units. Can you paste the output of 'systemctl status
> >>> ceph-<FSID>@mon.<MON>'? If you have it, please also share the logs
> >>> from the adoption process.
> >>> What I did notice in a test cluster a while ago was that I had to
> >>> reboot a node where I had to "play around" a bit with removed and
> >>> redeployed osd containers. At some point they didn't react to
> >>> systemctl commands anymore, but a reboot fixed that. But I haven't
> >>> seen that in a production cluster yet, so some more details would be
> >>> useful.
> >>>
> >>> Zitat von Boris Behrens <bb@xxxxxxxxx>:
> >>>
> >>> > Hi,
> >>> > is there a way to have the pods start again after reboot?
> >>> > Currently I need to start them by hand via ceph orch start
> >>> mon/mgr/osd/...
> >>> >
> >>> > I imagine this will lead to a lot of headache when the ceph cluster
> gets
> >>> a
> >>> > powercycle and the mon pods will not start automatically.
> >>> >
> >>> > I've spun up a test cluster and there the pods start very fast. On
> the
> >>> > legacy test cluster, which got adopted to cephadm, it does not.
> >>> >
> >>> > Cheers
> >>> > Boris
> >>> > _______________________________________________
> >>> > ceph-users mailing list -- ceph-users@xxxxxxx
> >>> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
> >>>
> >>>
> >>> _______________________________________________
> >>> ceph-users mailing list -- ceph-users@xxxxxxx
> >>> To unsubscribe send an email to ceph-users-leave@xxxxxxx
> >>>
> >>
> >>
> >> --
> >> Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
> >> groüen Saal.
> >
> >
> >
>
--
Die Selbsthilfegruppe "UTF-8-Probleme" trifft sich diesmal abweichend im
groüen Saal.
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
