Hi All...
Just upgraded from infernalis 9.2.0 to jewel 10.2.2 in centos7.
I do have a question regarding ceph-mon.target and
ceph-mds.target systemd dependencies.
Before the upgrade, I had the following situation in a mon host,
data host with 8 osds and mds host:
# systemctl list-dependencies ceph.target
ceph.target
● └─ceph-mon@rccephmon1.service
# systemctl list-dependencies ceph.target
ceph.target
● ├─ceph-osd@0.service
● ├─ceph-osd@1.service
● ├─ceph-osd@2.service
● ├─ceph-osd@3.service
● ├─ceph-osd@4.service
● ├─ceph-osd@5.service
● ├─ceph-osd@6.service
● └─ceph-osd@7.service
# systemctl list-dependencies ceph.target
ceph.target
● └─ceph-mds@rccephmds2.service
After upgrade (and reboot), I have:
# systemctl list-dependencies ceph.target
ceph.target
● ├─ceph-mon@rccephmon3.service
● ├─ceph-mds.target
● ├─ceph-mon.target
● └─ceph-osd.target
# systemctl list-dependencies ceph.target
ceph.target
● ├─ceph-osd@0.service
● ├─ceph-osd@1.service
● ├─ceph-osd@2.service
● ├─ceph-osd@3.service
● ├─ceph-osd@4.service
● ├─ceph-osd@5.service
● ├─ceph-osd@6.service
● ├─ceph-osd@7.service
● ├─ceph-mds.target
● ├─ceph-mon.target
● └─ceph-osd.target
● ├─ceph-osd@0.service
● ├─ceph-osd@1.service
● ├─ceph-osd@2.service
● ├─ceph-osd@3.service
● ├─ceph-osd@4.service
● ├─ceph-osd@5.service
● ├─ceph-osd@6.service
● └─ceph-osd@7.service
# systemctl list-dependencies ceph.target
ceph.target
● ├─ceph-mds@rccephmds2.service
● ├─ceph-mds.target
● ├─ceph-mon.target
● └─ceph-osd.target
I understand that ceph-{mon,osd,mds.target} are available in all
hosts because the ceph metapackages pulls all
ceph-{mon,osd,mds}.rpm even if we do not want to install a given
service in a host.
# rpm -qR ceph
ceph-osd = 1:10.2.2-0.el7
ceph-mds = 1:10.2.2-0.el7
ceph-mon = 1:10.2.2-0.el7
binutils
systemd
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(PayloadIsXz) <= 5.2-1
However, what I am not understanding is why all ceph-osd@X.service
were correctly set as dependencies of ceph-osd.target but
ceph-mds@rccephmds2.service was not set as dependency of
ceph-mds.target (and similarly, why ceph-mon@rccephmon3.service was
not set as a dependency of ceph-mon.target).
The configuration of the unit seems fine but I do not understand why
it is not applied (after systemctl daemon-reload or even after
rebooting):
# systemctl cat ceph-mon@rccephmon3.service
# /usr/lib/systemd/system/ceph-mon@.service
[Unit]
Description=Ceph cluster monitor daemon
# According to:
# http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget
# these can be removed once ceph-mon will dynamically change
network
# configuration.
After=network-online.target local-fs.target time-sync.target
ceph-create-keys@%i.service
Wants=network-online.target local-fs.target time-sync.target
ceph-create-keys@%i.service
PartOf=ceph-mon.target
[Service]
LimitNOFILE=1048576
LimitNPROC=1048576
EnvironmentFile=-/etc/sysconfig/ceph
Environment=CLUSTER=ceph
ExecStart=/usr/bin/ceph-mon -f --cluster ${CLUSTER} --id %i
--setuser ceph --setgroup ceph
ExecReload=/bin/kill -HUP $MAINPID
PrivateDevices=yes
ProtectHome=true
ProtectSystem=full
PrivateTmp=true
TasksMax=infinity
Restart=on-failure
StartLimitInterval=30min
StartLimitBurst=3
[Install]
WantedBy=ceph-mon.target
Am i the only one seeing the issue? Is it really an issue?
Cheers
G.
--
Goncalo Borges
Research Computing
ARC Centre of Excellence for Particle Physics at the Terascale
School of Physics A28 | University of Sydney, NSW 2006
T: +61 2 93511937
