"I haven't noticed this behavior myself, and attempted a repro here on
v253 (Arch x86_64) just now."
-> Sorry for being unclear, I am talking about an embedded system with very significant customization (yocto based) and fully expect that the customization introduced the issue, so I didn't expect you to be able to reproduce. I am rather debugging this and asking for input on how to debug this kind of issue.
"What do you mean by "disabled"?"
-> I mean specifically that targets "basic.target", "local-fs.target" and "multi-user.target" are in the state "LOAD=loaded, ACTIVE=inactive, SUB=dead" after I call systemctl daemon-reload, whereas they are in the state "LOAD=loaded,ACTIVE=active,SUB-active" after I boot the system. I am trying to understand what causes this transition, but I don't see it in the logs at the moment.
I am using systemd version 250.5 (plus the yocto patches):
# systemctl --version
systemd 250 (250.5+)
+PAM +AUDIT +SELINUX -APPARMOR +IMA -SMACK +SECCOMP -GCRYPT -GNUTLS -OPENSSL +ACL +BLKID -CURL +ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -BZIP2 -LZ4 +XZ -ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=hybrid
With the default info log-level, I only see those logs after calling "systemctl daemon-reload":
~ # Reloading.
systemd-logind.service: Deactivated successfully.
systemd-resolved.service: Deactivated successfully.
systemd-networkd.service: Deactivated successfully.
... (other services getting "deactivated")
~ #
I also tried reading the debug log-level logs, as well as the output of dbus-monitor --system but I still don't understand where the error is. Taking local-fs.target as an exemple, those are the logs from dbus-monitor --system after running systemctl daemon-reload, but those logs actually don't show an error:
```
signal time=1702459928.818819 sender=:1.0 -> destination=(null destination) serial=509 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=Reloading
boolean true
...
signal time=1702459928.978736 sender=:1.0 -> destination=(null destination) serial=1288 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=UnitRemoved
string "local-fs.target"
object path "/org/freedesktop/systemd1/unit/local_2dfs_2etarget"
...
signal time=1702459929.769768 sender=:1.0 -> destination=(null destination) serial=3395 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=UnitNew
string "local-fs.target"
object path "/org/freedesktop/systemd1/unit/local_2dfs_2etarget"
...
signal time=1702459929.781640 sender=:1.0 -> destination=(null destination) serial=3629 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=JobNew
uint32 829
object path "/org/freedesktop/systemd1/job/829"
string "local-fs.target"
signal time=1702459929.781678 sender=:1.0 -> destination=(null destination) serial=3630 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=Reloading
boolean false
```
systemctl status local-fs.target also doesn't show why the unit is "inactived (dead)". How would you debug this kind of issue?
Thanks,
Etienne
v253 (Arch x86_64) just now."
-> Sorry for being unclear, I am talking about an embedded system with very significant customization (yocto based) and fully expect that the customization introduced the issue, so I didn't expect you to be able to reproduce. I am rather debugging this and asking for input on how to debug this kind of issue.
"What do you mean by "disabled"?"
-> I mean specifically that targets "basic.target", "local-fs.target" and "multi-user.target" are in the state "LOAD=loaded, ACTIVE=inactive, SUB=dead" after I call systemctl daemon-reload, whereas they are in the state "LOAD=loaded,ACTIVE=active,SUB-active" after I boot the system. I am trying to understand what causes this transition, but I don't see it in the logs at the moment.
I am using systemd version 250.5 (plus the yocto patches):
# systemctl --version
systemd 250 (250.5+)
+PAM +AUDIT +SELINUX -APPARMOR +IMA -SMACK +SECCOMP -GCRYPT -GNUTLS -OPENSSL +ACL +BLKID -CURL +ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -BZIP2 -LZ4 +XZ -ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=hybrid
With the default info log-level, I only see those logs after calling "systemctl daemon-reload":
~ # Reloading.
systemd-logind.service: Deactivated successfully.
systemd-resolved.service: Deactivated successfully.
systemd-networkd.service: Deactivated successfully.
... (other services getting "deactivated")
~ #
I also tried reading the debug log-level logs, as well as the output of dbus-monitor --system but I still don't understand where the error is. Taking local-fs.target as an exemple, those are the logs from dbus-monitor --system after running systemctl daemon-reload, but those logs actually don't show an error:
```
signal time=1702459928.818819 sender=:1.0 -> destination=(null destination) serial=509 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=Reloading
boolean true
...
signal time=1702459928.978736 sender=:1.0 -> destination=(null destination) serial=1288 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=UnitRemoved
string "local-fs.target"
object path "/org/freedesktop/systemd1/unit/local_2dfs_2etarget"
...
signal time=1702459929.769768 sender=:1.0 -> destination=(null destination) serial=3395 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=UnitNew
string "local-fs.target"
object path "/org/freedesktop/systemd1/unit/local_2dfs_2etarget"
...
signal time=1702459929.781640 sender=:1.0 -> destination=(null destination) serial=3629 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=JobNew
uint32 829
object path "/org/freedesktop/systemd1/job/829"
string "local-fs.target"
signal time=1702459929.781678 sender=:1.0 -> destination=(null destination) serial=3630 path=/org/freedesktop/systemd1; interface=org.freedesktop.systemd1.Manager; member=Reloading
boolean false
```
systemctl status local-fs.target also doesn't show why the unit is "inactived (dead)". How would you debug this kind of issue?
Thanks,
Etienne
On Tue, Dec 12, 2023 at 10:07 PM Lennart Poettering <lennart@xxxxxxxxxxxxxx> wrote:
On Di, 12.12.23 19:06, Etienne Cordonnier (ecordonnier@xxxxxxxx) wrote:
> Hello,
> I am debugging some embedded system running systemd. The behavior I am
> observing is that many systemd targets such as multi-user.target are
> disabled after I run systemctl daemon-reload (as shown by systemctl
> list-units --type target --all). This causes many systemd units to be
> disabled, and forces me to reboot the system.
What do you mean by "disabled"?
in systemd targets can be active and inactive, and that's what
"systemctl list-unit" shows. They can also be enabled/disabled, but
that's what "systemctl list-unit-files" shows. But targets such as
multi-user.target cannot be enabled nor disabled, they are considered
"static", i.e. always enabled if you so will. Which "systemctl
list-unit-file" should actually show.
Hence, I don#t really grok what you are trying to say here...
> Is there a way to debug this systemd target transition? I already
> enabled systemctl
> log-level debug, but I still don't understand why the systemd target is
> changing when I call systemctl daemon-reload on this particular system.
Please state OS, systemd version and provide relevant logs. Otherwise
this is not actionable.
Lennart
--
Lennart Poettering, Berlin
