Re: systemd boot up hung after sysroot.mount in initrd phase

[RAJESH DASARI <raajeshdasari@xxxxxxxxx>]

On Wed, May 10, 2023 at 4:43 AM Benjamin Godfrey
<mr.benjamingodfrey@xxxxxxxxx> wrote:
>
> I'm trying to be helpful, but it could still be an issue with the systemd-fsck-root.service
> The initrd phase is typically very fast because it doesn't need to load many files.
> You might try booting from a live USB to boot.  You can use the Gparted tool to check the root file system for errors.
we are booting a VM in openstack.
>
> I'm not clear about what you are saying, when you report "issue has disappeared"  Try double checking the dracut logs.
>
> An error like "Failed to mount /dev/sda1" indicates there's a problem with the root files system.
There are no errors in the dracut logs.
> "Executing command '/sbin/fsck -C -f /dev/sda1'" would mean the fsck command is executing.
> Something like "Command '/sbin/fsck -C -f /dev/sda1' took 10 seconds to execute" might help you find the issue.
I tried below actions from my end:
I have added fsck check in systemd-fsck-root.service and redirected
the output to console with StandardOutput= in the service file. There
are no erros in the file system. But still boot up stuck after
sysroot.mount.
I have added TimeoutSec=30 in the systemd-fsck-root.service and
sysroot.mount , but bootup is  still hung after sysroot.mount.
I have added one second sleep in systemd-fsck-root.service
"ExecStartPre=/usr/bin/sleep 1\n"  , with a sleep of 1 second bootup
is not hung. To me it looks like a race condition where delay is
fixing the bootup hang problem.
If i add debug logs in systemd or enable dracut logs also issue is not
seen , i.e boot up is not hung.
>
> Benjamin Godfrey
>
> On Tue, May 9, 2023 at 5:00 AM <systemd-devel-request@xxxxxxxxxxxxxxxxxxxxx> wrote:
>>
>> Send systemd-devel mailing list submissions to
>>         systemd-devel@xxxxxxxxxxxxxxxxxxxxx
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         https://lists.freedesktop.org/mailman/listinfo/systemd-devel
>> or, via email, send a message with subject or body 'help' to
>>         systemd-devel-request@xxxxxxxxxxxxxxxxxxxxx
>>
>> You can reach the person managing the list at
>>         systemd-devel-owner@xxxxxxxxxxxxxxxxxxxxx
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of systemd-devel digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Re:  Reg: systemd boot up hung after sysroot.mount in initrd
>>       phase (RAJESH DASARI)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Tue, 9 May 2023 10:24:56 +0300
>> From: RAJESH DASARI <raajeshdasari@xxxxxxxxx>
>> To: systemd-devel@xxxxxxxxxxxxxxxxxxxxx
>> Subject: Re:  Reg: systemd boot up hung after
>>         sysroot.mount in initrd phase
>> Message-ID:
>>         <CAPXMrf-uH14BXEeJVmOz=WapCxe7S2T6-pC5LFL7PLcj+0RV0g@xxxxxxxxxxxxxx>
>> Content-Type: text/plain; charset="UTF-8"
>>
>> Hi ,
>>
>> Could someone please respond ?
>>
>> Thanks,
>> Rajesh Dasari.
>>
>> On Sun, May 7, 2023 at 8:28?PM RAJESH DASARI <raajeshdasari@xxxxxxxxx> wrote:
>> >
>> > Hi ,
>> >
>> > We are issuing reboot with this command  systemctl  --no-block reboot
>> > and  boot up is stuck at initrd phase , we are using dracut and
>> > systemd in initramfs phase. Boot up is stuck after  sysroot.mount is
>> > executed(Issue is always reproducible), I have enabled debug logs by
>> > passing systemd.log_level=debug systemd.log_target=console to kernel
>> > command line , issue is disappeared and I also tried to enable the
>> > dracut debug logs rd.debug=1, issue is disappeared. Then I have added
>> > sleep 1 in the systemd-fsck-root.service , then the issue is not seen.
>> > So it looks like a timing issue.
>> >
>> >
>> > Starting [0;1;39msystemd-fsck-root.service[0m - File System Check on
>> > /dev/disk/by-label/rootfs...
>> > Finished [0;1;39msystemd-fsck-root.service[0m - File System Check on
>> > /dev/disk/by-label/rootfs.
>> > Mounting [0;1;39msysroot.mount[0m - /sysroot...
>> >  EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
>> >  Mounted [0;1;39msysroot.mount[0m - /sysroot. -> boot up stuck after this.
>> >
>> >
>> > We are using systemd version-251.13 and with systemd  version-249.14
>> > we didn't notice this. Is there any new issue with this version or Is
>> > there any known issue if not Could you please let me know how to debug
>> > this further.
>> >
>> > Regards,
>> > Rajesh.
>>
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> systemd-devel mailing list
>> systemd-devel@xxxxxxxxxxxxxxxxxxxxx
>> https://lists.freedesktop.org/mailman/listinfo/systemd-devel
>>
>>
>> ------------------------------
>>
>> End of systemd-devel Digest, Vol 157, Issue 8
>> *********************************************