Re: [External] Re: How to set the timeout of emergency mode

Hello Itxaka,

I have tried the scenario,

Update the emergency service with JobRunningTimeoutSec=300 and JobTimeoutAction=reboot-force; (/usr/lib/systemd/system/emergency.service)
Reload service: systemctl daemon-reload
Start the service manually: systemctl start emergency; (Is this manual action OK?)
But from the result, the VM still in emergency mode after 300 seconds;

[root@qrms6-host02 ~]# virsh console 47

Connected to domain 'qrms6-sig-fw-b'

Escape character is ^] (Ctrl + ])

Give root password for maintenance

(or press Control-D to continue):

Any other suggestions.

Thanks,

Mark

From: Mark Zhang (NSB)
Sent: 2024年6月12日 16:42
To: Itxaka Serrano Garcia <itxaka.garcia@xxxxxxxxxxxxxxxx>
Cc: systemd-devel@xxxxxxxxxxxxxxxxxxxxx
Subject: RE: [External] Re: [systemd-devel] How to set the timeout of emergency mode

Hello Itxaka,

Thanks for the valuable suggestions.

We will try that.

Thanks,

Mark

From: Itxaka Serrano Garcia <itxaka.garcia@xxxxxxxxxxxxxxxx>
Sent: 2024年6月12日 15:38
To: Mark Zhang (NSB) <mark.zhang@xxxxxxxxxxxxxxx>
Cc: systemd-devel@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: [External] Re: [systemd-devel] How to set the timeout of emergency mode

CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See http://nok.it/nsb for additional information.

In that case, why not use the timeout stuff from the services?

You could probably override the service to set a timeout for its running with

[Unit]
JobRunningTimeoutSec=300

JobTimeoutAction=reboot-force

So after starting you get up to 5 minutes of it running and then it should reboot. Probably the JobTimeoutAction is not needed as I'm sure it will reboot if the emergency.service fails, but better safe than sorry :)

That gives you the best of both worlds. You get some time to debug and if not it will auto reboot after a bit and maybe fix itself :D

On Wed, Jun 12, 2024 at 2:21 AM Mark Zhang (NSB) <mark.zhang@xxxxxxxxxxxxxxx> wrote:

Hello Itxaka,

Thanks for the suggestion.

If we override with reboot, that may impact the ability of debug in case encountered real issue.

Thanks,

Mark

From: Itxaka Serrano Garcia <itxaka.garcia@xxxxxxxxxxxxxxxx>
Sent: 2024年6月11日 16:08
To: Mark Zhang (NSB) <mark.zhang@xxxxxxxxxxxxxxx>
Cc: systemd-devel@xxxxxxxxxxxxxxxxxxxxx
Subject: [External] Re: [systemd-devel] How to set the timeout of emergency mode

CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See http://nok.it/nsb for additional information.

Maybe you could override the emergency.service to reboot automatically? Either on the ExecStartPost or even earlier if you dont want to even show the logging, on ExecStartPre?

I think that could work.

On Tue, Jun 11, 2024 at 5:19 AM Mark Zhang (NSB) <mark.zhang@xxxxxxxxxxxxxxx> wrote:

Hello Experts,

Consult one question about how to set the timeout of emergency mode?

Scenario:

We have encountered one problem of Openstack host, and the host was up at the end, and all things are good from host level after that.

But the VM stuck in the emergency mode forever. If we input ctrl+D, the VM could reboot normally again.

So we would like to consult, if any method could avoid to stuck in the emergency mode forever? Be better, there is some timeout setting for emergency mode. Then the VM could auto-recover by itself for such scenario.

Thanks,

Mark