Re: Apache processes are killed by a watchdog

Tom Evans <tevans.uk@xxxxxxxxxxxxxx> · Mon, 29 Sep 2008 15:50:26 +0100

On Mon, 2008-09-29 at 07:00 -0700, faust wrote:
> I have apache-2.2.9_5 under openvds (jail) in the FreeBSD system
> (6.3-STABLE). I configured Apache HTTP Server to create few virtual hosts.
> Starting Apache server is successful and it works ok. I have FreeBSD's
> watchdog script which performs some actions every 10 minutes and this
> watchdog just kills Apache processes. I starter Apache at 16:21:30 and at
> 16:30:02 it was killed by a watchdog. Here are the log messages:
> 
> /var/log/debug.log:
> Sep 29 16:30:00 heaven watchdog[23377]: Watchdog started
> Sep 29 16:30:00 heaven watchdog[23377]: Watchdog create pid file
> Sep 29 16:30:00 heaven watchdog[23377]: Before init
> Sep 29 16:30:00 heaven watchdog[23377]: After init
> Sep 29 16:30:00 heaven watchdog[23377]: Load WatchDog config
> Sep 29 16:30:00 heaven watchdog[23377]: Config loaded
> Sep 29 16:30:00 heaven watchdog[23377]: Begin check
> Sep 29 16:30:00 heaven watchdog[23377]: Wait for finish
> Sep 29 16:30:00 heaven watchdog[23377]: Check WWW
> Sep 29 16:30:01 heaven watchdog[23377]: thread MySQL finished with status OK
> Sep 29 16:30:01 heaven watchdog[23377]: thread HTTP finished with status ERR
> Sep 29 16:30:01 heaven watchdog[23377]: Begin restart
> Sep 29 16:30:01 heaven watchdog[23377]: stop command
> '/usr/local/sbin/apachectl stop'
> Sep 29 16:30:12 heaven watchdog[23377]: process name 'httpd'
> Sep 29 16:30:12 heaven watchdog[23377]: start command
> '/usr/local/sbin/apachectl startssl'
> Sep 29 16:30:22 heaven watchdog[23377]: Wait for finish
> Sep 29 16:30:22 heaven watchdog[23377]: Check WWW
> Sep 29 16:30:22 heaven watchdog[23377]: thread HTTP finished with status ERR
> Sep 29 16:30:22 heaven watchdog[23377]: Done
> 
> /var/log/httpd-error.log:
> [Mon Sep 29 16:21:30 2008] [warn] Init: Session Cache is not configured
> [hint: SSLSessionCache]
> [Mon Sep 29 16:21:32 2008] [notice] Digest: generating secret for digest
> authentication ...
> [Mon Sep 29 16:21:32 2008] [notice] Digest: done
> [Mon Sep 29 16:21:33 2008] [notice] Apache/2.2.9 (FreeBSD) mod_ssl/2.2.9
> OpenSSL/0.9.7e-p1 DAV/2 PHP/5.2.6 configured -- resuming normal operations
> [Mon Sep 29 16:30:02 2008] [notice] caught SIGTERM, shutting down
> 
> As far as I understand, the watchdog sends SIGTERM signal to a httpd daemon.
> Why it could happen? Please, advise
> 
> Thanks

Judging from the log, it looks at the www service to see if it is still
up, and if it isn't, or if it errors, it restarts apache.

This watchdog doesn't seem to be watchdog(4), watchdog(8), watchdog(9)
or watchdogd(8), so I'm guessing it is some 3rd party software that has
been installed on your server to manage apache and restart it when it is
in trouble. Without knowing what software it is, it is hard to help.

Cheers

Tom