Hi
I am running old PHP under Apache httpd-2.4.
During a typical day:
Server load: 0.03 0.03 0.05
Total accesses: 16028 - Total Traffic: 1.4 GB
CPU Usage: u20.92 s1.24 cu.01 cs.23 - .00163% CPU load
.0116 requests/sec - 1104 B/second - 92.7 kB/request
2 requests currently being processed, 8 idle workers
Though, ever few weeks, we see sudden increase in workers who never seem to retire:
[Fri Nov 10 02:43:20.019924 2017] [mpm_prefork:error] [pid 13584] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
user@server[/var/www]$ ps aux | grep [h]ttpd | wc -l
257
It's my belief that this occurs due to malicious activity involving our old PHP sites, given this version has multiple known denial of service vulnerabilities, however the only thing I see in logs, during the time when workers were spawned, are light spider and bot activity.
We are running mod_security, mod_evasive, and mod_reqtimeout.
apachectl -t -D DUMP_MODULES | grep -e timeout -e security -e evasive
reqtimeout_module (shared)
security2_module (shared)
evasive20_module (shared)
httpd.conf:
MaxKeepAliveRequests 50
KeepAlive On
Timeout 30
KeepAliveTimeout 10
<IfModule reqtimeout_module>
RequestReadTimeout header=20-40,MinRate=500 body=20,MinRate=500
</IfModule>
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 2
MaxSpareServers 10
MaxRequestWorkers 128
MaxRequestsPerChild 50
MaxRequestWorkers 100
</IfModule>
modsecurity.conf:
SecRuleEngine on
mod_evasive.conf:
DOSPageCount 50
DOSSiteCount 100
DOSPageInterval 1
DOSSiteInterval 1
php.ini:
max_execution_time = 10
max_input_time = 10
memory_limit = 32M
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
log_errors = On
I set MaxRequestWorkers to 100 though it seems that threshold was passed meanwhile the server's no longer serving data, as the failover's now active, but these httpd workers refuse to die!
If my VirtualHosts were under DoS, in a manner that exploits PHP, then would I even be able to detect them in the logs?
Based upon my limited experience, I should be protected against both "slow" and "fast" DoS though of course not DDoS. Greatly appreciate the insight and assistance. We plan on replacing our old PHP sites but until then I want to do what I can to ensure this stops happening other than bringing up the failover.
Thanks,
Douglas Duckworth, MSc, LFCS
HPC System Administrator
Scientific Computing Unit
Physiology and Biophysics
