I was wrong, it just happened again.
As predicted, the OOM killer did it's job.
The problem is actually that some cracker has managed to upload httpds.c into /tmp/.bd/ (via apache, still investigating how).
He then managed to compile and run it.
I took a look at the source code, and it seems to be a DDOS util. Why it killed our server instead of the target of the DDOS I do not know, but I guess it might be due to our firewall rejecting all the attempts to connect.
I guess I'll fix this problem the same way I did at another server. I'll make a partition for /tmp and mount it with noexec, or are there better ways to do that?
On public servers, I now put /tmp /var/tmp
as seperate partitions with noexec,nosuid on them. We may also put nodev on them but I am not sure if that broke things or not. Each are limited to 100->500 megs in size. We were looking at a script that did an hourly cleanup of files that were in it so that nothing stayed too long, but I think we dropped that in case we needed to keep an audit trail.
We tried mounting other parts of the system as ro unless an update was done but I think that caused a couple of problems with the version of RH we had and some python items that wanted to make new .pyc.
I am hoping SELinux for dummies gets published or that the NSA does a 'SELinux Bootcamp' although I hope without drill seargeants. I am not sure I can still handle an Army or Marine Drill Sgt yelling at me to keep my ACLs in line.
'Mister is that an AVC message I see there.. GIVE ME TWENTY'.
Although my weight would probably go down.
-- Stephen John Smoogen smoogen@xxxxxxxx Los Alamos National Lab CCN-5 Sched 5/40 PH: 4-0645 Ta-03 SM-1498 MailStop B255 DP 10S Los Alamos, NM 87545
