Had a strange, and as yet unexplained, 'event' (I wasn't in front
of the
machine when things went weird) that took place while a system was
left
running a large rsync over ssh. On returning, a majority of the
directories under /var vanished, and a number of services refused to
start after a reboot, including auditd, nfsd, system message bus,
hpiod,
hpssd, mysql, syslogd, httpd, sm-client, and setroubleshootd.
In the cases of most of these services, there seemed to be problems
either with orphaned /var/run/*.pid files, or with orphaned
/var/lock/subsys/* lock files. Also, many services were reporting
'subsys locked'. Deleting orphaned files, followed by relabeling the
filesystem selinux permissions did the trick, with relabeling
being the
key to getting things going again. Debugging was made more
challenging
by the fact that I had no logs to refer to.
Now, almost all seems well, but I can't get setroubleshootd to start
unless I select 'setroubleshootd_disable_trans'. Without this
checked,
setroubleshootd seems to start, but then fails:
[root@file1 subsys]# rm setroubleshootd
rm: remove regular empty file `setroubleshootd'? y
[root@file1 subsys]# service setroubleshoot status
setroubleshootd is stopped
[root@file1 subsys]# service setroubleshoot start
Starting setroubleshootd: [ OK ]
[root@file1 subsys]# service setroubleshoot status
setroubleshootd dead but subsys locked
Attempting to run setroubleshoot generates the error:
'attempt to open server connection failed: (2, 'No such file or
directory')
Since someone might ask about permissions:
[root@file1 subsys]# ls -laRZ /var/log | grep setroubleshoot
drwxr-xr-x root root system_u:object_r:setroubleshoot_var_log_t
setroubleshoot
/var/log/setroubleshoot:
drwxr-xr-x root root system_u:object_r:setroubleshoot_var_log_t .
-rw-r--r-- root root system_u:object_r:setroubleshoot_var_log_t
setroubleshootd.log
-rw-r--r-- root root system_u:object_r:setroubleshoot_var_log_t
setroubleshootd.log.1
-rw-r--r-- root root system_u:object_r:setroubleshoot_var_log_t
setroubleshootd.log.2
Can anyone explain why setroubleshootd_disable_trans should need
to be
selected? Also, since this entire event seems to have close ties to
selinux, would anyone have an idea what might have happened to
this system?
Thanks for any ideas; it's been a long day...
Steven Stromer
