on 17:50 Thu 24 Mar, Lamar Owen (lowen@xxxxxxxx) wrote:
> On Thursday, March 24, 2011 05:37:41 pm Dr. Ed Morbius wrote:
> > on 17:14 Thu 24 Mar, Lamar Owen (lowen@xxxxxxxx) wrote:
> > > Prior to PostgreSQL supporting syslog I used [logger] to
> > > pipe PostgreSQL output to syslog. Worked fine.
>
> > I haven't, looking at it.
>
> It is one option that is definitely in vanilla CentOS.
Quite.
> > OK. Any pointers on configuration are greatly appreciated. Docs, etc.
>
> Whew. Large scale remote syslog operation is a large subject; I've
> never had anything large-enough scale to need more than logwatch or
> site-grown scripts to do processing. The biggest thing to do is set
> up NTP and have three reference time sources (three so that if one is
> wrong you know which one). Otherwise, log correlation is impossible.
It is. There've been a few advances in sysadmin practice since the
Nemeth books were first produced, and while there are some titles
dealing with portions of this, codifying practices in docs would be a
wonderful thing. I've considered (and been approached regarding)
tackling at least parts of this myself.
Useful logging is definitely part of this.
> > Yeah, we're aware of that (I mentioned this in my first post to the
> > thread).
>
> Yep, that you did.
>
> > We've got a locally-compiled version of nginx, so patching isn't out of
> > the question. Just looking at all our options.
>
> While CentOS doesn't provide nginx itself, it does provide tools for
> dealing with logs; I saw several things doing a 'yum list | grep log'
> (I know there's easier ways of doing that; that's just the way I
> prefer to go about it). Also try grepping a yum list for 'watch' as I
> remember some logwatching stuff.....
Right, and the general solution also generalizes to other tools.
Postgresql (which we aren't using currently) also has its own log
handler (a small frustration of mine with the database).
And I turned up the rsyslogd feature:
http://www.rsyslog.com/doc/imfile.html
Text File Input Module
Module Name: imfile
Author: Rainer Gerhards <rgerhards@xxxxxxxxxxx>
Description:
Provides the ability to convert any standard text file into a syslog
message. A standard text file is a file consisting of printable
characters with lines being delimited by LF.
The file is read line-by-line and any line read is passed to
rsyslog's rule engine. The rule engine applies filter conditons and
selects which actions needs to be carried out.
As new lines are written they are taken from the file and processed.
Please note that this happens based on a polling interval and not
immediately. The file monitor support file rotation. To fully work,
rsyslogd must run while the file is rotated. Then, any remaining
lines from the old file are read and processed and when done with
that, the new file is being processed from the beginning. If
rsyslogd is stopped during rotation, the new file is read, but any
not-yet-reported lines from the previous file can no longer be
obtained.
When rsyslogd is stopped while monitoring a text file, it records
the last processed location and continues to work from there upon
restart. So no data is lost during a restart (except, as noted
above, if the file is rotated just in this very moment).
Currently, the file must have a fixed name and location (directory).
It is planned to add support for dynamically generating file names
in the future.
Multiple files may be monitored by specifying $InputRunFileMonitor
multiple times.
--
Dr. Ed Morbius, Chief Scientist / |
Robot Wrangler / Staff Psychologist | When you seek unlimited power
Krell Power Systems Unlimited | Go to Krell!
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos
