On Thu, 2010-04-08 at 13:52 -0700, Ray Van Dolson wrote:
> So how are people out there in mailing list land handling centralized
> logging?
>
> I'd like to mirror my web farm's logfiles on a central server, but want
> to be sure to preserve the W3C-ness so it's easy to run awstats, etc
> against them from off this central box.
>
> The options I see so far are:
>
> - Piping to syslog via external process (perl script, etc).
> * Downside being that we end up with syslog timestamps
> - Pipe to something like netcat
> * Custom-ish / hacky
> - Fire off some sort of synchronization process from central box to
> pull in logs (rsync) periodically.
> * Asynchronous -- there's a somewhat large window of time where
> logs will be missing on the internal server.
> - Log directly to shared, remote file server (via NFS)
> * Log server network issues or technical problems could affect
> Apache servers
>
> How do you guys typically deal with this?
I log to a fifo with Apache, then have syslog-ng picking it up and
firing it to a central syslog-ng server.
Local system:
mkfifo /var/log/httpd/weblogs_vhost.fifo
Local Apache Vhost:
CustomLog "/var/log/httpd/weblogs.fifo" combined
Local syslog-ng:
source s_apache_vhost { pipe("/var/log/httpd/weblogs_vhost.fifo"); };
destination d_weblogs_vhost { tcp("log-server" port("5140") template("$MESSAGE\n") template_escape(no)); };
log { source(s_apache_vhost); destination(d_weblogs_vhost); };
Remote syslog-ng:
source s_weblogs_vhost { tcp(ip(0.0.0.0) port(5140)); };
destination d_weblogs_vhost { file("/export/syslog/httpd/<vhost>/$YEAR/$MONTH/$DAY/access.log" template("$MESSAGE\n") template_escape(no)); };
log { source(s_weblogs_vhost); destination(d_weblogs_vhost); };
I usually dedicate a syslog-ng port to each vhost so I don't need to
setup any filters. YMMV.
The template and template_escape stuff ensures that the log entry on the
log servers disk is the exact same format as you'd get directly from
Apache using normal logs. You won't get syslog timestamps with this.
Mark.
--
Mark Watts BSc RHCE MBCS
Senior Systems Engineer, Managed Services Manpower
www.QinetiQ.com
QinetiQ - Delivering customer-focused solutions
GPG Key: http://www.linux-corner.info/mwatts.gpg
Attachment:
signature.asc
Description: This is a digitally signed message part
|