On Sun, 13 Nov 2011 23:09:23 +0100, Markus Thüs wrote:
Hi,
here’s the case: I’ve implemented a squid proxy at a school which
requires
the users to authenticate against an LDAP Server. That means when the
user
enters a web-address in the browser the Proxy requires the user to
authenticate himself, meanwhile squid logs everything in the
background.
Day by day where gathering ~ 550 MB of Access.logs a day.
Fine so far… Now theoretically let’s say a note from the local police
station arrives saying that some user watched something illegal - via
the
schools DSL Line - the data protection officer must be able to tell
who of
the users did that.
How can I give that kind of functionality to that officer !? In
that case
he needs to analyze all logs of that year (365 Files) by means of per
user
analysis and per Page / Domain. So an analysis which pages the
user
visited when and how often from which place AND a search for which
users
view a certain page / domain.
You are going beyond log analysis there (pretty graphs) and into data
mining.
The old popular sarg, calamaris tools will give you graphs with a bit
of drill-down into those categories. But not searching AFAIK.
The various database log tools and analysers are probably where you
want to look. There are several appearing in popularity now that daemon
loggers can be plugged into Squid and pipe the log entries to DB.
Amos
