Search squid archive

Re: Common log file format conversion??

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Amos Jeffries <squid3@xxxxxxxxxxxxx> escribió:

On 29/04/11 16:03, michel@xxxxxxxxxx wrote:
Hi List

I recently installed Mysar utility for generating reports from Squid
logs, but the script does importer.php only import files in native
format logs into the database.

I modified the configuration of my squid to generate logs in native
mode, since even here there are no problems, statistics generated Mysar
me perfectly.

Where is the problem?

You already said the answer to that:
  "the script does importer.php only import files in native format logs"


Well, but I have several logs in common format, which would convert to
import into my database.

There is a program or script that allows to convert several common log
files in native format?

Maybe, but unlikely. Apache "common" format is missing many fields that are present in the native "squid" format. Conversion is usually the other way around, forcing squid logs into a format the apache log analyzers can handle.


and that in turn allows to import the logs converted to mysql??

Squid-3.2 bundles with a DB logging daemon, log_db_daemon. Also available from sourceforge under the name log_mysql_daemon.

That depends on your mysql database accepting squid native log format though. It provides many useful DB views, but is still a little rough around the edges on the graphing and display areas (ie none exist publicly).

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.7 and 3.1.12.1


Hi Amos

First of all, I thank you for answering my email quickly.

I use CentOS, and my version of squid is squid-2.6.STABLE21-6.el5

I configured my squid to generate logs in native mode to Mysar could generate the reports. as I mentioned in my previous email to the list.

  reports are generated fine.


1304127653.928 56014 10.10.1.1 TCP_MISS/200 289 GET http://0.161.channel.facebook.com/x/692408761/1079584136/false/p_100000248321976=384 - DIRECT/66.220.151.82 text/plain 1304127653.956 2354 10.10.1.50 TCP_MISS/200 1539 POST http://www.facebook.com/ajax/photos/theater/session_logging.php? - DIRECT/69.63.189.26 application/x-javascript



But I need to import some old logs that are common formats:

10.10.1.1 - - [01/Apr/2011:06:27:40 -0400] "GET http://clients1.google.com/complete/search? HTTP/1.1" 200 636 TCP_MISS:DIRECT 10.10.1.50 - - [01/Apr/2011:06:27:40 -0400] "GET http://clients1.google.com/complete/search? HTTP/1.1" 200 619 TCP_MISS:DIRECT


Is that this format is not containing such information as the previous one and that the conversion process much information is lost, you still need to import into the database even if you lose certain fields.

Know any script that's already done by someone to allow me to do this, or even a functional or bash script that generates the dates in unix timestamp from the log in common format to begin I make my own script?

I found several scripts that allow me to make the conversion of logs from native format in common. but neither do it backwards.

Thanks


----------------------------------------------
Webmail, servicio de correo electronico
Casa de las Americas - La Habana, Cuba.




[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux