Re: [squid-users] Common log file format conversion??

Amos Jeffries <squid3_at_treenet.co.nz> escribió:

> On 29/04/11 16:03, michel_at_casa.co.cu wrote:
>> Hi List
>>
>> I recently installed Mysar utility for generating reports from Squid
>> logs, but the script does importer.php only import files in native
>> format logs into the database.
>>
>> I modified the configuration of my squid to generate logs in native
>> mode, since even here there are no problems, statistics generated Mysar
>> me perfectly.
>>
>> Where is the problem?
>
> You already said the answer to that:
> "the script does importer.php only import files in native format logs"
>
>>
>> Well, but I have several logs in common format, which would convert to
>> import into my database.
>>
>> There is a program or script that allows to convert several common log
>> files in native format?
>
> Maybe, but unlikely. Apache "common" format is missing many fields
> that are present in the native "squid" format. Conversion is usually
> the other way around, forcing squid logs into a format the apache
> log analyzers can handle.
>
>>
>> and that in turn allows to import the logs converted to mysql??
>
> Squid-3.2 bundles with a DB logging daemon, log_db_daemon. Also
> available from sourceforge under the name log_mysql_daemon.
>
> That depends on your mysql database accepting squid native log
> format though. It provides many useful DB views, but is still a
> little rough around the edges on the graphing and display areas (ie
> none exist publicly).
>
> Amos
> --
> Please be using
> Current Stable Squid 2.7.STABLE9 or 3.1.12
> Beta testers wanted for 3.2.0.7 and 3.1.12.1
>

Hi Amos

First of all, I thank you for answering my email quickly.

I use CentOS, and my version of squid is squid-2.6.STABLE21-6.el5

I configured my squid to generate logs in native mode to Mysar could
generate the reports. as I mentioned in my previous email to the list.

   reports are generated fine.

1304127653.928 56014 10.10.1.1 TCP_MISS/200 289 GET
http://0.161.channel.facebook.com/x/692408761/1079584136/false/p_100000248321976=384 - DIRECT/66.220.151.82
text/plain
1304127653.956 2354 10.10.1.50 TCP_MISS/200 1539 POST
http://www.facebook.com/ajax/photos/theater/session_logging.php? -
DIRECT/69.63.189.26 application/x-javascript

But I need to import some old logs that are common formats:

10.10.1.1 - - [01/Apr/2011:06:27:40 -0400] "GET
http://clients1.google.com/complete/search? HTTP/1.1" 200 636
TCP_MISS:DIRECT
10.10.1.50 - - [01/Apr/2011:06:27:40 -0400] "GET
http://clients1.google.com/complete/search? HTTP/1.1" 200 619
TCP_MISS:DIRECT

Is that this format is not containing such information as the previous
one and that the conversion process much information is lost, you
still need to import into the database even if you lose certain fields.

Know any script that's already done by someone to allow me to do this,
or even a functional or bash script that generates the dates in unix
timestamp from the log in common format to begin I make my own script?

I found several scripts that allow me to make the conversion of logs
from native format in common. but neither do it backwards.

Thanks

----------------------------------------------
Webmail, servicio de correo electronico
Casa de las Americas - La Habana, Cuba.
Received on Sat Apr 30 2011 - 01:55:06 MDT