Re: Logging commands: TCP logger vs Logging daemon

On 01/24/2013 10:00 PM, Amos Jeffries wrote:
> On 24/01/2013 8:52 a.m., Alex Rousskov wrote:
>> Hello,
>>
>> I noticed that "tcp://host:port" logger does not get "rotate logs"
>> and other commands that the "daemon:foo" helper gets. The TCP logger
>> just gets the logging line, without any prefix to accommodate future
>> commands or extensions. This seems rather unfortunate because it
>> prevents Squid from reporting significant logging-related events to the
>> remote TCP logger.
>>
>> Log rotation is one example. Another example would be reporting the
>> number of lost (due to connectivity problems) access.log records to a
>> remote logger (if we start supporting such a feature).
>>
>>
>> Is it to late to adjust documentation so that all logging helpers can
>> get logging commands?
>>
>> If it is too late, how about adding an "understands commands" option to
>> access_log so that loggers that can understand commands can receive them?
>>
>>
>> Thank you,
>>
>> Alex.
>
> There are at least two users making use of the TCP output method that I
> am aware of.
>
> At this point it woudl be a little messy but still possibly worth doing
> as a concept.
>
> However, this tcp: module is designed as a stream replacement of
> access.log stdio output all the tools are designed to operate with.

Right, but I assume that none of those stdio tools can deal with a TCP
stream "as is" and if a tool is adjusted to deal with the TCP stream, it
can be also adjusted to filter commands out.

Even in the trivial case of a script using "nc" or similar to convert a
TCP stream into a file on a remote host, that script can add a grep step
to get rid of commands.

I agree that we will inconvenience the existing TCP loggers. If that is
a significant problem, we would have to add an "understand commands"
option (if there is consensus that such logging commands are useful, of
course).

> Details such as when rotation is requested by the (now external) admin
> or when Squid starts/stops/reconfigures have never been logged to the
> access.log files before (udp: and syslog: do not get them either) and
> cache.log is not sent over the tcp: module.

That design seems reasonable to me as far as syslog and files are
concerned: Neither a file nor syslog can handle commands. On the other
hand, a custom program receiving our entries can.

Please note that I am not trying to argue with your points (which are
all valid) or defend a particular solution at this point. I am just
discussing and evaluating available options...

> As for lines lost. That might be worth recording. I am thinking a log
> line " ... lost NNN lines during TCP connection outage ..." would be
> sufficient ??

but logged where?

* Logging the above line to cache.log would be easy, but it would be
difficult for automation tools to collect that information and correlate
it with actual log entries, especially if cache.log is not sent over the
tcp: module to the remote processing daemon.

* Logging the above line to access.log (or equivalent stream) as if it
were a log entry, will violate the log format (which would be worse than
adding command support IMO).

Thank you,

Alex.
Received on Sat Jan 26 2013 - 05:44:03 MST