Re: [squid-users] Fwd: Howto Measure bandwidth consumption of HTTPS traffic?

M Admin wrote:
> Amos and Sir June -- Thanks for the reply. I have a couple more questions.
>
> Sir June -- Can you actually get a protocol breakdown and user
> breakdown from monitoring ETH0 and using MRTG?
>
> Amos -- Great tip. Will the log change that you suggested accurately
> capture all HTTPS traffic from the client to the internet server? I
> assume that all HTTPS traffic is routed through the proxy. i.e. If the
> client send 1 mb of data to Gmail, will my SQUID logs show 1 MB of
> data? It doesnt seem like it does.

I've found it to capture the data-size going outward through
CONNECT/POST/PUT. Which the default squid log misses. It will also log
the bytes used for HTTP headers in that count.

Don't forget the data is encrypted, probably compressed, and maybe
HTTP-form-encoded too which may alter the data size considerably.

That log format %S tag captures each byte being transferred by Squid.

Amos

>
> On Thu, Apr 16, 2009 at 4:00 AM, Amos Jeffries <squid3_at_treenet.co.nz> wrote:
>> M Admin wrote:
>>> Hello everyone --
>>>
>>> I am currently proxying all traffic from the client through SQUID. I
>>> am trying to measure the amount of bandwidth used by the client. The
>>> client is connecting to Gmail.com via HTTPS.
>>>
>>> I see the connections to Gmail in the access.log as such:
>>>
>>> 1239680667.335 216115 172.19.240.27 TCP_MISS/200 2964 CONNECT
>>> mail.google.com:443 - DIRECT/74.125.155.18 -
>>>
>>> but it doesnt seem like all client requests show up in the log. I am
>>> running Firebug 1.3.1 in the client and I see many GET and POST
>>> requests from the client to Google that don't show up in the
>>> access.log.
>>>
>>> Can I use SQUID for this function? Ie measure bandwidth for HTTPS
>>> traffic for 1 and eventually multiple users? Is it accurate?
>>>
>> Default squid log formats are currently NOT accurate to the byte for
>> accounting.
>>
>> For byte-accurate accounting you need to use the format:
>> logformat altsquid %ts.%03tu %6tr %>a %Ss/%03Hs %st %rm %ru %un %Sh/%<A %mt
>>
>> or for common log format:
>> logformat althttpd %>a %ui %un [%tg] "%rm %ru %rv" %Hs %st %Ss:%Sh:%<A
>> "%{Referer}>h" "%{User-Agent}>h"
>>
>> (NP: the above are meant to be single long lines, watch the whitespace
>> wrap).
>>
>>
>> Amos
>> --
>> Please be using
>> Current Stable Squid 2.7.STABLE6 or 3.0.STABLE14
>> Current Beta Squid 3.1.0.7
>>

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE14
   Current Beta Squid 3.1.0.7