Re: filtering HTTPS

On 14.03.2012 11:27, Marcus Kool wrote:
> Henrik Nordström wrote:
>> tis 2012-03-13 klockan 12:12 -0300 skrev Marcus Kool:
>>
>>> Where does the filtering gets involved? Also NoneSSL sites (aka
>>> tunnelmode) need to be filtered/blocked and/or scanned for virusses.
>> Squid is not the tool for filtering non-http(s) traffic beyond
>> requested
>> hostname.
>
> I agree. Squid is not. This task is for the URL rewritors and ICAP
> servers.
> One way or another, Squid should offer all data that passes through
> it (1)
> to a filter. I like ICAP, but ICAP is designed for HTTP and not
> HTTPS
> and certainly not for non-HTTP, non-HTTPS data streams.
>
> (1) a virusscanner and a URL filter do not need the *whole* data
> stream.
> The first max-64K upload and the first max-64K download is most
> likely
> sufficient to determine what to do, pass or block. The protocol
> should
> have a feature that the filter is able to tell to Squid "Continue
> with
> this data stream, but I am not interested in it any more".

It does. http://www.squid-cache.org/Doc/config/icap_206_enable/

>
>> But it would be trivial to extend tunnel mode with a filter pipe,
>> both
>> in normal tunnel mode and SSL relay mode (decrypted & encrypted,
>> tunneling between two SSL connections).
>
> A filter pipe is interesting. A question is on how to implement it.
> ICAP has no support for it and in my opinion ICAP should be extended
> to support this. I know it is a long way to extend existing protocols
> but maybe it works by just doing it and making it a de facto
> standard.

Yes the idea is interesting. I've been trying to figure out how to do
it for a while.

We first need to get CONNECT requests flowing through that BodyPipe
system though. Right now it is hard-coded to do raw FD->FD packet
shuffling straight out of ConnStateData.

Step 1 is some good tests to find out what actually happens right now
when the hard-coded CONNECT handling is dropped out of ConnStateData and
http.cc is allowed to process CONNECT requests through the pipe system.

I expect it would work okay with maybe with some minor pipe arrangement
decisions to tweak, and speed problems from the storage mechanisms
getting in the way.

Alex recent work on storage has moved a few steps closer to shunting
the storage bottleneck out of the way, but AFAIK nobody has yet taken
that final step to specifically handle the CONNECT method in the
store-vs-transit decision.

>
> The question is what works best:
> A) use extended ICAP for regular HTTP(S) and data streams
> B) use ICAP for regular HTTP(S) and a new data stream protocol for
> data streams

(B) would if the switch can be determined in Squid easily. ICAP
wrapping and unwrapping adds a huge processing overhead to every
request.

Another option is to notice any Upgrade: headers in the CONNECT
requests. That is a major hint about what the tunnel contains.

Amos
Received on Tue Mar 13 2012 - 23:33:54 MDT