Re: squid3HEAD/TPROXY: interception log entries

Ritter, Nicholas wrote:
>
>
> -----Original Message-----
> From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
> Sent: Thursday, July 24, 2008 9:43 PM
> To: Ritter, Nicholas
> Cc: Amos Jeffries; squid-dev_at_squid-cache.org
> Subject: RE: squid3HEAD/TPROXY: interception log entries
>
>> Amos-
>>
>> Thanks for your reply.
>>
>> Ok...I think I am understanding what you are saying. Now translating
>> that to the situation at hand, I am not sure which route to take. I
>> switched my setting for http_port to:
>>
>> http_port 3128 tproxy intercept disable-pmtu-discovery=always
>>
>> Which I understand doesn't address the issue you commented below about
>
>> seperating intercept and tproxy traffic. If I take out the intercept,
>> I get an error on the client.
>
>> What error on the client? That sounds like a bug. tproxy option AKAIK
> should be independent. Maybe that is a bug
>> that need fixing.
>
> The error the client gets is:
>
> Connection to <server name Failed.
> The system returned:
> (99) Cannot assign requested address.
>
> It is recreateable with this one part of the specific app, it something
> related to server-side redirection. Doing a reload on the broswer makes
> the error go away.
>
>
>
>> I can't take tproxy out because that is what is being redirected via
>> iptables rules and provides the client IP preservation to the remote
>> site.
>
>>> Agreed. My belief is that all you should need is the 'tproxy' option.
> If thats failing, we need to find out why >>and fix it.
>
> Yup...I take take intercept keyword out and it doesn't work correctly.
>
>> I am confused as to how to seperate
>> what with where given incoming WCCP traffic from the router.
>
>>> I'll have to learn a little more about WCCP before I can answer that.
>
> I know the router redirects the traffic to the squid box on the GRE
> tunnel, then either a problem with my iptables rules or something else
> hits a snag of some kind. the tproxy rule is in the prerouting chain, so
> it gets the HTTP redirected traffic before it hits the GRE tunnel...if I
> understand things correctly. But I notice that the whole setup will not
> work correctly if the GRE tunnel is not there. The other odd thing is
> that the whole setup works regardless of if I allow incoming WCCP
> packets on 2048 or not, which I thought was a violation of the WCCP
> spec, or maybe just would prevent extra WCCP features from working.
>
>
>
> <snippet from followup>
>> Ooops...I should have added the wccp2_server_info setting from
>> squid.conf:
>>
>> wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240
>> ports=80 wccp2_service_info 90 protocol=tcp
>> flags=dst_ip_hash,ports_source priority=240 ports=80
> </snippet>
>
>> On the router I have wccp redirection rules as note on the squid wiki
>> (note: the squid box is 10.48.33.2, clients are 10.48.1.0/24):
>>
>> ip wccp 80 redirect-list 121
>> ip wccp 90 redirect-list 121
>>
>> interface FastEthernet0/1.1
>> encapsulation dot1Q 1 native
>> ip address 10.48.1.1 255.255.255.0
>> ip helper-address 10.2.5.101
>> ip wccp 80 redirect in
>> ip wccp 90 redirect out
>> !
>> interface FastEthernet0/1.33
>> encapsulation dot1Q 33
>> ip address 10.48.33.1 255.255.255.0
>> ip wccp redirect exclude in
>>
>>
>> On the squid box I have TProxy and GRE related rules:
>>
>> TPROXY rules:
>> -A PREROUTING -p tcp -m socket -j DIVERT -A PREROUTING -p tcp -m tcp
>> --dport 80 -j TPROXY --on-port 3128 --on-ip 0.0.0.0 --tproxy-mark
>> 0x1/0x1 -A DIVERT -j MARK --set-mark 0x1 -A DIVERT -j ACCEPT
>>
>> GRE and WCCP rules:
>> -A INPUT -i gre0 -j ACCEPT
>> -A INPUT -i gre0 -j ACCEPT
>> -A INPUT -p gre -j ACCEPT
>> -A INPUT -i eth0 -p gre -j ACCEPT
>> -A FORWARD -j LocalFW
>> -A LocalFW -i lo -j ACCEPT
>> -A LocalFW -s 10.48.33.1/32 -p udp -m udp --dport 2048 -j ACCEPT
>>
>> Given what you are saying, am I to assuming correctly that my iptables
>
>> rules would need to be adjusted? I understand in principle that I
>> would need to create a redirection rule to squid on one port with
>> TPROXY, and another redirection/NAT/DNAT rule to a squid port with
>> intercept. Is this correct?
>
>>> Only if you wanted to use both. The one TPROXY rule(s) as you have
> should be enough.
>
> I think I must need both because leaving out the intercept keyword
> causes breakage, and this is what WCCP is for.

Ah, here was have a small misunderstanding.

Squid has two types of transparency:
  - DNAT/REDIRECT
      Uses the 'intercept' option to flag that NAT tables should be
tested to discover and log the original IP the client was trying to
connect to. Squid will make a brand new connection from its own IP to
the server.

  - TPROXY
      Uses 'tproxy' option to flag that special TPROXY tables should be
looked up in the kernel to find the original IP the client was trying to
connect to. This IP is then used as the source when replying to the
client. Traffic will exit the squid box with basically 'random' IPs
destined to local clients.

This all works fine if the squid box is the router. But of squid is on a
different box to the router. The traffic needs to be shuffled sideways
to the squid _box_ without changing.

WCCP is merely one of the ways to do the shuffling. It wraps the packets
in a gre 'tunnel' header and send to squid box where the squid box
itself unwraps them.

There are no special settings needed in squid to differ from WCCP policy
routing, or squid running on the router itself. It's all just traffic
coming in to the squid box eth0 interface.

This is why I think there is a flow problem thats blocking things for
you. Something is blocking the TPROXY bit flowing _out_ of squid. Either
by looping it back in again, or making duplicate outbound links at the
same time.

Hmm, I've found a spot which was possibly doing outbound bind without
pre-specifying TPROXY flags. Does the attached patch enable things to
work with just:

   http_port 3128 tproxy disable-pmtu-discovery=always

>
>>> What pops into my head here, without knowing the error your clients
> show is a feeling that some bit of the
>>> overall flow is playing nasty.
>>> Can you explain what the full traffic flow behavior is vs what should
> be?
>
> I agree, I will do a write up of the application and send it off on
> Monday. I have to deal with a power problem in a data center now and in
> to the weekend.
>
>
> Nick

Amos

-- 
Please use Squid 2.7.STABLE3 or 3.0.STABLE8