Re: Policy routing MTU issue (was: transparent proxying)

On Fri, 25 Sep 1998, Apiset Tananchai wrote:

> > It is hard to tell which redirection mechanism that is the best one of
> > the two. Linux ipfwadm is fast but it has some MTU related problems
> > (does not work well together with MTU path discovery Squid->client).
>
> Would you please explain more about MTU path discovery problem when using
> transparent proxy on Linux? I'm using cisco's ip policy route-map to
> redirect web traffic to squid running on Linux2.0.36p10. The problem is
> that some client can't browse the web while most others can and it seems
> to relate to MTU setting on each hop from client to our server.

I don't know about linux, but there is a cisco related MTU issue that
occures when doing policy routing. This may have been addressed in newer
IOS images, but it's not fixed in 11.2(5)P on the 4500.

The problem I have noticed occures when policy routing is enabled on a
cisco. When a large packet is encountered and it needs to be forwarded
down a link with a smaller MTU than the originally transmitted size,
usually 1500 for ethernet, an ICMP message is supposed to be sent back to
the client requesting it to decrease the packet size. This doesn't always
need to happen, as some devices can be configured to 'store and forward'
the packet, allowing it to be re-assembled and fragmented into the
required size.

Some people have speculated that the cause of the MTU problem with policy
routing on cisco's is that this message is mistakenly sent to the
destination instead of the source of the packet, not only is this
incorrect it doesn't make any sense.

Debugging ICMP on my cisco indicates that the problem is not related to
this ICMP message at all. With a little help from IP-Filter on a FreeBSD
box it becomes evident what happens. I tested pinging 4k packets between
two FreeBSd boxes seperated by a Cisco 4500 doing policy routing. The 4k
packet is being fragmented by the freebsd box before it can be sent via
ethernet (1500 MTU). These packets weren't being policy routed, they are
just being compared against the route map and being passed on as per the
route table. Since 3 fragmented packets are sent and only one is received
it would indicate that the cisco is actually dropping the remaining
fragments of the original packet. (I tested this on a live system, so I
couldn't enable lots of debugging to work out all the specifics)

Here is an example of this in action (FreeBSD -> c4500 -> FreeBSD):
>ping -s 1472 remote
PING remote (x.x.x.x): 1472 data bytes
1480 bytes from x.x.x.x: icmp_seq=0 ttl=254 time=7.291 ms
1480 bytes from x.x.x.x: icmp_seq=1 ttl=254 time=7.343 ms

>ping -s 1473 remote
PING remote (x.x.x.x): 1473 data bytes
--- remote ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

Using IP-Filter to monitor the traffic, I see the remote machine
receives a single 1500 byte packet for both commands. Increasing the
packet size further results in the same single 1500 byte packet.

(Assuming cisco haven't already fixed this)
The only way to fix this is to ensure that every network segment between
the client and your cisco has an MTU no smaller than the cisco's and
clients' MTUs. And the clients MTU isn't larger than the cisco's. There
is also the issue that any client using any protocol that generates
fragmented packets will fail to work if they get routed by this cisco.

Hope this helps.

Seeya...Q

               -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
                        
                          _____ / Quinton Dolan - q@fan.net.au
  __ __/ / / __/ / / Systems Administrator
     / __ / _/ / / Fast Access Network
  __/ __/ __/ ____/ / - / Gold Coast, QLD, Australia
                    _______ / Ph: +61 7 5574 1050
                           \_\ SAGE-AU Member
Received on Fri Sep 25 1998 - 21:25:53 MDT