Re: Transparent IPv6 proxying on OpenBSD

Hello,

> Your expectation is correct.
> For outgoing connections comm_openex() should always receive a
> destination IP address and port in the remote field.

Well this is odd actually, because I got it working this morning and
comm_openex() still has remote=[::]

A few details need to be cleared out first. My setup looks like this:

+---| Internet |---+
       |
       |
    +-----+
    | fw1 |
    +-----+
       |
       |
    +-----+
    | br1 |
    +-----+
       |
       |
+---Shared LAN---+

Squid is running on br1, which, as the name suggests is a bridge.
From the wiki page, it seemed bridges can be sources of problems with
TPROXY, and the solution is to block IPv6 traffic on the bridging
device.

Since there is no equivalent to ebtables on OpenBSD, and I couldn't get
PF to do it, I assigned IPv6 addresses on the bridge's interfaces.
After setting up the corresponding routes, it worked!

The client machine behind the bridge can browse the IPv6 internet and I
see the corresponding entries in access.log
It also gets blocked when the destination URL matches an ACL in
squid.conf (squidGuard doesn't support IPv6 AFAIK).

Some mystery still remains though:
   - the remote address reported by comm_openex()
   - the fact that I had to add this rule before the rules handling the
     redirect part:

      pass quick inet6 proto udp from <lan_networks> to any port domain

     I should add that without the redirect rules, both the bridge and the
     machines behind it can do DNS lookups.

   - the fact that I had to make br1 visible on the IPv6 network. I say
     that this is odd, as a test with relayd(8) (a relay daemon provided
     in the OpenBSD base system) showed me that transparent IPv6
     proxying works fine on a bridge, without any additional rules
     for DNS queries.

> The above is wrong. tempaddr should always meet the not-set test
> immediately after being allocated.
>
> + struct in6_addr tempaddr = { };
> + inet_pton(AF_INET6, "2001:db8:100::1", &tempaddr);
>
>
> Please also use an identifiably bogus address provided the test still
> works reliably with it. The Linux tests use ::2 for this purpose.

Onto the patch details now; firstly thank you for reviewing it.

Yes, using the IN6_IS_ADDR_UNSPECIFIED doesn't make much sense.
As for the IP address used, I suppose you can change it to ::2 if you
want - 2001:db8::/32 should never be encountered either as it is the
IPv6 documentation prefix.

--
Marios
On 7 March 2012 12:13, Marios Makassikis <mmakassikis_at_gmail.com> wrote:
> Hello list,
>
> I am in the process of setting up a lab to test out IPv6.
> The host operating system is OpenBSD, -current as of 12/02/2012.
> From what I've read on the wiki, it is possible to do transparent IPv6
> interception
> of HTTP requests. However, this is specific to Linux, as it uses the
> tproxy kernel
> module.
>
> After some researching and discussion on #squiddev, I believe this would also be
> possible on OpenBSD.
> I first compiled squid-3.2.0.15 and tested it with IPv4 to make sure I
> can get a basic
> configuration working. The next step is to get PF (OpenBSD's firewall)
> to intercept
> IPv6 HTTP requests and redirect them to squid. This is done using the
> following rules:
>
> pass in quick on $int_if inet6 proto tcp from <lan_networks> to any port www \
>        route-to lo0 divert-to ::1 port 3129
> pass out quick inet6 from <lan_networks> divert-reply
>
>
> The second rule is needed to receive replies for sockets that are
> bound to addresses
> not local to the machine.
> Obviously, some changes to squid's source code were needed, to make it
> accept the
> redirected packets.
> After applying the patch, one can use
>
> http_port [::1]:3129 tproxy
>
> in squid.conf
>
> At this point, I can see squid receiving the HTTP requests. By using
> debug_options 11,2
> and tcpdump/wireshark I can see that the TCP connection happening
> between the client
> and the proxy. At no point however is squid initiating a request with
> the destination server.
> The client gets a "(60) Connection timed out" after a (long) while.
>
> I tried to find where the problem might be coming from, but I haven't
> found anything.
> Using debug_options, I had this message at some point:
>
> comm_openex: Opened socket local=[ client_ipv6_address ] remote=[::]
> FD 14 flags=1 : family=24, type=1, protocol=6
> Write.cc(132) HandleWrite: FD 14 write failure: (32) Broken pipe.
>
> write(2) manpage says the following:
>     [EPIPE]            An attempt is made to write to a socket of type
>                        SOCK_STREAM that is not connected to a peer socket.
>
> Which seems to confirm my suspicions.
> Is the remote address reported by comm_openex normal? I'd expect to
> see the destination's
> IP in there.
>
> Does anyone see what is missing here to make this work?
>
> Thanks in advance,
>
> --
> Marios
>
> diff --git a/src/comm.cc b/src/comm.cc
> index 3a90a04..0545c26 100644
> --- a/src/comm.cc
> +++ b/src/comm.cc
> @@ -496,9 +496,20 @@ comm_set_transparent(int fd)
>         /* mark the socket as having transparent options */
>         fd_table[fd].flags.transparent = 1;
>     }
> +#elif _SQUID_OPENBSD_
> +    int tos = 1;
> +    enter_suid();
> +    if (setsockopt(fd, SOL_SOCKET, SO_BINDANY, (char *) &tos,
> sizeof(int)) < 0) {
> +        debugs(50, DBG_IMPORTANT, "comm_open: setsockopt(SO_BINDANY)
> on FD " << fd << ": " << xstrerror());
> +    } else {
> +        /* mark the socket as having transparent options */
> +        fd_table[fd].flags.transparent = 1;
> +    }
> +    leave_suid();
>  #else
>     debugs(50, DBG_CRITICAL, "WARNING: comm_open:
> setsockopt(IP_TRANSPARENT) not supported on this platform");
>  #endif /* sockopt */
> +
>  }
>
> /**
> diff --git a/src/ip/Intercept.cc b/src/ip/Intercept.cc
> index 446b3ea..64e0891 100644
> --- a/src/ip/Intercept.cc
> +++ b/src/ip/Intercept.cc
> @@ -141,7 +141,7 @@ Ip::Intercept::NetfilterInterception(const
> Comm::ConnectionPointer &newConn, int
>  bool
>  Ip::Intercept::NetfilterTransparent(const Comm::ConnectionPointer
> &newConn, int silent)
>  {
> -#if LINUX_NETFILTER
> +#if LINUX_NETFILTER || _SQUID_OPENBSD_
>     /* Trust the user configured properly. If not no harm done.
>      * We will simply attempt a bind outgoing on our own IP.
>      */
> @@ -428,8 +428,36 @@ Ip::Intercept::ProbeForTproxy(Ip::Address &test)
>         }
>     }
>
> -#else /* undefined IP_TRANSPARENT */
> -    debugs(3, 3, "setsockopt(IP_TRANSPARENT) not supported on this
> platform. Disabling TPROXYv4.");
> +#elif _SQUID_OPENBSD_
> +    debugs(3, 3, "Detect BINDANY support on port " << test);
> +
> +    int tos = 1;
> +    int tmp_sock = -1;
> +
> +    if (test.IsIPv6()) {
> +        debugs(3, 3, "...Probing for IPv6 SO_BINDANY support.");
> +
> +        struct sockaddr_in6 tmp_ipv6;
> +        struct in6_addr tempaddr = { };
> +        if (!IN6_IS_ADDR_UNSPECIFIED(&tempaddr)) {
> +            inet_pton(AF_INET6, "2001:db8:100::1", &tempaddr);
> +        }
> +        tmp_ipv6.sin6_addr = tempaddr;
> +        tmp_ipv6.sin6_family = AF_INET6;
> +        tmp_ipv6.sin6_port = htons(0);
> +
> +        enter_suid();
> +        if ((tmp_sock = socket(AF_INET6, SOCK_STREAM, IPPROTO_TCP)) >=0 &&
> +            (setsockopt(tmp_sock, SOL_SOCKET, SO_BINDANY, (char *)&tos,
> +                       sizeof(tos)) == 0) &&
> +            (bind(tmp_sock, (struct sockaddr*)&tmp_ipv6,
> sizeof(struct sockaddr_in6)) == 0)) {
> +            leave_suid();
> +            debugs(3, 3, "IPv6 BINDANY support detected. Using.");
> +            close(tmp_sock);
> +            return true;
> +        }
> +    }
>  #endif
> +
> +    debugs(3, 3, "setsockopt(IP_TRANSPARENT) not supported on this
> platform. Disabling TPROXYv4.");
> +
>     return false;
>  }