Re: [squid-users] Re: squid 3.2.0.14 with TPROXY => commBind: Cannot bind socket FD 773 to xxx.xxx.xxx.xx: (98) Address

On 09/14/2013 10:13 PM, Nikolai Gorchilov wrote:
> On Sat, Sep 14, 2013 at 9:36 PM, Eliezer Croitoru <eliezer_at_ngtech.co.il> wrote:
>> Hey,
>>
>> it can be tested in a matter of minutes.
>> If we have some test candidate I will write a small tproxy script to
>> verify the suspect.
>
> The pseudo code I have provided is based on my real-world experiment.
> I did the test myself, before posting my findings here.
>
> Let's focus our attention on how to overcome the issue. The only
> working application-level solution suggested till now is to select a
> random port before bind and retry on EADDRINUSE with another port.
>
> Any other ideas?
>
I have found the problem and I will rephrase the problem description:
While using tproxy the main issue is that the ports of the source IP is
beeing decreased to half for the same pair of ip:Xport to ip:Xport.
Which means that 192.168.1.1 cannot connect like regular proxy to 65k
ports but to 32k ports which makes IP "cheaper".
it's the same for server and client both..
While using the port range of:
# cat /proc/sys/net/ipv4/ip_local_port_range
32768 32867
#end
the main issue is that the OS tries to bind using a 0 value maximum
ports per IP by the above mentioned value.
the kernel itself wont even try to bind an already binded ip+port so
there is no need for the upper layers of the user-land to "recover" from
such a state.
leaving these matters to the kernel level is much more appropriate from
any aspect you look at the OS.
while running the test you have suggested I have seen the next issue:
#!/usr/bin/ruby

require 'socket'
times = 0
110.times do
  remote_connection = Socket.new(Socket::AF_INET,Socket::SOCK_STREAM,0)
  remote_connection.setsockopt(Socket::SOL_IP, 19, 1)
  sockaddr = Socket.sockaddr_in(0, '192.168.1.234')
  times += 1
  begin
    result = remote_connection.bind(sockaddr)
  rescue
    puts "Bind problem at try #{times} with ip 192.168.1.234"
  end
  puts "Bind result: #{result}, #{times}"
end

puts "and just try another couple times to see that the
ip_local_port_range looks at each IP as a local IP"

times = 0
10.times do
  remote_connection = Socket.new(Socket::AF_INET,Socket::SOCK_STREAM,0)
  remote_connection.setsockopt(Socket::SOL_IP, 19, 1)
  sockaddr = Socket.sockaddr_in(0, '192.168.1.233')
  times += 1
  begin
    result = remote_connection.bind(sockaddr)
  rescue
    puts "Bind problem at try #{times} with ip 192.168.1.233"
  end
  puts "Bind result: #{result}, #{times}"
end

puts "end if test"
#end

Which shows clearly that the main issue dosn't belong to the application
land at all but to the kernel level and understanding the main issue
about how tproxy and other sockets works on linux.

There is an option about this ip_local_port_range thing that the source
port will be already opend towards the server if the application tries
to choose the port.
and as a answer about 3des or rsa encryption: how would you prefer the
driver of the encryption cards to work? on the kernel land or the user
land??

Eliezer
Received on Sat Sep 14 2013 - 21:52:44 MDT