Re: [squid-users] squid ssl-to-ssl reverse proxy issue

On 15-11-2009 12:21, Angelo Höngens wrote:
> On 13-11-2009 23:00, Angelo Höngens wrote:
>> Hey guys and girls, I'm back on the list again to ask for your help with
>> a really strange problem.
>>
>>
>> I am having issues with ssl-to-ssl proxying with multiple applications,
>> but the one I am describing below is bugging me the most. I hope you can
>> help me debug and solve the issue.
>>
>>
>> THE SETUP
>>
>> I have a setup where I have a web application (Microsoft Team Foundation
>> Server Web Access) on an IIS6 machine, running on https port 8091
>> running in my internal lan.
>>
>> I want to publish this to the internet, and our policy is to run
>> everything through a Squid in our DMZ. This is a 32-bit ESX VM running
>> FreeBSD 6.2 with Squid 3.0.STABLE20 , the latest 3.0 from the ports.
>> (Next on my list is to upgrade this machine to FreeBSD 7.2).
>>
>> I've set up a NAT rule so external visitors go to
>> https://externalname:8091, and they end up on the squid in the dmz. The
>> squid, in the backend, connects to https://tfsserver:8091 in the
>> internal lan.
>>
>> The reason I use https on the backend, is that the MS web application is
>> buggy, and if I use http, the application sends absolute redirects to
>> point clients to http on port 8090. I don't have this issue when I use
>> https in the backend.
>>
>> THE PROBLEM
>>
>> External access does not work, I get strange timeouts. Will explain more
>> below.
>>
>> TROUBLESHOOTING
>>
>> I've been able to reproduce the problem with a single http request. I
>> make a specific request from an external machine:
>>
>> curl \
>> --cookie-jar cookie.txt \
>> --cookie cookie.txt \
>> --basic --insecure \
>> -o out.txt \
>> -D - \
>> --user username:password \
>> "https://externalname:8091/index.aspx?pname=Project" \
>>
>> If I run the request, I get a status 200 and some nice response headers,
>> and then comes the content.. after 18152 bytes of data (should be around
>> 20478 bytes), it stalls most of the time for minutes (sometimes it
>> completes though). This is the issue. Instead of completing in several
>> milliseconds, it just hangs there, and I don't know what's hanging. The
>> backend IIS server reports the page was successfully retrieved by squid
>> within 0-30ms.
>>
>> After a while things time out (set read_timeout to 30 secs in the
>> following example), and the curl prints the message: "curl: (18)
>> transfer closed with 2327 bytes remaining to read", and in my access.log
>> I see: "1258129110.004 29433 82.94.123.123 TCP_MISS/200 18534 GET
>> https://externalname:8091/index.aspx?pname=Project -
>> FIRST_UP_PARENT/tfsweb-https text/html", like nothing strange happened
>> except the long time. Nothing in the cache.log.
>>
>> However, if I make the exact same request without the squid in between
>> (so directly from external to the TFS server), the request always
>> completes in a few milliseconds. If I make the exact same request from
>> the squid machine to the TFS server it also completes in a few
>> milliseconds. That leads me to believe network is not the problem.
>>
>> If I use the http backend (http://tfsserver:8090 instead of
>> https://tfsserver:8091) it works fine as well (although the application
>> gives other problems because of the absolute redirects). So this leads
>> me to believe it's something in squid..
>>
>> Does anyone have any idea on how to troubleshoot this issue, for example
>> what debug section and what level? I've spent a few hours debugging all
>> on level 4, but I get thousands of lines in the logging, and can't find
>> anything interesting.
>>
>>
>> The interesting lines from my config (there are more sites there, but
>> not interesting for now):
>>
>> https_port 8091 cert=/bla/externalname.pem key=/bla/externalname.pem
>> vhost vport
>> cache_peer tfsserver parent 8091 0 no-query originserver ssl
>> sslcafile=/etc/ssl/tfsroot.pem name=tfsweb-https login=PASS
>> cache_peer_access tfsweb-https deny port80
>> cache_peer_access tfsweb-https deny port443
>> cache_peer_access tfsweb-https allow port8091
>> cache_peer_domain tfsweb-https dstdomain external name
>>
>> Thanks in advance for your time (replies to list please).
>>
>
> (Reply to myself) I have upgraded the machine to the latest FreeBSD 7.2
> release to be sure its not some strange OS issue, but that hasn't solved
> the problem.
>
> Ideas anyone?
>

I downgraded to squid-2.7.7, the latest squid2 from the ports, and the
problem is not there,everything is blazing fast.

There must be a bug somewhere in 3.x.

-- 
With kind regards,
Angelo Höngens
systems administrator
MCSE on Windows 2003
MCSE on Windows 2000
MS Small Business Specialist
------------------------------------------
NetMatch
tourism internet software solutions
Ringbaan Oost 2b
5013 CA Tilburg
+31 (0)13 5811088
+31 (0)13 5821239
A.Hongens_at_netmatch.nl
www.netmatch.nl
------------------------------------------