Re: [squid-users] Help with accelerated site

Hi Ron
Thanks for your reply and clarifications
Yes I have fully understood how it should work
My internal clients are all configured to use the proxy to have access to
the internet
with the proxy local IP address and port 3128.

Second when I try to access my servers from outside my network, meaning from
a totally different connection.
I get this

First I must apologise to everyone here, if my report is too long
please take a look at it and see what you can make out of it
Many thanks
Read bellow

#This configuration works as far as my network clients are concerned

http_port 80 accel defaultsite=reseaumondial.org vhost
cache_peer www.mysite.org parent 80 0 no-query originserver name=main

acl our_site dstdomain www.mysite.org
http_access allow our_site
cache_peer_access main allow our_site
cache_peer_access main deny all
*************************************************************************

acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32

acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network

acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT

#Acces conrol
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localnet
http_access deny all

icp_access allow localnet
icp_access deny all

htcp_access allow localnet
htcp_access deny all
http_port 3128
access_log /usr/local/squid/var/logs/access.log squid
cache_log /usr/local/squid/var/logs/cache.log
 cache_store_log /usr/local/squid/var/logs/store.log

refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320

cache_effective_user squid
visible_hostname proxy
icp_port 3130
 hosts_file /etc/hosts
coredump_dir /usr/local/squid/var/cache

#############################################################
So the above config works as far as letting my clients access the internet

here are some of my syslog logs, I have cleared the syslog and then made a
request outside my local area network to the website

Mar 26 18:20:13 proxy kernel: [257799.237070] IN=eth1 OUT=
MAC=XX:XX:XX:XX:18:00:0f:35:XX:XX:XX:XX:XX SRC=81.98.100.153
DST=81.98.104.57 LEN=52 TOS=0x00 PREC=0x00 TTL=62 ID=8741 DF PROTO=TCP
SPT=1395 DPT=3128 WINDOW=65535 RES=0x00 SYN URGP=0
*****************************************************************************************************************
The first IP address is the IP (81.98.100.153) address from which I am
trying to connect, it's a completely different connection altogether.

The second IP address (81.98.104.57) is the IP address of router BOX, the
router and the proxy server are both on the same Machine.
the eth1 is the wan or (internet interface)
eth2 is the lan (local network)

This is what I get on my browser
The connection was reset

  The connection to the server was reset while the page was loading.
    * The site could be temporarily unavailable or too busy. Try again in
a few
          moments.

    * If you are unable to load any pages, check your computer's network
          connection.

    * If your computer or network is protected by a firewall or proxy,
make sure
          that Firefox is permitted to access the Web.
###########################################################################################
here is my Iptables

#!/bin/sh
# squid server IP
SQUID_SERVER="192.168.1.4"
# Interface connected to Internet
INTERNET="eth1"
# Interface connected to LAN
LAN_IN="eth2"
# Squid port
SQUID_PORT="3128"
#
# Clean old firewall
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
# Load IPTABLES modules for NAT and IP conntrack support
modprobe ip_conntrack
#modprobe ip_conntrack_ftp

# For win xp ftp client
#modprobe ip_nat_ftp
echo 1 > /proc/sys/net/ipv4/ip_forward
# Setting default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
#iptables -t nat -A PREROUTING -i eth2 -p tcp -m tcp --dport 80 -j
DNAT --to-destination 192.168.1.3:80
#iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j
REDIRECT --to-ports 3128
# Unlimited access to loop back
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Allow UDP, DNS and Passive FTP
iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j
ACCEPT
# set this system as a router for Rest of LAN
iptables --table nat --append POSTROUTING --out-interface $INTERNET -j
MASQUERADE
iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT
# unlimited access to LAN
iptables -A INPUT -i $LAN_IN -j ACCEPT
iptables -A OUTPUT -o $LAN_IN -j ACCEPT
# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT)
transparent proxy
iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to
$SQUID_SERVER:$SQUID_PORT
# if it is same system
iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j
REDIRECT --to-port $SQUID_PORT
# DROP everything and Log it
iptables -A INPUT -j LOG
#iptables -A INPUT -j DROP

# modprobe ip_nat_ftp

if I type this http://localhost I get the default apache webpage "It Works"
if I type 192.168.1.3 I get the same thing as above
I stopped the apache on the webserver, and I still get the same page as
above
I stopped both apaches the one on the proxy machine and the webserver
I still get the default apache page "It Works" when I type 192.168.1.3

I am running out of ideas where does this come from, I have cleared the
browser's cache and I still get it, when I stop the proxy server, obviously
I get connection refused.

I have one question does Squid packaged with apache?
that's the only thing I can think of

You help would be much appreciated

Regards
Adam

----- Original Message -----
From: "Ron Wheeler" <rwheeler_at_artifact-software.com>
To: "Adam_at_Gmail" <adbasque_at_googlemail.com>
Sent: Friday, March 26, 2010 5:14 PM
Subject: Re: [squid-users] Help with accelerated site

>
> There are 2 uses for Squid:
> 1) to act as a proxy for browsers inside your network that want to get out
> to the Internet and you want to avoid 2 people downloading the same big
> file by having squid remember pages that it sees go by and giving the
> second requester the copy that is already in cache on its disk. In this
> case it is usually watching on port 3128 on the NIC attached to your
> internal LAN for requests that should be sent out on the public address.
>
> 2) To act as an accelerator for people outside who want pages from your
> web server. In this case it is watching for requests coming in on port 80
> on the NIC that carries the public address and cheching to see if the page
> that they are requesting is in its cache and if it is, it responds to the
> request without bothering the webserver.
>
> Note in Case 2, it is not doing anything for your people on the inside
> since they do NOT come in through the ethernet interface that Squid is
> watching.
>
> You have to be clear in your configuring and testing that you are testing
> with the right connections.
> If you are testing case 2, you need to be outside your network to test.
> If you come into port 80 on the ethernet NIC that is part of your internal
> LAN, your accelerator may not even see it.
>
> Make sure that your firewall setup matches what you are trying to do.
>
> If you have got everything set up for whichever case you are testing, you
> might want to ask some of these questions to see what is happening.
>
> What happens when you try to reference the proxy with a browser on port
> 80?
> What is showing up in your squid log when you make the request?
> What is showing up in your firewall log when you make the request?
> What is showing up in the Apache log when you make the request?
>
> Post some of these results when asking for help. The answer usually is in
> the logs.
>
> Ron
>
> Adam_at_Gmail wrote:
>> Hi Al,
>> thanks for your reply, I don't acutally have a problem with the apache
>> because the webserver is on another machine as the backend server
>> switching off the apache running on the proxy machine doesn't bother me
>> what I am having a problem with is that it doesn't pull the website from
>> the backend server
>> and right now it won't even allow me access from the local network
>> I have commented out all of the deny accesses and yet it still won't
>> allow any machine on my local network to access the internet.
>
>
>
> You can do both with Apache but the configurations and problems are very
> different.
>
> What exactly are you trying to do?
> Try to get one working first and then go after the other.
>
>> That's what I found very strange.
>> My proxy server runs freely on a dedicated machine nothing else runs on
>> that machine.
>>
>> Regards
>> Adam
>> ----- Original Message ----- From: "Al - Image Hosting Services"
>> <azick_at_zickswebventures.com>
>> To: "Adam_at_Gmail" <adbasque_at_googlemail.com>
>> Cc: <squid-users_at_squid-cache.org>
>> Sent: Friday, March 26, 2010 1:24 AM
>> Subject: Re: [squid-users] Help with accelerated site
>>
>>
>>> Hi,
>>>
>>> Although you can't have apache and squid listening on port 80 on the
>>> same IP, you can have them both running on port 80 on the same machine.
>>> Just do this:
>>>
>>> Change your apache config to:
>>> "Listen 127.0.0.1:80"
>>>
>>> Change your squid config to:
>>> "cache_peer 127.0.0.1 parent 80 0 no-query originserver" "http_port
>>> 1.2.3.4:80 accel vhost"
>>>
>>> Where 1.2.3.4 is, put your public IP.
>>>
>>> -Al
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, 25 Mar 2010, Adam_at_Gmail wrote:
>>>
>>>> Date: Thu, 25 Mar 2010 16:30:33 -0000
>>>> From: "Adam_at_Gmail" <adbasque_at_googlemail.com>
>>>> To: Ron Wheeler <rwheeler_at_artifact-software.com>
>>>> Cc: Amos Jeffries <squid3_at_treenet.co.nz>, squid-users_at_squid-cache.org
>>>> Subject: Re: [squid-users] Help with accelerated site
>>>>
>>>> Hi All,
>>>> Thank you guys for your help
>>>> I have tried your suggestions,
>>>> Yes Ron I know that two programmes can't both listen on the same port
>>>> at the same time
>>>> but I thought the Apache was essential for the Proxy server, so thanks
>>>> for the suggestion,
>>>> I am including bits of my config here, because now I am getting "Access
>>>> Denied" even from a local network:
>>>> Can you guys please take a look at it and see if you can spot what's
>>>> causing the access denied.
>>>> note I have tried to allow everything and removed all the "deny"
>>>> directives and yet it's still denies any access from my local network.
>>>> That is why I get so confused with Squid, I don't understand it's logic
>>>> to be perfectly honest, and let me remind you that this config used to
>>>> work just fine at least it used to allow access to the internet to all
>>>> the clients on my local network.
>>>>
>>>>
>>>> #############################
>>>> # Other Access Controls
>>>> #############################
>>>> acl manager proto cache_object
>>>> acl localhost src 127.0.0.1/32
>>>> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
>>>> acl our_networks dst 192.168.1.0/32
>>>> acl our_sites dstdomain www.mysite.org
>>>> acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
>>>> acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
>>>> # acl localnet src 192.168.0.0/32 # RFC1918 possible internal network
>>>> acl localnet src 192.168.1.0/32 #Local Network
>>>> acl myaccelport port 80
>>>>
>>>> # acl FTP proto FTP
>>>> acl SSL_ports port 443
>>>> acl Safe_ports port 80 # http
>>>> acl Safe_ports port 21 # ftp
>>>> acl Safe_ports port 443 # https
>>>> acl Safe_ports port 70 # gopher
>>>> acl Safe_ports port 210 # wais
>>>> acl Safe_ports port 1025-65535 # unregistered ports
>>>> acl Safe_ports port 280 # http-mgmt
>>>> acl Safe_ports port 488 # gss-http
>>>> acl Safe_ports port 591 # filemaker
>>>> acl Safe_ports port 777 # multiling http
>>>> acl CONNECT method CONNECT
>>>>
>>>> http_access allow manager localhost
>>>> #http_access deny manager
>>>> # http_access deny !Safe_ports
>>>> http_access allow localnet
>>>> #http_access deny all
>>>> # http_access allow intranet
>>>> # http_access deny all
>>>> http_access allow our_networks
>>>>
>>>> icp_access allow localnet
>>>> #icp_access deny all
>>>> htcp_access allow localnet
>>>> #htcp_access deny all
>>>> http_acceess allow CONNECT
>>>> #http_access deny all
>>>> hosts_file /etc/hosts
>>>> visible_hostname proxy
>>>>
>>>> http_port 3128
>>>>
>>>> hierarchy_stoplist cgi-bin ?
>>>>
>>>> cache_effective_user squid
>>>> access_log /usr/local/squid/var/logs/access.log squid
>>>> cache_log /usr/local/squid/var/logs/cache.log
>>>> cache_store_log /usr/local/squid/var/logs/store.log
>>>> pid_filename /usr/local/squid/var/logs/squid.pid
>>>>
>>>> refresh_pattern ^ftp: 1440 20% 10080
>>>> refresh_pattern ^gopher: 1440 0% 1440
>>>> refresh_pattern . 0 20% 4320
>>>>
>>>> icp_port 3130
>>>> htcp_port 4827
>>>> # allow_underscore on
>>>>
>>>> coredump_dir /usr/local/squid/var/cache
>>>>
>>>>
>>>> Can anyone see what's wrong with this config and if possible to point
>>>> it out to me, your help would be much appreciated
>>>>
>>>> Thanking you in advance
>>>> Regards
>>>> Adam
>>>>
>>>> ----- Original Message ----- From: "Ron Wheeler"
>>>> <rwheeler_at_artifact-software.com>
>>>> To: "Adam_at_Gmail" <adbasque_at_googlemail.com>
>>>> Cc: "Amos Jeffries" <squid3_at_treenet.co.nz>;
>>>> <squid-users_at_squid-cache.org>
>>>> Sent: Thursday, March 25, 2010 1:58 AM
>>>> Subject: Re: [squid-users] Help with accelerated site
>>>>
>>>>
>>>>> Adam_at_Gmail wrote:
>>>>>> Hello there,
>>>>>> Thanks for the reply Ron and Amos
>>>>>>
>>>>>>
>>>>>> Maybe my original e-mail wasn't clear a bit confusing I am sorry if I
>>>>>> confused you
>>>>>>
>>>>>> I have squid running on Machine A with let's say local ip 192.168.1.4
>>>>>> the backend server is running on machine B and ip address 192.168.1.3
>>>>>>
>>>>>> Now, instead of getting the website that is located on Machine B
>>>>>> 192.168.1.3 which is listening on port 81 not 80.
>>>>>> I am getting the default Apache Page on the Proxy server Machine
>>>>>> which is 192.168.1.4
>>>>>>
>>>>>> And I do have the vhost in my configuration
>>>>>> Well there are two apaches running on the two machines, the proxy
>>>>>> machine and the web-server machine, except the web-server apache
>>>>>> listens on port 81, logically (technically) speaking it should work,
>>>>>> but for some reason it doesn't.
>>>>>> I hope it makes more sense to you what I am trying to describe here
>>>>>
>>>>> Very helpful.
>>>>> You can not have apache listening for port 80 on 192.168.1.4 and Squid
>>>>> trying to do the same thing.
>>>>> Only one process can have port 80.
>>>>> You will very likely find a note in the squid logs that says something
>>>>> to the effect that squid can not bind to port 80.
>>>>> If you shutdown apache on 192.168.1.4 and restart squid, your proxy
>>>>> will work (if the rest of the configuration is correct)
>>>>> If you then try to start apache on 192.168.1.4 it will certainly
>>>>> complain loudly about port 80 not being free.
>>>>>
>>>>> If you want to use Apache on both 192.168.1.4 and 192.168.1.3 you need
>>>>> to set the apache on 192.168.1.4 to listen on port 81 and set squid to
>>>>> proxy to the apache on 192.168.1.4 and use apache's proxy and vhost
>>>>> features to reach 192.168.1.5 which can be set to listen on port 80.
>>>>> This will support
>>>>> browser=>Squid on 192.168.1.4 ==> Apache on 192.168.1.4:81 (vhost)
>>>>> ==>Apache 192.168.1.3:80
>>>>> That is a pretty common approach.
>>>>>
>>>>> Ron
>>>>>
>>>>>
>>>>>>
>>>>>> Thank you all for your help
>>>>>> Regards
>>>>>> Adam
>>>>>>
>>>>>> ----- Original Message ----- From: "Amos Jeffries"
>>>>>> <squid3_at_treenet.co.nz>
>>>>>> To: <squid-users_at_squid-cache.org>
>>>>>> Sent: Thursday, March 25, 2010 1:01 AM
>>>>>> Subject: Re: [squid-users] Help with accelerated site
>>>>>>
>>>>>>
>>>>>>> On Wed, 24 Mar 2010 19:48:27 -0400, Ron Wheeler
>>>>>>> <rwheeler_at_artifact-software.com> wrote:
>>>>>>>> What is squid proxying?
>>>>>>>> Usually the normal behaviour is exactly what you are getting since
>>>>>>>> squid
>>>>>>>
>>>>>>>> normally proxies Apache on 80.
>>>>>>>> Browser ==> Squid on 80==>proxied to Apache on port 81.
>>>>>>>>
>>>>>>>>
>>>>>>>> If Squid is not proxying Apache, then it looks like you have Apache
>>>>>>>> running on 80.
>>>>>>>>
>>>>>>>> If you are trying to redirect port 80 to another program that is
>>>>>>>> not
>>>>>>>> Apache, then you need to get Apache off port 80.
>>>>>>>> You can not have 2 programs listening to port 80.
>>>>>>>>
>>>>>>>> If Apache is running and owns port 80, Squid will not start.
>>>>>>>>
>>>>>>>> If this is the case, You likely have errors in the logs to this
>>>>>>>> effect.
>>>>>>>>
>>>>>>>> Shut down Apache and and restart Squid.
>>>>>>>>
>>>>>>>> Try to start Apache and now it should howl with anger (or log in
>>>>>>>> anger)
>>>>>>>> at not getting port 80.
>>>>>>>>
>>>>>>>>
>>>>>>>> Ron
>>>>>>>>
>>>>>>>> Adam_at_Gmail wrote:
>>>>>>>>> Hello All,
>>>>>>>>>
>>>>>>>>> I have followed this configuration, but when I try and access the
>>>>>>>>> website from outside my network
>>>>>>>>> All I get is the default page of the apache on the machine where
>>>>>>>>> the
>>>>>>>>> Squid proxy is installed
>>>>>>>>>
>>>>>>>>> Here is the link:
>>>>>>>>>
>>>>>>>>> http://wiki.squid-cache.org/ConfigExamples/Reverse/BasicAccelerator
>>>>>>>>>
>>>>>>>>> here is the configuration I followed
>>>>>>>>>
>>>>>>>>> http_port 80 accel defaultsite=your.main.website.name(changed my
>>>>>>>>> port
>>>>>>>>> to 81 my backend server listens on port 81)I havehttp_port 81
>>>>>>>>> accel
>>>>>>>>> defaultsite=www.my.website.org vhostand then used thiscache_peer
>>>>>>>>> ip.of.webserver parent 80 0 no-query originserver
>>>>>>>>> name=myAccelcache_peer 192.168.1.5 parent 81 0 no query
>>>>>>>>> originserver
>>>>>>>>> name=myAccel(myAccel I have put a name)and then acl our_sites
>>>>>>>>> dstdomain my.website.org
>>>>>>>>> http_access allow our_sites
>>>>>>>>> cache_peer_access myAccel allow our_sites
>>>>>>>>> cache_peer_access myAccel deny all Anybody with any suggestions
>>>>>>>>> please?Any help would be appreciated thank youRegardsAdam
>>>>>>>>>
>>>>>>>
>>>>>>> Sorry, took me a while to un-mangle that original email text.
>>>>>>>
>>>>>>> You are missing the "vhost" option on https_port 80. All traffic
>>>>>>> Squid
>>>>>>> receives on port 80 will go to Apache's default virtual host.
>>>>>>>
>>>>>>> Amos
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
>
Received on Fri Mar 26 2010 - 19:18:24 MDT