Re: [squid-users] Questions about SSL logging

On 12.09.2012 01:24, David Touzeau wrote:
> Dear Amos
>
> Have no such acl in my conf:
> So by understanding your last answer, HTTPS requests must be logged
>

Provided it actually goes through the proxy. Yes.

> Here it is my settings
>
> # IS 3.2 YES
> # IS 3.1 YES
> acl localhost src 127.0.0.1/32
> acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
> acl squidclient proto cache_object
> #--------- LDAP AUTH settings
> #Authentification mode, building using squid compiled for
> 127.0.0.1:389
> auth_param basic program /lib/squid3/basic_ldap_auth -b
> "dc=my-domain,dc=com" -D "cn=Manager,dc=my-domain,dc=com" -w "secret"
> -f "(&(objectClass=userAccount)(uid=%s))" -v 3 -h 127.0.0.1 -p 389
> #--------- GLOBAL
> external_acl_type ldap_group %LOGIN /lib/squid3/ext_ldap_group_acl -D
> "cn=Manager,dc=my-domain,dc=com" -w "secret" -b "dc=my-domain,dc=com"
> -f "(&(objectClass=posixGroup)(gidNumber=%a)(memberUid=%v))" -S -v 3
> -h 127.0.0.1 -p 389
> auth_param basic children 5
> auth_param basic credentialsttl 2 hour
> auth_param basic realm Squid proxy-caching web server
> authenticate_ttl 1 hour
> authenticate_ip_ttl 60 seconds
> acl ldapauth proxy_auth REQUIRED
> #--------- TWEEKS PERFORMANCES
> # http://blog.last.fm/2007/08/30/squid-optimization-guide
> memory_pools off
> quick_abort_min 0 KB
> quick_abort_max 0 KB
> log_icp_queries off
> client_db off
> buffered_logs on
> half_closed_clients off
> #--------- UfdbGuard
> #Disabled enable_UfdbGuard=0
> #--------- squidGuard
> #Disabled enable_squidguard= 0
> url_rewrite_bypass off
> #--------- SQUID PARENTS (feature not enabled)
> #--------- acls
> acl blockedsites url_regex "/etc/squid3/squid-block.acl"
> acl CONNECT method CONNECT
> acl purge method PURGE
> acl FTP proto FTP
> acl office_network src all
> acl group_password external ldap_group
> #--------- GROUPS definition
> #no groups
> #--------- MAIN RULES...
> always_direct allow FTP
> # --------- SAFE ports
> acl Safe_ports port 80 #http
> acl Safe_ports port 22 #ssh
> acl Safe_ports port 443 563 #https, snews
> acl Safe_ports port 1863 #msn
> acl Safe_ports port 70 #gopher
> acl Safe_ports port 210 #wais
> acl Safe_ports port 1025-65535 #unregistered ports
> acl Safe_ports port 280 #http-mgmt
> acl Safe_ports port 488 #gss-http
> acl Safe_ports port 591 #filemaker
> acl Safe_ports port 777 #multiling http
> acl Safe_ports port 631 #cups
> acl Safe_ports port 873 #rsync
> acl Safe_ports port 901 #SWAT
> acl Safe_ports port 20 #ftp-data
> acl Safe_ports port 21 #ftp#
> acl SSL_ports port 9000 #Artica
> acl SSL_ports port 443 #HTTPS
> acl SSL_ports port 563 #https, snews
> acl SSL_ports port 6667 #tchat
> # --------- Change HTTP headers:
> # --------- 0 active entry
> # --------- Use x-forwarded-for for load balancers
> follow_x_forwarded_for allow localhost
> acl_uses_indirect_client on
> delay_pool_uses_indirect_client on
> log_uses_indirect_client on
> acl whitelisted_mac_computers arp
> "/etc/squid3/whitelisted-computers-by-mac.acl
>
> # --------- RULES DEFINITIONS
> http_access allow purge localhost
> http_access allow whitelisted_mac_computers
> url_rewrite_access deny whitelisted_mac_computers
> http_access allow squidclient manager
> http_access allow to_localhost
> url_rewrite_access deny localhost
> url_rewrite_access deny squidclient
> url_rewrite_access allow all
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost
> http_access allow purge localhost
> http_access deny purge
> http_access deny blockedsites
> http_access allow ldapauth
> http_access allow group_password
> http_access allow office_network
> http_access deny all
> # --------- ICAP Services.(0 service(s))
>
>
> # --------- eCAP Services
> # --------- ident_lookup_access
> hierarchy_stoplist cgi-bin ?
> # --------- General settings
> visible_hostname proxyweb
> # --------- time-out
> dead_peer_timeout 10 seconds
> dns_timeout 2 minutes
> connect_timeout 1600 seconds

In squid-3.2 this means up to 27 minutes waiting for *each* TCP SYN
packet attempted to return. You can drop this down to your expected
maximum response time from remote servers. Only after this timeout or an
ICMP error packet will squid try another path. With forward_max_tries at
10 (default) that is up to 4.5 hours before the client gets a failure
page showing up from servers behind an ICMP blackhole network.

> persistent_request_timeout 3 minutes
> pconn_timeout 1600 seconds
> maximum_object_size 600 MB
> minimum_object_size 0 KB
> maximum_object_size_in_memory 1024 KB
> #http/https ports
> http_port 3128
> http_port 3140
> icp_port 3130
> # --------- SSL Rules
> # --------- Caches
> cache_effective_user squid
> #cache_replacement_policy heap LFUDA
> cache_mem 207 MB
> cache_swap_high 90
> cache_swap_low 95
> # --------- DNS and ip caches
> ipcache_size 51200
> ipcache_low 90
> ipcache_high 95
> fqdncache_size 51200
> positive_dns_ttl 72 hours
> negative_dns_ttl 6 seconds
> # Personal settings
> # To add your own tokens, just create a file under
> /etc/squid3/squid-me.conf,
> # it will be merged here
> # --------- SPECIFIC DNS SERVERS
> dns_nameservers 192.168.1.1
> dns_nameservers 192.168.1.1

duplicate line.

> #--------- FTP specific parameters
> ftp_passive on
> ftp_sanitycheck off
> ftp_epsv off
> ftp_epsv_all off
> ftp_telnet_protocol off
> debug_options ALL,1
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern . 0 20% 4320
> refresh_pattern -i (/cg-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
> refresh_pattern -i (/cg-bin/|\?) 0 0% 0

duplicate lines.

> #Logs-------------------------------------------------
> coredump_dir /var/squid/cache
> cache_log /var/log/squid/cache.log
> pid_filename /var/run/squid.pid
> netdb_filename stdio:/var/log/squid/netdb.state
> logformat csv
>
> "%{%Y-%m-%d}tl","%{%H:%M:%S}tl","%>a","%>A","%>eui","%<a","%<A","%[un","%rm","%ru","%rv","%>Hs","%<st","%Ss:%Sh","%{User-Agent}>h","%{X-Forwarded-For}>h"
> access_log stdio:/var/log/squid/access.csv csv !squidclient
> logformat common MAC:%>eui %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv"
> %>Hs %<st %Ss:%Sh UserAgent:"%{User-Agent}>h"
> Forwarded:"%{X-Forwarded-For}>h"
> cache_store_log stdio:/var/log/squid/store.log
> access_log syslog:authpriv.info common !squidclient
> access_log stdio:/var/log/squid/sarg.log squid !squidclient
> #--------- Multiple cpus -- (disabled)
> workers 1
> cache_dir aufs /var/cache/squid 10000 16 256
> # --------- OTHER CACHES
>

Okay. I can't see anything in there that would affect SSL logging
either.

Stetching at a few unlikely possibilities...
  It's possible that the CONNECT to YT is being re-used as a persistent
HTTPS connection by the client(s) and not being logged for a very long
time.
  Or that you have some connectivity issue to YT and they are very
patient clients waiting on that 1600 second connect timeout.
  Or that the clients are using some other protocol (SPDY? WebSockets?
something else?) to connect to YT. This may show up as a very long-life
CONNECT request to 443 (ie tunnelling HTTPS over SPDY over HTTP), or not
at all.

Amos

> -----Original Message----- From: Amos Jeffries
> Sent: Tuesday, September 11, 2012 1:11 AM
> To: squid-users_at_squid-cache.org
> Subject: Re: [squid-users] Questions about SSL logging
>
> On 11.09.2012 10:42, David Touzeau wrote:
>> Dear, i’m using squid 3.2
>>
>> Sometimes the Squid-cache log correctly the SSL connections to web
>> sites
>>
>> Sep 11 00:30:37 kav4proxy squid[8504]: MAC:64:27:37:02:53:3d
>> 192.168.1.158 -
>> dtouzeau [11/Sep/2012:00:30:37 +0200] "CONNECT www.artica.fr:443
>> HTTP/1.1"
>> 200 26051 TCP_MISS:HIER_DIRECT UserAgent:"Mozilla/5.0 (Windows NT
>> 6.1;
>> WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1" Forwarded:"-"
>>
>> Sep 11 00:31:10 kav4proxy squid[8504]: MAC:64:27:37:02:53:3d
>> 192.168.1.158 -
>> dtouzeau [11/Sep/2012:00:31:10 +0200] "CONNECT ssl.gstatic.com:443
>> HTTP/1.1"
>> 200 2582 TCP_MISS:HIER_DIRECT UserAgent:"Mozilla/5.0 (Windows NT
>> 6.1; WOW64;
>> rv:15.0) Gecko/20100101 Firefox/15.0.1" Forwarded:"-"
>>
>> But when i’m browsing to https://www.youtube.com there no entry in
>> squid
>> access.log ??
>> Is there any limitation that ban squid to log https requests..?
>>
>
> Not unless you configured such a ban or SSL-bumped those requests.
>
> log_access - to block a request from being logged anywhere.
>
> access_log <log> [acl acl ...] - to block a request from being logged
> to a specific log.
>
> SSL-bump will log the bumped requests inside the CONNECT tunnel as
> https://* URLs individually, instead of the overview CONNECT (varies
> with squid version whether the CONNECT is *also* logged).
>
> Amos
Received on Wed Sep 12 2012 - 02:11:03 MDT