Re: [squid-users] Cache URL with "?"

> On tis, 2008-07-22 at 23:37 +1200, Amos Jeffries wrote:
>
> Only one issue has come up: under sibling relationships, squids method of
> relaying requests to sibling is slightly bad and may break the proper expected
> behavior. All other setups should be fine.
>
>
>> Also seen in parent relations.
>>
>> Regards Henrik

Hi,

What is the nature of the problems with dynamic caching and
sibling/parent relationships? I tried googling for more info but
couldn't find anything.

I'm using 4 squid nodes [2.7STABLE3] as accelerators and encountered
some odd behavior recently after enabling dynamic caching. There are
sibling relationships among the 4 nodes using ICP and cache digests.
Under medium load I've been able to successfully cache and serve
dynamic URLs:

      GET http://contentreal.foobar.com/clown_widget.swf?version=20080820
      STORE_OK IN_MEMORY SWAPOUT_NONE PING_NONE
      CACHABLE,VALIDATED

But under heavier load [20-50mbit per node] I get multiple objects [20-30] like:

      GET http://contentreal.foobar.com/clown_widget.swf?version=20080820
      STORE_PENDING NOT_IN_MEMORY SWAPOUT_NONE PING_DONE
      RELEASE_REQUEST,DISPATCHED,PRIVATE,VALIDATED

      GET http://contentreal.foobar.com/clown_widget.swf?version=20080820
      STORE_OK NOT_IN_MEMORY SWAPOUT_NONE PING_DONE
      RELEASE_REQUEST,DISPATCHED,PRIVATE,VALIDATED,ABORTED

and requests for this URL timeout after about 25-50% of the download is done.

Headers are being set via apache/mod_expires:

  [root_at_mongo ~]# curl -I
http://content5.foobar.com/clown_widget.swf?version=20080820
  HTTP/1.0 200 OK
  Date: Thu, 21 Aug 2008 09:53:50 GMT
  Server: Apache
  Last-Modified: Fri, 08 Aug 2008 06:07:10 GMT
  ETag: "35b0f"
  Accept-Ranges: bytes
  Content-Length: 219919
  Cache-Control: max-age=86400
  Expires: Fri, 22 Aug 2008 09:53:50 GMT
  Content-Type: application/x-shockwave-flash
  X-Cache: MISS from squid03.foobar.com
  X-Cache-Lookup: MISS from squid03.foobar.com:80
  Via: 1.1 squid03.foobar.com:80 (squid)
  Connection: close

I followed http://wiki.squid-cache.org/ConfigExamples/DynamicContent,
acl QUERY is disabled and the swf is < maximum_object_size_in_memory.
Any assistance would be greatly appreciated.

Thanks to all who maintain squid, this is great and extremely useful software.

murray

##### squid.conf #####
http_port 80 defaultsite=contentreal.foobar.com
cache_peer 110.0.2.99 parent 80 0 no-query no-digest originserver
cache_peer 110.0.2.76 sibling 80 3130
cache_peer 110.0.2.77 sibling 80 3130 proxy-only
cache_peer 110.0.2.78 sibling 80 3130 proxy-only
digest_generation on
#hierarchy_stoplist cgi-bin ?
#acl QUERY urlpath_regex cgi-bin \?
#cache deny QUERY
shutdown_lifetime 10 seconds
strip_query_terms off
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
cache_mem 5200 MB
maximum_object_size_in_memory 500 KB
ipcache_size 4096
fqdncache_size 4096
cache_dir null /tmp
logformat squid %tl.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
access_log /usr/local/squid/var/logs/access.log squid
negative_ttl 30 seconds
cache_access_log /dev/null
cache_store_log /dev/null
buffered_logs on
client_db off
refresh_pattern html$ 0 20% 1
refresh_pattern txt$ 0 20% 1
refresh_pattern xml$ 0 20% 5
refresh_pattern swf$ 0 20% 5
refresh_pattern flv$ 0 20% 10
refresh_pattern . 0 20% 4320
refresh_pattern (cgi-bin|\?) 0 0% 0
#refresh_pattern ^ftp: 1440 20% 10080
#refresh_pattern ^gopher: 1440 0% 1440
acl N1 src 110.0.2.76
acl N2 src 110.0.2.77
acl N3 src 110.0.2.78
icp_access allow N1
icp_access allow N2
icp_access allow N3
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl CONNECT method CONNECT
acl purge method PURGE
http_access allow purge localhost
http_access deny purge
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow all
http_reply_access allow all
cache_mgr webmaster_at_foobar.com
cache_effective_user squidmaster
httpd_suppress_version_string on
coredump_dir /usr/local/squid/var/cache

##### config.log #####
./configure --enable-storeio=null,aufs,diskd,ufs --enable-cache-digests
Received on Thu Aug 21 2008 - 19:22:56 MDT