Re: [squid-users] How can I clean out old objects - refresh patterns and really old items

Files aren't deleted when they expire.

Files are deleted when:

* A request occurs and squid checks the file for freshness, or
* Squid issues a validation requests and determines the local copy is stale, or
* Squid needs to make space (as the disk store is full) and starts running
  the object replacement policy to purge objects - but then, it doesn't maintain
  a list of "stale" objects to purge; it just deletes the 'oldest' objects.

Adrian

On Tue, Sep 18, 2007, Nicole wrote:
> Hate to respond to myself, but I wanted to add more info..
>
> In a well duh moment I ran find and found objects going back to July.
> find /cache -type f -mtime +30 -exec ls {} \;
>
> If my headers from my web servers are set to expire in 2 weeks:
> Cache-Control: max-age=1728000
> Connection: close
> Date: Wed, 19 Sep 2007 00:38:51 GMT
> Accept-Ranges: bytes
> ETag: "1155193587"
> Server: lighttpd
> Content-Length: 68424
> Content-Type: image/jpeg
> Expires: Tue, 09 Oct 2007 00:38:51 GMT
> Last-Modified: Mon, 17 Sep 2007 20:03:00 GMT
> Client-Date: Wed, 19 Sep 2007 00:38:51 GMT
> Client-Response-Num: 1
>
> How did my expires make it keep files for so long and why did my new ones not
> start a purge?
>
>
> Thanks
>
>
> Nicole
>
>
> > Hello all
> >
> > I have a few squid servers that seem to have gotten a bit out of control.
> >
> > They are using up all the systems memory and starting to serve items slowly.
> >
> > As near as I can tell, it seems to just want more memory than I have to
> > serve
> > and manage all the objects in the cache.
> >
> > Internal Data Structures:
> > 20466526 StoreEntries
> > 24888 StoreEntries with MemObjects
> > 24870 Hot Object Cache Items
> > 20466434 on-disk objects
> >
> >
> > I have tried reducing my refresh pattern from:
> > refresh_pattern -i \.jpg 10080 150% 40320 ignore-reload
> > to:
> > refresh_pattern -i \.jpg 5040 100% 4320 ignore-reload
> >
> > and doing a reload.
> >
> > However, I have not noticed it expiring out old objects and freeing up disk
> > space like I thought it would.
> >
> > Do objects get stored based on their original refresh pattern? So even if I
> > change it, they won't expire until they expire based on the pattern they were
> > stored with?
> >
> > Is there any way to tell the age of the objects eating up my cache storage
> > space? Any reccomendations on how to reduce my object count besides reducing
> > disk space? This is for a reverse proxy cache and we have the cache
> > header set to expire objects in 2 weeks. I really can't believe that I have
> > 20Million 2 week old objects.
> >
> >
> >
> > Thanks!
> >
> >
> >
> > Nicole
> >
> >
> > --
> > |\ __ /| (`\
> > | o_o |__ ) )
> > // \\
> > - nmh@daemontech.com - Powered by FreeBSD -
> > ------------------------------------------------------
> > "The term "daemons" is a Judeo-Christian pejorative.
> > Such processes will now be known as "spiritual guides"
> > - Politicaly Correct UNIX Page
>
>
> --
> |\ __ /| (`\
> | o_o |__ ) )
> // \\
> - nmh@daemontech.com - Powered by FreeBSD -
> ------------------------------------------------------
> "The term "daemons" is a Judeo-Christian pejorative.
> Such processes will now be known as "spiritual guides"
> - Politicaly Correct UNIX Page
>
>

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level bandwidth-capped VPSes available in WA -