Re: adding content to the cache

I am looking into adding objects at runtime, on a regular basis, probably 2 or 3 every minute or so. Those objects are stored on the machine running Squid and I also have the HTTP headers for those objects. I just want to support one storage scheme (the one by default I guess).

Looking at your answer, you are saying that adding objects at runtime is quite complicated due to the way Squid keeps that information internally, am I correct ?

I started looking at store*.c files to see if I could modify the store at runtime and add those objects. Any approach would you recommend if I wanted to go down this road.

Thanks.

Laurent

----- Original Message ----
From: Alex Rousskov <rousskov_at_measurement-factory.com>
To: Laurent Luce <laurentluce49_at_yahoo.com>
Cc: squid-dev_at_squid-cache.org
Sent: Sunday, May 10, 2009 12:01:20 PM
Subject: Re: adding content to the cache

On 05/09/2009 08:04 PM, Laurent Luce wrote:

> Actually, I am looking at a way of adding it directly to the squid
> cache. Basically, take the file and add it to the cache. I am looking
> into patching Squid to provide an API to do that. How complicated do
> you think it is if I want to add the file content along with the
> metadata directly into the cache ?

Do you want to add objects runtime or offline? How many objects do you
need to add (ballpark estimates: hundreds, thousands, millions)? How
often do you need to add them (once, daily, weekly, etc.)? Are those
objects stored on the machine running Squid? How are those objects
stored now? Do stored objects come with HTTP response headers?

If you want to add objects while Squid is running and want them to
become available as they are being addeded, fetching those objects using
wget/curl may be the best short-term solution (which can be optimized
and tuned using special headers and a local script pretending to be an
origin server).

If you want to add objects offline, do you want to support multiple
Squid storage schemes (e.g., ufs, COSS, RockStore, etc.)? The best
implementation will probably depend on that and on the answers to the
questions above.

Thank you,

Alex.

> ----- Original Message ----
> From: Amos Jeffries <squid3_at_treenet.co.nz>
> To: Laurent Luce <laurentluce49_at_yahoo.com>
> Cc: squid-dev_at_squid-cache.org
> Sent: Friday, May 8, 2009 1:42:53 AM
> Subject: Re: adding content to the cache
>
> Laurent Luce wrote:
>> I am looking for a way to manually add content to the cache. Is there an API to do that ?
>>
>> For
>> example, I have the following file image.gif and I want to add it to
>> the proxy cache so it can be served from there when needed.
>>
>> Laurent
>>
>
> Not at present.
> You need to place it somewhere on a web server, assign it sufficient cache-control settings to store for a long while and then request it from squid.
>
> Common tools such as wget or squidclient can do that for you.
>
> Amos
> -- Please be using
> Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15
> Current Beta Squid 3.1.0.7
Received on Sun May 10 2009 - 20:08:18 MDT