Re: Store Object length

Quoting earlier responses to your question in case you missed them:

>> On 25/02/2013 8:00 p.m., anita wrote:
>>> I am trying to write an API that will fetch objects from the squid cache
>>> given the url and http method.
>>> When I analyzed the code, I found that this can be achieved with a
>>> storeGetPublic(url, http_method) to see if the url has an entry in the
>>> cache.
>>> This followed by storeClientCopy() call.
>>> However, the StoreIOBuffer used here has been initialized with a
>>> lenght of HTTP_REQBUF_SZ (4096 or 4 k) by default.

> On 02/25/2013 04:45 AM, Amos Jeffries wrote:
>> Err. Yes storeGetPublic() is the API to do this with.
>>
>> Why are you writing a new one? what are you trying to do with Squid?

On 02/25/2013, Alex Rousskov wrote:
> I agree with Amos that these questions are important, for many reasons,
> ranging from compatibility with official code to being able to get
> support on squid-dev.
>
>
> To answer your buffer size question: The API does not restrict what
> buffer sizes the caller should use. Some client side code uses
> HTTP_REQBUF_SZ, but your code may use a different size. The store_client
> subscriber will receive callbacks when more data is available. It is a
> complex, iterative process; not a single "give me the whole response
> data" call.
>
> In theory, even a 1-byte buffer should work, but, besides being too
> slow, I suspect some Store code assumes that HTTP response headers will
> fit into the first buffer given to Store by store_client. Thus, it would
> be unwise to use smallish buffers without fixing Store first.
>
>
> Also, please note that storeGetPublic() returns an entry that is not
> necessarily fully cached. In the extreme case of collapsed forwarding,
> it returns an entry that is only expected to be cached when received
> from the origin server.

>>> I would like to know how larger objects are stored and retrieved from
>>> cache? How can I determine the length of the object read in that case?

> In general, you should not rely on knowing the response size a priori
> because some store modules may allow storage of objects where the size
> is not known when swap out begins (that is a good thing) and do not
> update the recorded size when swap out ends (that would be a performance
> optimization). I doubt Squid fully supports that today, but eventually
> it should, so it is better not to make a priori size knowledge a
> critical precondition in your code (if possible).
>
> Please describe what problem you are trying to solve and why you think
> you need a new API for what is already supported. With some luck,
> squid-dev folks will be able to help you along the way.
>
>
> Thank you,
>
> Alex.

>>> Version used: Squid ver 3.1.16
>>> squid.conf : default configuration file with cache_dir directive
>>> uncommented.
>>>
>>> Any pointers are appreciated.
>>
>> When playing with the Squid code please always upgrade to the most
>> recent you can before starting. Preferrably that would be 3.HEAD which
>> is our rolling development version.
>> 3.1 Squid series has been officially deprecated for about five months
>> now, and even 3.2 is nearing end of life.
>>
>> The Squid-3 series is one very long process of converting the code from
>> C code constructs to C++ objects APIs and pluggable mechanisms. So you
>> may find it much easier to code later releases, or that your fix has
>> already been done.
>>
>> Amos
Received on Tue Feb 26 2013 - 19:04:18 MST