Re: multi-streamed http transfers

Hi,
I understand now - one has to bear in mind the case where the amount of
requested data of a similar size to the cache size. So as you said, the
better solution is to use a multi-streamed client and have squid deal
with the partial file objects. It would also require one of these
clients to support http proxy use.
This seems like a widely useful feature - what would it take to get this
on the squid devel wish list? Meanwhile I`m going to bother the aget author.

Cheers,
Rod.

Henrik Nordstrom wrote:

>
>
> On Fri, 17 Dec 2004, Rod Walker wrote:
>
>> Hi,
>> Of course I don`t know the inner workings of Squid but I thought the
>> problem would be that the file chunks would always be different
>> depending
>> on how many streams the user chose. So even if Squid cached partial file
>> objects, it's unlikely the client would ask for exactly those blocks.
>> For
>> example, a client downloads a file with 2 streams and then another
>> client
>> with 3
>> 2 streams: 1-500 & 501-1000 would result in 2 objects cached.
>> 3 streams: 1-333,334-666,667-1000 - would these requests give cache
>> hits?
>
>
> Yes, if the cache is implemented correctly.
>
>
>> As for the squid-squid, squid-server multi-stream ...
>> If the requested object will not be cached because it`s larger than the
>> max cachable size, then I can see the problems you mention. But if it
>> is to be cached then it could be written straight to disk, and read back
>> in a single stream to the client.
>
>
> True, but how fast should this be done? the fetching of the parts
> which we can not yet send to the client. And how do we guarantee we do
> not run out of disk space? Concider the case there may be several of
> these requests in parallell for different resources.
>
>> I guess you normally keep the objects in
>> memory though. I guess my conceptual problem is that I`m talking about
>> objects around 1GB, and you objects more like 1MB.
>
>
> No, I am talking about large objects. For small objects there is no
> noticeable benefit for multi-streamed transfers as the majority of the
> overhead is the session setup.
>
>> I think I can protoype the system with single streams, in the knowledge
>> that multi-stream is at least possible.
>> Thanks a lot for your help.
>
>
>
>
> Regards
> Henrik
>
>

-- 
Rod Walker +1 6042913051