Re: [squid-users] http_pipelining prefetch ..

On 24/03/11 00:01, Víctor José Hernández Gómez wrote:
> Dear squid-users,
>
> I have been reading about pipelining
> (http://en.wikipedia.org/wiki/HTTP_pipelining), and we are thinking to
> change the pipeline_prefetch directive to "on" in our Squid-based
> installation, but have found some notes which prevent us to try it
>
> Notes: (Now defaults to off for bandwidth management and access logging
> reasons)
>
> Can someone go into a more detailed explanation on this? Is it really
> worth to try?

Not that much worth in it. The consensus seems to be that it better to
spread the load over multiple parallel connections than wait on a
sequential feed.

I'm not sure what those access logging reasons were, the bandwidth ones
are a design flaw in the pipelining model itself.

In HTTP/1.0 (ie all Squid releases 3.1 and older) the required default
action is to close the connection after completing one request, unless
explicitly asked not to.
Also a large portion of sites are dynamic and send out page objects of
unknown length. The *only* way to signal completion of these objects in
HTTP/1.0 is for the server to terminate the connection to Squid, and
Squid to relay that termination on to the client.

In the pipeline N requests are in transit on a single connection before
the first result comes back.
  Problem the first: N requests require M about of bandwidth to receive.
  Problem the second: they also require M amount of RAM to hold.
  Problem the third: What happens to N-1 requests, M-1 bandwidth and M-1
memory after the first request closes the client connection? *poof* gone.
  Problem the fifth: pipeline client usually "recovers" by re-scheduling
N-1 requests. loop back to problem 1.

*major* waste of resources whenever the client pipelines to a dynamic
website.

That directive you see allows us to benefit a little be pre-parsing the
second request before the first has finished. Squid caps pipeline at 2,
so at most one requests is wasted. Squid does *not* pipeline to servers,
it spreads the load over multiple parallel connections instead.

Even if pipelining is turned off in Squid the client software is free to
schedule N requests in the TCP link buffers. So in that respect Squid
already supports pipelining without any effort.

In my experience client software has either attempted and badly failed
to do pipelining properly (ie error recovery is critical), or has done
it so well nobody noticed.

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE9 or 3.1.11
   Beta testers wanted for 3.2.0.5