Re: [MERGE] Support large response headers

On Sun, Apr 06, 2008, Henrik Nordstrom wrote:
> Resubmissions as I haven't received any comments on the previous merge
> request.
>
> This patch fixes Bug #2001 by reworking how the client_side_*.cc code
> deals with response headers. Instead of parsing the headers again it
> clones the already parsed header and blindly skips the unparsed headers
> in the data stream.
>
> 3.0 version submitted separately.

As discussed on IRC, I'm not sure where in 2.5/2.6 the response status/headers
are allowed to grow past 4k _AND_ be parsed.

In 2.6, clientSendMoreHeaderData():

    assert(http->out.offset == 0);
    rep = http->reply = clientBuildReply(http, buf, size);
    if (!rep) {
        /* Forward as HTTP/0.9 body with no reply */
        MemBuf mb;
        memBufDefInit(&mb);
        memBufAppend(&mb, buf, size);
        http->out.offset += body_size;
        comm_write_mbuf(http->conn->fd, mb, clientWriteComplete, http);
        return;
    }

The above code turns a "too large" reply into a HTTP/0.9 response, which
means Squid just pipes it straight to the client without any further processing.
It doesn't do any further reply processing on the reply - it just starts
feeding data to the client. The client "happens" to figure out that there
are headers. :)

I haven't checked 2.5, and its possible that my axe removal of extra
headersEnd() calls removed some magic check that didn't allow the above code
to run until the full reply was available, -but- I don't recall any spot
in the earlier code that passed storeClientCopy() a buffer greater than 4k.

This is going to take a bit of effort to fix properly in 2.HEAD/2.7 and 3.X..

Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -