[squid-users] ecap adapted body limited to 64k

It appears that squid limits the adapted body size of any ecap adapter to
65535, perhaps due to BodyPipe::MaxCapacity = 64*1024. I am certain of the
limitation, but not 100% certain BodyPipe::MaxCapacity is the cause.

My adapter properly receives all chunks of data via noteVbContentAvailable
and assembles one larger internal buffer. The first time that abContent()
is called in my adapter, it is called with abContent(offset=0,
size=4294967295). My test case (www.google.com) has a body size of 100043
bytes, so I pass it all back at once.

My adapter next sees a call to abContentShift(size=65535) and then no
additional calls to any ab* function in my adapter. I have set
debug_options to ALL,9 and been looking for relevant lines. Below are a few
lines that seem appropriate. Note that the lines starting with
"Adapter::Xaction(RESPMOD,...)" are from my adapter, showing the calls to
abContent

2012/11/23 15:01:54.986|
Adapter::Xaction(RESPMOD,0xa4ff090)::abContent(offset=0, size=4294967295)
2012/11/23 15:01:54.986| XactionRep.cc(648) moveAbContent: up to 100043
bytes
2012/11/23 15:01:54.987|
Adapter::Xaction(RESPMOD,0xa4ff090)::abContentShift(size=65535)

Note that XactionRep.cc shows that it has received the full 100043 bytes of
the adapted body from my adapter.

In looking at adaptation/ecap/XactionRep.cc, I see it in turn calls
BodyPipe::putMoreData(). In looking at the definition of BodyPipe, I see
enum { MaxCapacity = 64*1024 }. That would seem to potentially be the
source of the size limitation; ***HOWEVER*** I believe that the proper
behavior is that abContent should be called repeatedly until all of the body
has been transferred. So even with the MaxCapacity limitation, I believe
squid should be repeatedly or recursively calling abContent and
abContentShift in my adapter until it has retrieved the entire adapted body.

In XactionRep.cc, Adaptation::Ecap::XactionRep::moveAbContent(), I see that
libecap::nsize is being passed as the size parameter to abContent().
Perhaps it should instead pass a maximum value of the
BodyPipe::MaxCapacity??? At a minimum, the code should continue to call
abContent until it has acquired all of the adapted body. Perhaps the
function needs to check to see if "const size_t used" is less than c.size
and then "putMoreData()" ???

As far as I can tell, this seems like an bug that limits all adapters to a
maximum adapted body size of 64k.

--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/ecap-adapted-body-limited-to-64k-tp4657413.html
Sent from the Squid - Users mailing list archive at Nabble.com.