Squid-2/Lusca async io shortcomings..

Hi all,

I've been braindumping my thoughts into the Lusca blog during some
experimental development to eliminate the data copy in the disk store
read path. This shows up as the number 1 CPU abuser in my test CDN
deployment - where I see a 99% hit rate on a set of large objects (>
16meg.)

My first idea was to avoid having to paper over the storage code
shortcomings with refcounted buffers, and modify various bits of code
to keep the store supplied read buffer around until the completion of
said read IO. This mirrors the requirements for various other
underlying async io implementations such as posix AIO and windows
completion IO.

Unfortunately the store layer and the async IO code doesn't handle
event cancellation right (ie, you can't do it) but the temporary read
buffer in async_io.c + the callback data pointer check papers over
that. Store reads and writes may be scheduled and in flight when some
other part of code calls storeClose() and nothing really tries to wait
around for the read IO to complete.

So either the store layer needs to be made slightly more sane (which I
may attempt later), or the whole mess can stay a mess and be papered
over by abusing refcounted buffers all the way down to the IO layer.

Anyway, I know there are other developers out there working on
filesystem code for Squid-3 and I'm reasonably certain (read: at last
check a few months ago) the store layer and IO layers are just as
grimey - so hopefully my braindumping will save some more of you a
whole lot of headache. :)

Adrian
Received on Fri Apr 10 2009 - 16:03:18 MDT