Re: Suggestion

--MimeMultipartBoundary
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

I've had a look at the file-access module, and am less than thrilled. (I'=
ve
been staring at 1.1.18, for reference. I don't want to add anything to 1.=
2
until it's settled.)

The asynchronous I/O model is jim-dandy, unless you want to cram a little
extra data in there (into a cache object). Also, it's unclear as to wheth=
er
there are size-checks done in object validation that might be skewed by t=
he
actual stored object being larger than it is claimed to be. Nevertheless,=
 I
don't feel comfortable going in and messing with file-I/O. disk.c is pret=
ty
opaque after yesterday's effort of looking over it, and looks unsuited to
what I had in mind. (There was something else I was going to say here, bu=
t
it's flown right out of my head. Had a particularly poor night's sleep, d=
ue
to local wildlife)

Writing an extra file per cache-object works (and only took about 9 lines
of extra code, two of which are in unlinkd.c and the remainder down in
store.c) but it burns an extra inode per cache-object, and that gets
expensive _really_ quickly. It works fine on my test-box, and doesn't see=
m
to add any significant performance degradation, but I can't run performan=
ce
testing at the sort of scale where that becomes noticeable. Not today,
anyway.

An unordered list of alternatives that come to mind:

* Write a single meta-data file per cache-directory, containing the
information that would normally go in the swap log. That then burns a
minimum one inode per cache-directory, but there would be a trade-off in
manipulating that file. Unlinkd would have to get smarter, for one thing.
Still, this is a low-energy solution, and would be straightforward enough
to #ifdef out if not wanted. A perl script could then be used to rebuild
the swap-log (or it could be built into the code for startup).

* Turn the swap_log entry into a _REPLY_ header during reply-header
processing (or post-processing, pre-writing), where it can be neatly stor=
ed
in the company of the remainder of the object data, and there's no fighti=
ng
with disk.c when you're outputting the object to another cache or client.
Ideally, you would want to filter that header-line out before delivery is
the only trick. I don't have enough data to take a stab at the performanc=
e
of a late-bound rewriting procedure.

* Either of these schemes could be indulged in at shutdown time, with the
cache distributing relevant swap-log data to the cache-objects at shutdow=
n.
Ugly. Slow. But could also be done as an offline script. (Uglier and
slower).

D

Henrik Nordstrom wrote:

> Dancer wrote:
>
> > I've just patched store.c and unlinkd.c to handle storing
> > the information as extra data on my test-box. I've not
> > looked deeply enough at the file/disk model to confidently
> > prepend the necessary data into the cache-file itself. I
> > may end up doing that before the day is out.
>
> Good work. Please report to squid-dev@nlanr.net on how far you get, and
> if you find any problems on the way.
>
> Are you doing this on 1.1.X or 1.2?
>
> ---
> Henrik Nordstr=F6m
> Sparetime Squid Developer

--
Note to evil sorcerers and mad scientists: don't ever, ever summon powerf=
ul
demons or rip holes in the fabric of space and time. It's never a good
idea.
ICQ UIN: 3225440
--MimeMultipartBoundary--