squid-2.HEAD / Cacheboy code reorganisation: almost complete

This is aimed more as a "how I think things should be done" post as
this stuff isn't yet in Squid-2.HEAD. Still, I think its worth
pointing out.

I've almost finished my first round of code reorganisation along the lines
I suggested on squid-dev a few months ago (and some people disagreed
with.) I think this is the sort of house-cleaning that should have happened
years ago.

I've created two new libaries thus far (which could probably do with
further separation later to enforce finer-grained code boundaries) :

libcore:
  context debugging
  debugging
  dlink type
  fifo type
  gb_t
  kb_t
  syslog-related macros
  varargs related declarations
  valgrind related declarations
  misc stuff (safe_free, toMB/toKB, etc)

libmem:
  intlist
  wordlist
  MemBufs - the MEM_*K_BUF related stuff
  MemPool
  MemStr - the string buffer related stuff
  String

Plenty of the above included Squid-related statistics (ie, cachemgr) and those
bits will stay inside src/.

The legacy and not-so-legacy MemPool types which were included in memData
(ACLs, storeentry, memobject, etc) have been replaced with local mempools
in the "new" way (ie, an init() function creates the mempool.)

These have been moved since they're generally seperate from the "web cache"
bit of Squid itself and have well-defined dependencies. The only circular
dependency in the above is ctx <-> debug ; everything else should follow
a non-cyclic graph. This is good for a number of reasons, none of which
I'll go into here.

It uncovered a couple of potentially silly conditions with the fifo linked list
implementation - tail insertion was implemented as an O(n) operation.
I implemented a simple fifo type and converted a couple of instances with that
(helper queues and AUFS pending ops-before-file-opens come to mind.)
Neither of which ever showed up on CPU profiling but implementing a FIFO with
anything other than an O(1) operation is plain silly.

I'm going to shuffle MemBuf's and cbdata out of src/ in this pass.
I have to do something similar to that for the dozen or so cbdata types
that are defined in src/cbdata.c before I can shuffle cbdata out into
a seperate library.

I may remove src/leakfinder.c - I can't see anything that currently uses
it and there are better tools available nowdays to find memory leaks.
(Or i may just shove it in a seperate library just to keep it around
but out of the way.)

The intlist/wordlist code is just plain ugly and should be replaced
with something slightly more sensible as time permits. Its not that
important (yet!), so I left in similar kludges which should work.

That ends the "easy" code reorganisation pass. My next phase will
probably include fd, comm, disk, ipc and helper code so I can start
implementing a few unit tests and play with potential threading models.

http://code.google.com/p/cacheboy/wiki/StuffToTest lists things which
I've changed around and require testing. The bulk of the above changes
however are structural.

The change history is available at:

http://code.google.com/p/cacheboy/wiki/StuffToTest

Don't try browsing the source; your browser may hang.

Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -