Large Rock Store

Hello,

    We have started working on caching "large" (i.e., multi-slot)
responses in Rock Store. Initial design overview is available at the
following wiki page, along with several design alternatives we have
considered. Feedback is welcomed.

http://wiki.squid-cache.org/Features/LargeRockStore

As a part of Large Rock work, I will also try to fix at least one of
Store API quality problems that complicate any Store-related improvements:

1) Store class is a parent of too many, barely related classes:
StoreController, StoreHashIndex, SwapDir, MemStore (and TestStore). This
"top heavy" design forces us to add pure virtual methods to Store and
then supply dummy implementations in many Store kids. And, more
importantly, it makes it difficult to understand which part of the
storage API each Store kid is responsible for, leading to boundaries
violation and other problems.

2) There is no class dedicated to a non-shared memory cache.
StoreController currently implements most of non-shared memory cache
logic while MemStore implements shared memory cache. StoreController
non-shared memory sharing code should be moved to a dedicated class instead.

3) StoreHashIndex should probably become responsible for all disk caches
(SwapDirs) as a whole, while StoreController will coordinate disk and
memory caching (using StoreHashIndex and MemStore objects). Currently,
some disk-related manipulations reside in StoreHashIndex and some in
StoreController.

4) Get rid of the global store_table. Make it local to non-shared
caching code. As we have discussed previously, the existence of this
global makes shared caching life very difficult because we cannot share
such a complex table and yet some older code expects to find all entries
there. It also leads to problems with entry locking where the code
assumes that an idle entry will remain in the global table at least for
a while (I believe there are a few bugzilla reports about associated
core dumps). I doubt I can solve all the StoreEntry locking problems (it
may require significant client side rewrite), but removing store_table
global is a step in the right direction.

If you have suggestions on priorities or want to add other large-scale
Store annoyances to the above list, please let me know.

Thank you,

Alex.
Received on Tue Oct 16 2012 - 22:02:52 MDT