Frequency-based replacement and store_avg_object_size

Duane and other Squid Masters,

I ran into an ussue when using GDSF the other day. I was using the
default value of store_avg_object_size (13 KB) and ran into a problem
where filemap utilization was too high, eventually leading to a crash.
The reason is that GDSF leads Squid to store more, but smaller objects,
thereby improving hit rate. (By contrast LFUDA leads to storing fewer,
larger objects, reducing overall bandwidth demand. Both favor popular
objects, rather than recently accessed objects.)

In looking at the source I've come up with a couple suggestions on a
fix but want to run them by you. First, it can be addressed in the
config file with a comment to reduce store_avg_object_size. But who
will see this? These parameters are related, so I don't think this is
ideal.

Second it can be addressed whereever avgObjectSize is used, for example
to multiply it by a constant depending upon replacement policy (e.g.,
1.0 for LRU, 4.0 for GDSF, 0.4 for LFUDA). But this is not scalable and
leads to problems when someone forgets to include the appropriate
translation. Places that seem to matter are storeInitHashValues,
storeDigestCalcCap, maybe storeDirConfigure.

Finally the modification could be made where the value is read from the
config file (cf_parser.c). The disadvantage here is that the value used
at runtime is different from what is specified in the config file,
which may lead to confusion (but you'd know this better than I).

Instead of this modification, which seems like a hack to me, the config
parameter could be eliminated completely and Squid could determine the
average object size strictly based upon replacement policy without any
input from the user. I tend to favor making things as automatic as
possible, limiting the things users can screw up. Then the parameter
would be set as LRU: 10 KB; GDSF: 2 KB; LFUDA: 15 KB. What do you
think?

One final note - my personal home page has gone away on the HPL web
server when I "defected" to Akamai. There is an HPL technical report
that describes the policies, though, that can be referenced instead:
http://www.hpl.hp.com/techreports/1999/HPL-1999-69.html. Please replace
the URL in store_heap_replacement.c with that one... Thanks,

-- jad --
John Dilley
jad@akamai.com
Received on Wed Nov 03 1999 - 10:25:20 MST