"lookahead caching"

hi,

i was wondering if anyone's ever considered the idea of adding
"lookahead caching" to squid. that is, for speed (definitely not
bandwidth reduction purposes), do something like the following:

whenever a new page is added to the cache, try to download any page
linked from that page and put it in a (smaller) different cache called
the transient cache. when a page in the transient cache is viewed, it
moves to the stable cache. the stable cache is managed in squid's
standard fashion. the transient cache can be managed in an LRU fashion
indexed on the date that this webpage's (most recent) parent was viewed.

that way, the typical users experience of clicking on some page, reading
for awhile, and then clicking on one of the links will be greatly
accelerated. this idea is basically what a product called peak net.jet
was doing a couple of years back (it doesn't look like they're still in
business though), but only with netscape's builtin cache.

so my question are as follows:
    would this be difficult to implement?
    would it violate netiquette to implement this? (imagine a single
user on a cable modem sucking around 10 times as much bandwidth on the
net)
    any other thoughts?

thanks for reading...
brian szymanski
bks10@cornell.edu
Received on Mon Mar 05 2001 - 03:09:50 MST