Re: Squid regex fast purge.

On Fri, 2005-05-06 at 13:48, Henrik Nordstrom wrote:
> On Thu, 5 May 2005, Ernest Rider wrote:
>
> > Recently a requirement came up to be able to fast purge squid entries by
> > URL matching.
>
> Quite common request. Unfortunately not very easy to implement due to the
> design of the cache. The URL is for the major part of the cache only kept
> on disk, not in memory.
>
> > void
> > storeRemoveRegex(const char *regex) {
> > hash_link *walker;
> > int i=0;
> > regex_t compare;
> > regmatch_t pmatch;
> > debug(20, 2) ("storeRemoveRegex: Constructing Regex with %s\n",regex);
> > int code = regcomp(&compare, regex, REG_EXTENDED | REG_ICASE | REG_NEWLINE);
> > if(!code) {
> > debug(20, 2) ("storeRemoveRegex: Constructing Regex Done\n");
> > debug(20, 2) ("storeRemoveRegex: Finding first entry\n");
> > hash_first(store_table);
> > debug(20, 2) ("storeRemoveRegex: Finding first entry done\n");
> > debug(20, 2) ("storeRemoveRegex: Finding initial entry\n");
> > walker=hash_next(store_table);
> > debug(20, 2) ("storeRemoveRegex: Finding initial entry done (%p)\n",walker);
> > debug(20, 2) ("storeRemoveRegex: Entering Loop\n");
> > while(walker) {
> > debug(20, 2) ("storeRemoveRegex: Start Loop block with %p\n",walker);
> > StoreEntry *store_entry = (StoreEntry *) walker;
> > if(store_entry->mem_status == NOT_IN_MEMORY) {
> > debug(20, 2) ("storeRemoveRegex: The Entry object was null doing persistence retrieval\n");
> > <WHAT SHOULD GO HERE>
>
> Exacly.. this is the problem... The information you need is not available
> in memory and you need to swap in the object from disk.
>
> This is mostly the same as the external purge tool (see related software),
> except that doing it within Squid is a little more complex due to the
> multiplexed nature of Squid.
>

So there is no easy way to get it back into memory for comparison? No
API etc...?

>
> > debug(20, 2) ("storeRemoveRegex: The Entry object retrieved has pointer %p\n", store_entry->mem_obj);
> > }
> > debug(20, 2) ("storeRemoveRegex: Executing regex match on %s",storeUrl(store_entry));
> > if(!regexec(&compare, storeUrl(store_entry),1, &pmatch,0)) {
> > storeLockObject(store_entry);
> > debug(20, 2) ("storeRemoveRegex: Releasing Item (%5d) from the store:\n",i);
> > storeEntryDump(store_entry,2);
> > storeRelease(store_entry);
>
> This part is fine. As Squid is singlethreaded there is no locking needed.
>
> The loop may hit the same entry more than once (twice, once on the public
> location, and then once again on the private key location) if it is in
> use, but it is safe to call storeRelease multiple times so this is not a
> problem.
>
>
> Be warned that the hash_next function is not safe to be used in event
> operations. There can only be one hash_next iteration at a time. Some
> juggling will be required for doing this in a reasonable manner in an
> incremental event function without running into locking/concurrency
> issues.
>

Can you see a safe way of doing this?`

>
> Regards
> Henrik
>
Received on Tue May 10 2005 - 14:35:28 MDT