Re: Squid regex fast purge.

On Tue, 10 May 2005, Ernest Rider wrote:

> So there is no easy way to get it back into memory for comparison? No
> API etc...?

There is an API, The store client API.

Keep in mind that all Squid operations is asyncronous with callbacks. In
most operations you tell the API "please do this and call me when it is
done". In each of these state data is protected by a mechanism called
cbdata (see programmers guide).

When you have the StoreEntry you invoke a swapin by

1. Register as a store client to the object by calling
storeClientListAdd(). The data argument needs to be a cbdata registered
structure wher you keep your state about what you are supposed to be doing
with the swapped in object.

2. Initiate reading of data from the object by calling storeClientCopy().
The data argument must be the same cbdata structure you used in
the call to storeClientListAdd().

3. When the object has been swapped in the callback you provided to
storeClientCopy() is called with the now available object content and your
state data mentioned above. If you need additional data call
storeClientCopy() again.

4. When you are done with the object use storeUnregister() to remove your
store client from the object, then free your state cbdata structure.

cbdata is described in the programmers guide.

>> Be warned that the hash_next function is not safe to be used in event
>> operations. There can only be one hash_next iteration at a time. Some
>> juggling will be required for doing this in a reasonable manner in an
>> incremental event function without running into locking/concurrency
>> issues.
>>
>
> Can you see a safe way of doing this?`

Yes.

You need to walk a hash bucket at a time, build a list of objects you want
to swap in and keep these object locked until you are done with them.

You should use an event (see eventAdd()) wich a very short interval to

a) Open the next store object in your list.

b) Scan the next bucket if your list is empty.

this to ensure other processing continues while you are scanning the
cache. If not you risk causing Squid to block all operations for extended
periods of time while your function is scanning the cache.

And don't open more than a small number of store object at a time. I
would suggest limiting yourself to one at the time.

Regards
Henrik
Received on Tue May 10 2005 - 16:59:28 MDT