Re: [squid-users] storeDirWriteCleanLogs() blocking queries

> On tis, 2008-10-07 at 15:25 -0400, Chris Woodfield wrote:
>
>>
>> We've been noticing lately that the logrotation process is taking
>> longer and longer as our caches fill up - currently, with ~18 million
>> on-disk objects, we've seen it take as long as 12 seconds, during
>> which time squid is not answering queries.
>
> Yes..
>
>> Is this still the case that the storeDirWriteCleanLogs() function is
>> expected to take this long when the cache_dirs get this large?
>
> Time is expected to grow with size yes.
>
>> Is
>> there anything that can be done to mitigate this?
>
> Disable swap if you have any, to make sure the index data never gets
> swapped out. You generally don't need swap in a Squid server.
>
>> The issue is that we
>> rotate logs fairly frequently (multiple times per hour), which
>> amplifies this issue.
>
> Indeed.
>
>> As a workaround, we may disable the storeDirWriteCleanLogs() in
>> mainRotate() and trigger it on a different signal instead. Sound like
>> a reasonable workaround?
>
> Yes.

How about a global 'skip' timer like that used to silence NAT errors?

if(last_cleanup < squid_curtime - 3600) {
  last_cleanup = squid_curtime;
  storeDirWriteCleanLogs();
}

That way it doesn't need a new signal, but still gets run at reasonable
regularity. 3600, 86400 or similar maybe.

>
>> If so, what should the maximum time between
>> rotating swap.state be?
>
> Depends on the churn rate in your caches. The more that has changed in
> the cache since swap.state was last cleaned the longer Squid will take
> on an unscheduled restart.
>
> swap.state is not used during normal operations, only as a journal of
> the cache contents in case Squid gets restarted with no chance of
> writing out a clean index (power failure, kernel panic, kill -9, double
> seg fault, etc...)
>
> Regards
> Henrik
>
Received on Wed Oct 08 2008 - 01:39:22 MDT