Re: [squid-users] WARNING: An error inside Squid has caused an HTTP reply without Date

Thank again Amos for replying.

When the cache was being rebuilt (creating the swap.state file) I didn't
have any connection through squid and still was taking too much time:
Over 24hrs.

I had to delete the entire cache, something I really didn't want to do.
Also, upgraded to squid 3.1.14. Hope that version works better.

Thanks.

On 7/8/2011 1:04 AM, Amos Jeffries wrote:
> On 08/07/11 02:58, Wilson Hernandez wrote:
>> On 7/7/2011 10:42 AM, Wilson Hernandez wrote:
> <snip>
>>> Forgot to mention:
>>>
>>> I am running squid along with squidguard and I don't think it messes
>>> with squid's cache directory... This cache was used by an older squid
>>> version (3.1.9) that maybe messed it up.
>>>
>>> Any recommendation?
>
> SG does not play with Squids cache.
>
> It could be 3.1.9 ignoring absent Date: headers and letting those into
> the cache. In which case you should see those particular warnings
> reduce as the bad objects are found.
>
>
> On 08/07/11 02:36, Wilson Hernandez wrote:
>
> >
> > I did not override the waiting with -F. This cache directory is 200GB
> > not in the TB. I have this system running on a 3.2Ghz, 4GB RAM machine
>
> I refer you back to Squids message:
> "WARNING: Disk space over limit: 3615298144 KB > 204800000 KB"
>
> in other words:
> squid.conf says 200000 MB (~196GB). But Squid found 3.6 TB of stored
> objects in the cache_dir.
>
>
> I suspect this is a symptom of traffic going faster than Squids erase
> mechanisms can cope with. So lots of supposedly erased files are left
> on disk and "recovered" during the DIRTY scan.
> Only faster disks can fix this for the current Squid.
>
> > and is taking for ever to rebuild the cache (almost 20hrs), don't know
> > why is taking so looong. I am thinking of deleting the entire cache
> but,
> > don't want to loose it all.
>
> Squid is doing a "DIRTY" scan. It opens every folder, every file
> inside, loads teh fil data parses it into the index and oves on to the
> next one.
> There are 3.2 millions objects to do this for. At the same time as
> handling live traffic which is adding new ones rapidly. 10-20 hours is
> fairly normal for a 200 GB cache of small objects.
>
> The best way to avoid this lag in older Squid is to let it complete,
> save the results into swap.state. And to ensure that shutdown_timeout
> is long enough that swap.state is fully saved on shutdown in future.
>
> You can use a workaround of starting an instance of Squid without
> caching and letting clients use that directly. Starting a second in
> the background to do the caching. With a bit of config tweaking the
> first one can bypass the second until the cache is ready and available.
>
>
> The squid-3.2 "rock" store has some fixes trying to improve the
> situation. But I'm not yet sure how successful they are yet.
>
>
> On 08/07/11 02:58, Wilson Hernandez wrote:
>> Just installed ver 3.1.14 and get the same results, no internet and is
>> taking a looong time for swap.state to complete.
>>
>> 2011/07/07 10:52:27| Starting Squid Cache version 3.1.14 for
>> i686-pc-linux-gnu...
>> 2011/07/07 10:52:27| Process ID 13441
>> 2011/07/07 10:52:27| With 32768 file descriptors available
>> 2011/07/07 10:52:27| Initializing IP Cache...
>> 2011/07/07 10:52:27| DNS Socket created at [::], FD 7
>> 2011/07/07 10:52:27| DNS Socket created at 0.0.0.0, FD 8
>> 2011/07/07 10:52:27| Adding nameserver 172.16.0.2 from squid.conf
>> 2011/07/07 10:52:27| Adding nameserver 172.16.0.1 from squid.conf
>> 2011/07/07 10:52:27| helperOpenServers: Starting 10/10 'zapchain'
>> processes
>> 2011/07/07 10:52:27| Unlinkd pipe opened on FD 32
>> 2011/07/07 10:52:27| Store logging disabled
>> 2011/07/07 10:52:27| Swap maxSize 204800000 + 1048576 KB, estimated
>> 3216384 objects
>> 2011/07/07 10:52:27| Target number of buckets: 160819
>> 2011/07/07 10:52:27| Using 262144 Store buckets
>> 2011/07/07 10:52:27| Max Mem size: 1048576 KB
>> 2011/07/07 10:52:27| Max Swap size: 204800000 KB
>> 2011/07/07 10:52:27| Rebuilding storage in /var2/squid/cache (DIRTY)
>> 2011/07/07 10:52:27| Using Round Robin store dir selection
>> 2011/07/07 10:52:27| Current Directory is /workingdir/squid-3.1.14
>> 2011/07/07 10:52:27| Loaded Icons.
>> 2011/07/07 10:52:27| Accepting intercepted HTTP connections at
>> 172.16.0.1:3128, FD 34.
>> 2011/07/07 10:52:27| HTCP Disabled.
>> 2011/07/07 10:52:27| Squid plugin modules loaded: 0
>> 2011/07/07 10:52:27| Adaptation support is off.
>> 2011/07/07 10:52:27| Ready to serve requests.
>>
>
>
> Amos
Received on Sat Jul 09 2011 - 16:52:28 MDT