[squid-users] Out of memory problem in huge cache

Hi,

I have Squid 2.6 running on RedHat Linux with 8 GB memory, and
configured according to Squid wiki recomendations in
http://wiki.squid-cache.org/SquidFaq/SquidMemory#head-09818ad4cb8a1dfea1f51688c41bdf4b79a69991

However, it crashs every couple of hours with "FATAL: xmalloc: Unable
to allocate xxx bytes!" message
I wonder if I made something bad in configuration or it's a memory leak problem.

Any help will be appreciate

Following configuration and error details:

Squid version 2.6 STABLE 12
cache_dir size 100GB
cache_mem size 1.995 GB
memory_pools_limit 100 MB

So 2 GB for cacheMem and 100 * 10MB(per 1G disk in above wiki) is 3GB.
In addition there are additional 5 MB
for other system use, it should be enogh.

My Linux kernel support 4G memory space per process(tested), so the
problem is not in process memory limit.

I found in bugzilla a memory leak bug related to http_accept deny acl
rule, but my cache is configured with "http_access alow all" (it's a
private network and there are no security issues).

Some prints from cache.log
2008/09/21 22:40:31| Swap maxSize 102400000 KB, estimated 7876923 objects
2008/09/21 22:40:31| Target number of buckets: 393846
2008/09/21 22:40:31| Using 524288 Store buckets
2008/09/21 22:40:31| Max Mem size: 1997824 KB
2008/09/21 22:40:31| Max Swap size: 102400000 KB
2008/09/21 22:43:12| Finished rebuilding storage from disk.
2008/09/21 22:43:12| 14085510 Entries scanned
2008/09/21 22:43:12| 14084558 Objects loaded.
2008/09/21 22:43:12| 201 Duplicate URLs purged.
2008/09/21 22:43:12| 751 Swapfile clashes avoided.
2008/09/21 22:43:12| Took 161.5 seconds (87200.4 objects/sec).
2008/09/21 22:43:17| Completed Validation Procedure
2008/09/21 22:43:17| Validated 14084555 Entries
2008/09/21 22:43:17| store_swap_size = 100657756k

FATAL: xmalloc: Unable to allocate 65535 bytes!

Squid Cache (Version 2.6.STABLE12-20070429): Terminated abnormally.
CPU Usage: 5939.813 seconds = 2462.689 user + 3477.124 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 1
Memory usage for squid via mallinfo():
        total space in arena: -117092 KB
        Ordinary blocks: -129142 KB 286436 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 83308 KB 11 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 12049 KB
        Total in use: -45834 KB 136%
        Total free: 12049 KB -35%

thanks

Itzcak
Received on Wed Sep 24 2008 - 08:02:58 MDT