Re: [squid-users] high memory usage (squid 3.2.0)

Alex Rousskov wrote:
> On 04/10/2013 09:22 AM, Mr Dash Four wrote:
>
>
>> the total memory used of all objects placed in ram should
>> not be 6 times the cache_mem value
>>
>
> There is no such rule or law of physics. cache_mem is not squid_mem (a
> not-yet-supported option that would limit total Squid memory usage).
>
> The total memory used by Squid is, roughly:
>
> 1) initial memory used by Squid shortly after start +
> 2) memory used to store memory-cached objects +
> 3a) memory used to index memory-cached objects +
> 3b) memory used to index disk-cached objects +
> 4) memory used for current transactions +
> 5) memory leaked since Squid start
>
> cache_mem determines the maximum value of (2)
>
> cache_mem and mean object size in memory cache determine the maximum
> value of (3a) because they determine the maximum number of objects in
> the memory cache.
>
> cache_mem has no significant direct effect on the other four components.
> Depending on the local circumstances, every other component alone may
> exceed cache_mem 5 times or more, especially when cache_mem is
> relatively small, e.g., 200MB. And taken together, they can exceed
> 6*cache_mem level even easier.
>
> For example, a 10KB/s memory leak will consume 1GB of RAM in less than
> two days. Similarly, an overloaded Squid may grow (4) at rates exceeding
> 1MB/s.
>
Well, in that case, whoever did the testing of squid before releasing
such a buggy software should be shot on site or be locked in a dark
garage 24/7 somewhere and taught how to use valgrind.

> Now back to your actual problem. An idle Squid with 200 MB memory cache
> and 200 MB disk cache should not use 1.3GB of RAM unless component (1)
> is very high due to some huge ACL lists or other unusual complications.
>
Nope. I have 5 rules, that's all.

> If your (1) is just a few MBs and your idle Squid uses 1.3GB, then I
> would suspect a memory leak.
>
At start, squid memory consumption is about 25-30MB - give or take a few
megabytes, keeping in mind that according to the logs I have in excess
of 14500 objects (verified by squid at startup) in the disk cache.

> If you want to reduce Squid memory usage, I recommend eliminating memory
> leaks as suspects first because they are bugs and because they may
> consume all available RAM on your machine.
I am not a squid developer - I am a user.

> Whether your Squid leaks
> memory, I do not know. I do know that you are running v3.2.0 that
> probably does have leaks and a set of other nasty bugs fixed in
> currently supported Squid versions.
>
I'll try to see if the latest version has the above behaviour and if
that turns out to be the case and I have found no solution the the
problem I described in my initial post, then I have to just forget that
squid exists and use something else which is more reliable, like polipo
for example.

I do thank you for your time and your comments though - much appreciated.
Received on Wed Apr 10 2013 - 21:37:42 MDT