On 04/10/2013 03:37 PM, Mr Dash Four wrote:
> Alex Rousskov wrote:
>> The total memory used by Squid is, roughly:
>>
>> 1) initial memory used by Squid shortly after start +
>> 2) memory used to store memory-cached objects +
>> 3a) memory used to index memory-cached objects +
>> 3b) memory used to index disk-cached objects +
>> 4) memory used for current transactions +
>> 5) memory leaked since Squid start
>>
>> cache_mem determines the maximum value of (2)
>>
>> cache_mem and mean object size in memory cache determine the maximum
>> value of (3a) because they determine the maximum number of objects in
>> the memory cache.
>>
>> cache_mem has no significant direct effect on the other four components.
>> Depending on the local circumstances, every other component alone may
>> exceed cache_mem 5 times or more, especially when cache_mem is
>> relatively small, e.g., 200MB. And taken together, they can exceed
>> 6*cache_mem level even easier.
>>
>> For example, a 10KB/s memory leak will consume 1GB of RAM in less than
>> two days. Similarly, an overloaded Squid may grow (4) at rates exceeding
>> 1MB/s.
> Well, in that case, whoever did the testing of squid before releasing
> such a buggy software should be shot on site or be locked in a dark
> garage 24/7 somewhere and taught how to use valgrind.
FWIW, I recommend avoiding non-constructive and negative comments like
this if your primary objective is to get help solving your problem. Such
comments reduce the pool of people willing to help you. Find other ways
or other forums to vent your anger, frustration, or dissatisfaction.
My examples simply demonstrate how Squid memory management works. I am
trying to give you correct information about Squid so that you can
correct your assumptions and it becomes easier for you to troubleshoot
problems. Nothing else.
>> Now back to your actual problem. An idle Squid with 200 MB memory cache
>> and 200 MB disk cache should not use 1.3GB of RAM unless component (1)
>> is very high due to some huge ACL lists or other unusual complications.
>> If your (1) is just a few MBs and your idle Squid uses 1.3GB, then I
>> would suspect a memory leak.
> At start, squid memory consumption is about 25-30MB - give or take a few
> megabytes, keeping in mind that according to the logs I have in excess
> of 14500 objects (verified by squid at startup) in the disk cache.
This sound fine. How much does your _idle_ Squid consume after handling
traffic for a while (e.g., filling memory cache and then running under
load for a little bit more)?
Why am I asking about idle Squid? Because idle Squid does not have
component (4), which often dominates (especially if Squid is overloaded)
and which is difficult to measure otherwise.
>> If you want to reduce Squid memory usage, I recommend eliminating memory
>> leaks as suspects first because they are bugs and because they may
>> consume all available RAM on your machine.
> I am not a squid developer - I am a user.
Sure, but you do not have to be a developer to detect and eliminate
memory leaks. In most cases, you just need to follow instructions,
preferably without attacking software and people who are trying to help
you. The best first step is to upgrade to a supported Squid version.
The second step could be to measure memory consumption of an idle Squid
after it handles traffic. Producing a "memory consumption and load
versus time" graph often helps. More steps may be required, depending on
the outcomes of the first ones.
> I'll try to see if the latest version has the above behaviour and if
> that turns out to be the case and I have found no solution the the
> problem I described in my initial post, then I have to just forget that
> squid exists and use something else which is more reliable, like polipo
> for example.
While this is a reasonable plan if you do not need any Squid-specific
features, there is no need to pummel Squid while you are trying to get
Squid help. If you must vent about Squid, you can do that on the polipo
mailing list when you run into problems with that software (but I would
not recommend that either).
Cheers,
Alex.
Received on Thu Apr 11 2013 - 15:30:07 MDT
This archive was generated by hypermail 2.2.0 : Thu Apr 11 2013 - 12:00:03 MDT