RE: [squid-users] squid: Memory utilization higher than expected since moving from 3.3 to 3.4 and Vary: working

On 2014-07-08 19:38, Martin Sperl wrote:
> Well - there are 4 Points here:
> a) this is gradual behavior that takes month to accumulate
> b) extra used memory is much bigger than Cache (1.5x aprox), which
> looks wrong to me as the other parameters you have given
> (diskcache,...) are not
> c) we have not seen this behavior with 3.3 and have just switched to
> 3.4 two month ago and it is running since then showing this increase.
> d) if you look at the memory report (mgr:mem) which I have shared you
> find: 900MB for 542k HttpHeaderEntries and 2GB for 791k Short_Strings
> - and that is almost 3 out of 4.2G for this report alone.
>
> So I was guessing there was some sort of memory allocation bug or
> something - maybe something to do with the now working handling of
> "vary" headers, that triggers this memory leak.
> To me it looks as if those "Vary" nodes are not clean up properly and
> that is why we are slowly increasing memory...
>

Hmm. Yes that does sound like what we call a pseudo-leak (same effects
as a memory leak, but not an actual leak since the objects are still
being reference-counted by something). The headers and HttpHeaderEntries
are stored as collections of String's. The HttpHeaderEntries are
ref-counted objects used by HttpRequest and HttpReply transaction
objects, which are themselves ref-counted. If there are a lot of those
being held onto for no reason this needs to be tracked down.

Are you in a position to test the 3.HEAD code to see if it is something
we fixed already there or a new issue to debug from scratch?

> There are specially a few things in the msg:info block that made me
> suspicious:
> Memory usage for squid via mallinfo():
> Total space in arena: -2064964 KB
> Ordinary blocks: 2047465 KB 161900 blks
> Small blocks: 0 KB 0 blks
> Holding blocks: 38432 KB 9 blks
> Free Small blocks: 0 KB
> Free Ordinary blocks: 81875 KB
> Total in use: 81875 KB -4%
> Total free: 81875 KB -4%
> Total size: -2026532 KB
> Memory accounted for:
> Total accounted: 1269447 KB -63%
> memPool accounted: 9658055 KB -477%
> memPool unaccounted: -11684587 KB -0%
> memPoolAlloc calls: 507567155336
> memPoolFree calls: 508713005568
> Especially those negative Numbers and percentages - but maybe that is
> just a reporting bug, which - if so - is hindering debugging the real
> thing

64-bit mallinfo() bugs produce corrupt output. The only uncorrupted
values above are the memPool accounted KB size and probably the "calls"
counts.

>
> Thanks,
> Martin
>
> P.s: it would be helpful if the output of mgr:mem would be formatted
> in a way that would allow it to get easily pasted into Excel or
> similar - no spaces in pool names would be sufficient...
>

The report is a TSV spreadsheet. If you save it directly to a file then
import into Excel selecting Tab separator instead of the default comma
(CSV) it loads as a nice spreadsheet.

Amos
Received on Tue Jul 08 2014 - 09:40:04 MDT