SUMMARY: Memory Problems with 1.1.22 on Digital Unix

Thanks to the following people for emailing me about this:

        David Richards <dj.richards@qut.edu.au>
        Claude Morin <klode@isgtec.com>

I ended up rebuilding Squid with dlmalloc instead of GNU malloc and found
this to be much much better.

I am however still concerned that the memory usage of the Squid process
(131Mb) is far higher than the value of "Total Accounted" (75Mb) but as
Squid is no longer swapping hugely I'm not going worry too much about this.

My original email was:
> Hi,
>
> I'm having serious memory problems with Squid v1.1.22 on Digital Unix 4.0D
> (yes I know I should upgrade to v2.0 but this is in a production
> environment).
>
> The problem is that the Squid process keeps growing until eventually it
> starts swapping. I have compiled with gcc 2.8.1 and gnumalloc, plus I have
> followed the FAQ's suggestions on reducing memory usage but to no avail.
>
> Total memory used is far far higher than the figure given in the Total
> Accounted section of the Cache Information section of cachemgr even after
> I do a squid -k shutdown.
>
> System Configuration:
>
> DEC Alphaserver 1000
> 192Mb RAM
> 4GB disk with a UFS filesystem. (about 67% currently used)
>
> Squid Config:
> cache_swap 4090
> cache_mem 32
> maximum_object_size 2048
> memory_pools off
> reference_age 1 week
>
> The output from top is:
>
> load averages: 0.38, 0.42, 0.35 11:07:17
> 45 processes: 2 running, 7 sleeping, 36 idle
> Cpu states: 39.0% user, 0.0% nice, 9.2% system, 51.6% idle
> Memory: Real: 74M/185M act/tot Virtual: 22M/256M use/tot Free: 1056K
>
> PID USERNAME PRI NICE SIZE RES STATE TIME CPU COMMAND
> 3741 root 48 0 119M 110M run 30:11 36.70% squid
> ..
> ..
>
> and the Information section of cachemgr:
>
> {Squid Object Cache: Version 1.1.22}
> {Start Time: Fri, 09 Oct 1998 08:31:49 GMT}
> {Current Time: Fri, 09 Oct 1998 10:06:10 GMT}
> {Connection information for squid:}
> { Number of TCP connections: 40535}
> { Number of UDP connections: 0}
> { Connections per hour: 25777.4}
> { Select loop called: 304948 times, 18.564 ms avg}
> {Cache information for squid:}
> { Storage Swap size: 2299 MB}
> { Storage Mem size: 8809 KB}
> { Storage LRU Expiration Age: 7.00 days}
> { Requests given to unlinkd: 0}
> { Unused fileno stack count: 0}
> {Resource usage for squid:}
> { CPU Time: 1783 seconds (1544 user 239 sys)}
> { CPU Usage: 31%}
> { Maximum Resident Size: 111360 KB}
> { Page faults with physical i/o: 235}
> {Memory usage for squid via mstats():}
> { Total space in arena: 116080 KB}
> { Total free: 6761 KB 6%}
> {File descriptor usage for squid:}
> { Maximum number of file descriptors: 4096}
> { Largest file desc currently in use: 91}
> { Number of file desc currently in use: 73}
> { Available number of file descriptors: 4023}
> { Reserved number of file descriptors: 100}
> {Internal Data Structures:}
> { 284919 StoreEntries}
> { 6600 StoreEntries with MemObjects}
> { 6600 StoreEntries with MemObject Data}
> { 6578 Hot Object Cache Items}
> {Accounted Memory Usage:}
> { StoreEntry 284919 x 72 bytes = 20033 KB}
> { URL strings = 14949 KB}
> { IPCacheEntry 921 x 72 bytes = 64 KB}
> { FQDNCacheEntry 1 x 104 bytes = 0 KB}
> { Hash link 6578 x 24 bytes = 154 KB}
> { Pool MemObject structures 6600 x 168 bytes = 1082 KB ( 0 free)
> }
> { Pool for Request structur 1083 x 4424 bytes = 4678 KB ( 0 free)
> }
> { Pool for in-memory object 7161 x 4096 bytes = 28644 KB ( 0 free)
> }
> { Pool for disk I/O 7 x 8192 bytes = 56 KB ( 0 free)
> }
> { NetDB Address Entries 0 x 96 bytes = 0 KB}
> { NetDB Host Entries 0 x 32 bytes = 0 KB}
> { NetDB Peer Entries 0 x 32 bytes = 0 KB}
> { ClientDB Entries 190 x 312 bytes = 57 KB}
> { Miscellaneous = 6665 KB}
> { Total Accounted = 76386 KB}
> {Miscellaneous:}
> { Average Name-to-address lookup time: 1.343000 seconds}
> { Average Address-to-name lookup time: 0.010000 seconds}
> }
>
>
> Is there anything in ths short time I can do, with minimum disruption to
> our user community, while I plan the upgrade to 2.0?

        Rgds.,

                Nick.

# Nick O'Brien, Computer Officer "It gives me a headache just to #
# Canterbury Christ Church College think down to your level", Marvin #
# Phone: +44 1227 782468 the Paranoid Android, HHGTTG #
# Email:nick@cant.ac.uk Web:http://www.cant.ac.uk/staff/nick/home.html #
Received on Tue Oct 20 1998 - 02:46:34 MDT