Re: [squid-users] Squid 3.3 is very aggressive with memory

On Tue, Dec 24, 2013 at 7:21 AM, Eliezer Croitoru <eliezer_at_ngtech.co.il> wrote:
> OK so it seems like the basic issue is probably not in the level of FD or
> something similar.
> I do not know why the usage of ram is so high but the basic test I would do
> is:
> remove any cache_dir from squid
> remove any memory cache from squid

As mentioned in previous emails, I have already done this, and while
it did slow the memory growth down slightly, it didn't really stop it.

> make sure that the swap is at 0 state.
> make sure what process do run on this machine..

I am unsure what you're asking here. The primary function of the
machine in question is to function as a router - it performs
authentication, email proxying, VPN, DHCP leasing, DNS and many other
things on top of web access via Squid, so controlling the memory usage
to this degree is not really feasible. However, I can provide the
output of top for the system, if that would be helpful. Typically,
other processes use very very little memory, and none of them route
through Squid itself.

>
> I have a tiny proxy with 1GB ram and it works with no problems that I know
> of yet with 3.4.X.
>
> Since it's a 32bit machine and has couple things I have a spec file that
> worked fine for it on the latest build of squid.
>
> If you do not use authentication remove any of the non-relevant build
> instructions..

In the primary instance, we run NTLM authentication via direct proxy,
so I'll do a build using that without all of the other authentication
methods.

>
> Also if you can get into a position where all of your machine details are
> known such as "what each process uses??" you can answer yourself more.
>
> ps avx -L |grep squid
> can give more but if you do have top output for all squid process and
> subprocess it will be nice.
> as you can see these processes do use an amount of ram but if make sure that
> the ram is "slipping" from one way or another and in not accounted in top we
> have a more solid ground on that.
>
> What would "http://squid_ip:port/squid-internal-mgr/mem"
> Would give us?

The office will be closed until Monday, so I cannot answer the above
questions until then when the load has gone back up - I will report
back then with the information.

>
> it has a lot of details that can be analyzed..
>
> Can you by any chance describe the size of the network?
> We are not talking about a home network..
> 29 clients? ip addresses?

It's an office of ~30 people, the majority of which access the
Internet via transparent proxying, one or two using direct proxy for
various reasons.

>
> I would try to run the proxy with no cache at all for about 24\48 hours to
> see if there is a peek in the FD usage or any load on the service which can
> describe what you are talking about.

As mentioned earlier, I have already done this, and the load+FD usage
is essentially at a constant rate of what is described in the manager
info page linked to earlier.
>
> There is always the possibility for a memory leak and it's not always
> related directly to squid..
>
> What are the network requirements? Are you using WCCP(for example)?

Can you be a little more specific in what you're asking here? We're
not using WCCP.

Thanks,

Nathan.
Received on Sat Jan 04 2014 - 09:13:23 MST