Re: [squid-users] "Quadruple" memory usage with squid

A couple of updates today...

We do plan to give the logfile-daemon approach a try. There's some
concern that squid outright kills itself if one of the logfile daemons
exits. That creates some potential for data loss, but we're not sure
how squid could handle it significantly better without major, complex
changes.

With mem-only caching it restarts pretty much instantly, and since
that should never actually happen, it should be OK. We can live with
losing a few K of logs in the event of a crash as long as the crashes
are rare. :-)

It sounds like we'll need to write our own logfile daemon to handle
our specific rotation needs. That should enable us to eliminate the
daily squid rotation and restart it only for problems or
reconfigurations.

Thanks again for the suggestion!

On the superpages... this worked very well right up until the actual rotation:

Thu Nov 26 04:51:03 UTC 2009
vm.pmap.pv_entry_count: 61361
vm.pmap.pde.promotions: 23123
vm.pmap.pde.p_failures: 327946
vm.pmap.pde.demotions: 17848

--- rotate starts ---

Thu Nov 26 05:01:03 UTC 2009
vm.pmap.pv_entry_count: 1480641
vm.pmap.pde.promotions: 23159
vm.pmap.pde.p_failures: 343553
vm.pmap.pde.demotions: 20637

--- rotate finished ---

Thu Nov 26 05:11:03 UTC 2009
vm.pmap.pv_entry_count: 1908056
vm.pmap.pde.promotions: 23212
vm.pmap.pde.p_failures: 413171
vm.pmap.pde.demotions: 21470

Ever since then, it's gone back to the old way, with lots of constant
p_failures and frequent demotions. Note the machine has 10GB free RAM
at the moment, so there's no memory pressure. Currently:

Thu Nov 26 13:31:04 UTC 2009
vm.pmap.pv_entry_count: 2021342
vm.pmap.pde.promotions: 24950
vm.pmap.pde.p_failures: 941176
vm.pmap.pde.mappings: 1641
vm.pmap.pde.demotions: 21680

Thu Nov 26 13:41:04 UTC 2009
vm.pmap.pv_entry_count: 2021053
vm.pmap.pde.promotions: 24973
vm.pmap.pde.p_failures: 946147
vm.pmap.pde.mappings: 1641
vm.pmap.pde.demotions: 21681

So it seems like at present the superpages feature is great right up
til you need it, unless squid does something odd to its memory at
rotation time. I will report this to the freebsd-hackers list as a
possible problem with superpages, and see if they have any thoughts.

Thanks again everyone!
Received on Thu Nov 26 2009 - 14:24:21 MST