[squid-users] What is the max number of Squirm redirect_children?

Dear all,

I have a Squid transparent proxy which uses 40 Squirm URL rewriter
processes. Everything worked fine until today, when Squid crashed
after an error "179 pending requests queued":

2011/11/15 13:55:41| WARNING: All redirector processes are busy.
2011/11/15 13:55:41| WARNING: 40 pending requests queued
2011/11/15 13:55:41| Consider increasing the number of redirector
processes in your config file.
2011/11/15 14:05:41| WARNING: All redirector processes are busy.
2011/11/15 14:05:41| WARNING: 179 pending requests queued
2011/11/15 14:05:41| storeDirWriteCleanLogs: Starting...
2011/11/15 14:05:41| WARNING: Closing open FD 190
2011/11/15 14:05:41| 65536 entries written so far.
2011/11/15 14:05:41| 131072 entries written so far.
2011/11/15 14:05:41| 196608 entries written so far.
2011/11/15 14:05:41| 262144 entries written so far.
2011/11/15 14:05:41| 327680 entries written so far.
2011/11/15 14:05:41| 393216 entries written so far.
2011/11/15 14:05:41| 458752 entries written so far.
2011/11/15 14:05:41| 524288 entries written so far.
2011/11/15 14:05:41| 589824 entries written so far.
2011/11/15 14:05:42| Finished. Wrote 633017 entries.
2011/11/15 14:05:42| Took 1.49 seconds (426234.18 entries/sec).
FATAL: Too many queued redirector requests
Squid Cache (Version 3.1.7): Terminated abnormally.
CPU Usage: 91414.241 seconds = 55144.038 user + 36270.203 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 8
Memory usage for squid via mallinfo():
        total space in arena: 1007116 KB
        Ordinary blocks: 1003347 KB 100368 blks
        Small blocks: 0 KB 1 blks
        Holding blocks: 27164 KB 8 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 3768 KB
        Total in use: 1030511 KB 102%
        Total free: 3768 KB 0%

Restarting gave this error:

2011/11/15 14:14:02| WARNING: Disk space over limit: 11037556 KB > 10240000 KB
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 10: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 12: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 14: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 16: (2) No such file or
directory
2011/11/15 14:14:02| IpIntercept.cc(137) NetfilterInterception: NF
getsockopt(SO_ORIGINAL_DST) failed on FD 18: (2) No such file or
directory
(... snipped 60 entries similar as the one above ... )
2011/11/15 14:14:05| WARNING: All redirector processes are busy.
2011/11/15 14:14:05| WARNING: 40 pending requests queued
2011/11/15 14:14:05| Consider increasing the number of redirector
processes in your config file.

I tried increasing redirect_children to 300 in squid.conf, but since
each children spawns as a process, this brought the system to its
knees. Even with no active HTTP clients.
What is the max/reasonable number of Squirm redirect_children?

These two lines also puzzle me. I could rebuild the cache to get rid
of it, but I'd like to know why this occurred:

2011/11/15 14:14:02| WARNING: Disk space over limit: 11037556 KB > 10240000 KB
        Total in use: 1030511 KB 102%

Thanks a lot for your thoughts,

Leonardo
Received on Tue Nov 15 2011 - 15:59:53 MST