Re: [squid-users] Was: how to raise filedescriptors on kernel 2.4

Guy Fraser wrote:

> I think we all want the source code updated to allow Squid
> running on Linux machines to use the allocated maximum
> rather than the header variables.

The upper bound must currently be defined in FD_SETSIZE when Squid is
compiled. On most platforms where the kernel is neutral applications can
redefine FD_SETSIZE to the desired value, but the glibc headers do not
accept FD_SETSIZE being redefined.

However, for some versions of glibc a workaround have been found
allowing FD_SETSIZE to be increased at compiletime without having to
manually edit the glibc headers. See Squid-2.5.

> And one thing of note. I think I needed to set the ulimit
> for the current session before compiling.

Indeed, plus make sure the system at all can handle the number of
sockets you are compiling for. The squid configure script probes the
system to see that it really can handle at least the configured
filedescriptor limit..

> Another thing, put a ulimit command in the init script
> before the squid daemon is started.

Correct, or in a initscript (man init, or see my page on the subject)

> Could you fix the code so that we don't have to do this?

Not really. We cannot guess what value you need to support, and having
Squid use a too high value has negative performance impact.. so we rely
on ulimit and a full probe and tries to forward this to the system
header files where possible (and it is almost not possible on GNU libc
based systems such as Linux)

> There is a definition that stops squid on LINUX from being
> allowed automaticaly size the number of file descriptors,
> could that be fixed?

Convince the glibc maintainers that it is needed to be able to override
the FD_SETSIZE variable at compiletime in a sane manner. But we do have
a very ugly workaround nowdays that seems to work..

> In kernel 2.4 the maximum number of file descriptors is
> determined by the amount of ram installed. The value can
> be seen in /proc/sys/fs/file-nr .

This is only one of many parameters relevant to the subject. This
parameter only sets the global maximum of files that may be open in the
system at any given point in time. You also needs to tune the number of
inodes, the per-process ulimit and maybe some additional parameters..

> I hope this problem will be resolved in the source code,
> there is no reason it couldn't be and there is support for
> other operating systems.

On all systems the upper limit is set at compiletime.

A few systems allows the default value defined in the system headers to
be overridden at compile time. GNU/Linux libc does not despite the fact
that the kernel allows the use of any number (within reasonable bounds).

Regards
Henrik
Received on Wed Jun 19 2002 - 16:19:15 MDT