Re: [squid-users] Howto: Squid3 on Debian Sarge with Linux 2.6 + NPTL + epoll

Thanks Pauli for sharing this information.

i have aslo tried same kind of setup on Redhat

i am interested if any have run benchmarking on
(linux 2.4+squid-2.5) and (linux 2.6+epoll squid-3)

Thanks and Regards

--- Pauli Borodulin <boro@fixel.org> wrote:
> Hello,
>
> I tried the latest Squid3 today on Debian Sarge
> (testing) with Linux
> 2.6.3, NPTL and epoll and I was quite satisfied with
> it after a small
> test run. For everyone else interested trying it, I
> wrote a small
> howto-stylish description how to do it quite easily.
>
> If you don't know what NPTL and epoll are and why
> Linux 2.6 is (afaik)
> better for Squid3 than 2.4, here are short
> explanations:
>
> NPTL (The Native POSIX Thread Library for Linux) is
> a new thread library
> for Linux which is going to replace the old
> LinuxThreads-library
> distributed with glibc by many Linux-distributions.
> Old implementation
> is based on the principles outlined by the kernel
> developers at the time
> the code was written, 1996. Since that, many new
> kernel extensions have
> been introduced to kernel which makes it possible to
> create a new thread
> library far better than the old LinuxThreads in
> scalability and
> performance. For example Redhat has already included
> NPTL in their Red
> Hat Linux 9 and Enterprise-editions and others will
> probably follow.
> Squid uses threads for example with aufs. For more
> information about
> NPTL and it's design, read
> http://people.redhat.com/drepper/nptl-design.pdf
>
> Epoll is a replacement for poll which is used to
> delivery network events
> from kernel mode to user mode (here, Squid3).
> Because Squid3 heavily
> depends on network I/O, efficient delivery of events
> is vital. For more
> information about epoll and it's efficiency (for
> example compared to
> traditional poll), check
>
http://www.xmailserver.org/linux-patches/nio-improve.html
>
> Linux 2.6 kernel introduced (for example) a new mmap
> syscall which is
> O(1) compared to the old implementation of O(n). For
> cache/proxy servers
> it is important to map files into memory instead of
> having a buffer and
> reading the file contents into the buffer. Mapping
> the file into memory
> directly leaves more free memory for operating
> system to do I/O
> buffering. This is what the mmap syscall is used
> for. What does O(1) and
> O(n) mean, then? O(1) (ordo one) means that the
> operation (execution of
> mmap syscall) is always executed in a fixed time --
> number of mapped
> pages doesn't matter. O(n) (ordo n) means that the
> number of pages has
> an effect on the execution time (here, linear) --
> the more pages, the
> longer it takes. There's a page on the WWW that has
> some benchmarks
> about BSD and Linux but also Linux 2.4 and 2.6.
> This page is located at
> http://bulk.fefe.de/scalability
>
> After this short (hopefully) explanatory part, I
> assume that you have at
> least base install of Sarge running and have some
> knowledge on using apt
> and compiling stuff (like Linux kernel). Also
> remember that if you use
> epoll, you can't use delay pools or deferred reads
> (there is a notice
> about this in comm_epoll.cc).
>
> 1. Install all Debian packages needed to compile 2.6
> kernel and Squid3
> (you will need at least libc6-dev, bin86, binutils,
> make, gcc and g++).
>
> 2. Install Debian package "module-init-tools" which
> is needed for
> loading modules of 2.6 kernel. Sarge's scripts are
> smart enough to check
> which kernel version (2.4 or 2.6) you're running and
> uses the correct
> version of these tools, so you don't have to worry
> about this if you
> want to revert back to 2.4 kernel later.
>
> 3. Get the source code for 2.6 (for example from
> www.kernel.org's
> mirror), compile and reboot with it. This is needed
> for NPTL and epoll
> to work (it will work with 2.4 too, but it will need
> some serious patching).
>
> 4. Install Debian package "libc6-i686". This is an
> i686 optimized
> version of libc6 (so you will need a i686 class CPU,
> ie. "uname -m"
> should print "i686"). What's more important, it
> includes NPTL instead of
> LinuxThreads.
>
> 5. Extract Squid3 sources and execute configure.
> Remember to include
> "--enable-epoll --with-pthreads" on your configure
> line among other
> parameters you want.
>
> 6. Remove all occurences of "-lepoll" from Squid3's
> src/Makefile. This
> has to be done because Sarge's libc6-dev already
> contains epoll support
> and separate epoll library is not needed (Squid3
> will probably figure
> this out itself in future). I counted four
> occurences. Then compile
> Squid with "make" and install it with "make
> install".
>
> 7. You should now have Squid3 compiled with NPTL and
> epoll. Go have fun
> with it.
>
> And always the so needed disclaimer: I do not take
> any responsibility if
> something goes wrong and so on. Squid3 is not yet
> production ready so do
> not expect too much.
>
> Comments and corrections on the list, please. :-)
>
>
> Regards,
> --
> .------------------------------------------ -
> | Pauli Borodulin | snail. .
> | email. boro@fixel.org | Haukkamäenkatu 1 F 69 |
> boro@ircnet
> | gsm. +358 44 5555381 | 33560 Tampere, Finland |
> http://boro.fixel.org
>
'------------------------------------------------------------------
> -- -

__________________________________
Do you Yahoo!?
Get better spam protection with Yahoo! Mail.
http://antispam.yahoo.com/tools
Received on Sun Feb 29 2004 - 15:55:28 MST