> I recently wrote a perl script that read a word from a text file, opened a
> unique file, wrote the text and then closed that filehandle.
> I found that it was spending at least 20% of it's time in system mode,
> taking forever and being verry ineficient.
Yes, you are right about the big overhead. Every open has to go
through lots of OS blabla, including directory searches, inode caches,
directory caches, buffers, etc. etc. If Squid cache contents is looked
at like a huge database, then there should be lots of experience to be
taken from database developers. Many of them suggest to use totally
raw disks with their own filesystem on them to bypass OS totally. This
would both reduce memory needs and allows to optimize within application.
> I think that creating one huge file which contains all the cached pages
> could speed up squid substantially.
There are few problems with that, Having one open file enforces
serialization of access to it within application, switching off OS-es
internal optimizations, so to make things fast we'd need to do this
optimization within squid? Tough task.
Then, we could use mmap to map the whole file (or any parts of it)
in virtual memory space, but then we never exactly know when access
to any part of it will introduce iowait and blocks the whole process
upon reads. On the other hand, mapped page writes can be made async
[memcntl(2)], giving OS chance to optimize actual writes and skipping
write calls with buffer allocations and lots of other file operations.
Further, using mmap we could lock certain critical memory areas into
ram, overcoming incredible performance degradation in cases when these
areas would be otherwise paged out into swap, I think Store Database
would be the first candidate.
Using systemwide shared mapped cache contents could allow for using
multiple instances of squid sharing the same cache - might be useful,
but also leads coding to the next stage of complexity. But in its
simpliest form it could be used to allow for very fast squid startups.
Lets imagine a "dispatcher" proccess running that starts squid and hands
it over the once mapped in and checked file - no need to every time do
integrity checks, just attach to it and start serving. Will make crash
(core) recovery much faster. In its toughest form it might provide kind
of non-stop fault tolerance and possibly effective use of multiple cpu's
in the system...
> ---------------------------------
> I think that a lot of development would have to go into the "large file"
> thing... so the extra x<10% time spent less in system time might not
> be worth it.
Actually, the benefit will get bigger with bigger caches. We here have
about 0.5M objects in cache, thousands of directories, and I guess that
substantial part of all disk io is to do with directory lookup and
inode updates. To make things work faster, I had to give away lots of
ram for buffering and caching, kernel also takes its needs without
asking. I have about 30-40MB of ram locked for kernel needs, and thats
alot!
On the practical side, using single file will in any case mean total
rewrite of squid, So, if this will be considered practical, not before
rewritten squid appears, I guess.
Using single file we need to implement some efficient virtual file
system within this file, with its own directory indexes, deal with
fragmentation, etc. - lots of tricky stuff...
We will need a special handling for crash recovery, fschk type of
thingy.
Select loop cannot go anywhere, so there would be some major headache
glueing those together in an efficient way. Async IO (kaio) might be very
useful for reads instead of depending on pagefaults. Btw, socket io might
read from socket and write directly into mapped file and read directly
from freshly written parts of it - proxying with cacheing will become
almost no-memory-hungry deal.
Squid might use several files for different domains or some other
criteria, say - use separate databases for local domains, or short-living
pages, or whatever. If it can be predefined what size each file is, it
would be much better to utilize different disk sizes...
In summary, to me it seems to give substantial benefit in terms of
speed of squid and efficient use of system resources. Then again, this
means a new squid, alot more complex one.
Would some of you experienced coders please estimate the level of
complexity of rewriting squid to use something like this?
-------------------------------------------------------------------
Andres Kroonmaa Telefon: 6308 909
Network administrator
E-mail: andre@ml.ee Phone: (+372) 6308 909
Organization: MicroLink Online
EE0001, Estonia, Tallinn, Sakala 19 Fax: (+372) 6308 901
-------------------------------------------------------------------
Received on Tue Jul 29 2003 - 13:15:40 MDT
This archive was generated by hypermail pre-2.1.9 : Tue Dec 09 2003 - 16:11:14 MST