Re: async-io network / future Squid design

On 16 Sep 2000, at 18:42, Henrik Nordstrom <hno@hem.passagen.se> wrote:

> > With some speed improvements I hope? The signaled async network stuff
> > Nikita posted a few days back was cool but quite slow in it's then
> > current state. Has that been addressed? (Or was the problem between
> > the chair and the keyboard? i.e. Mistakes in configuration on my part?)
>
> I don't see the point of async-io network. It already is asyncronous via
> the TCP socket buffers (which is related to but not limited to the TCP
> window) and non-blocking I/O.
>
> What might be needed is to address the CPU overhead of poll(). When you
> have a large number of connections poll() becomes a CPU hog (not as bad
> as select() was, but still..).

 Actually, I'm starting to think that async network io could have some
 benefits. Sure not like disk io, per-event, but via some sort of
 queueing mechanism. Main idea is to offload main squid thread from
 wasting time on io, leaving it free to process requests in ram. I'm of
 course thinking of SMP scaling here. My wild thought of putting poll
 on its own thread maybe even isn't that crazy idea afterall.

 My tests show that poll() itself isn't actually a CPU hog. To reach 1ms
 overhead on my system I'd need to poll over 2600 disk FD's. With 1000 open
 files, running squid, I measured (with kernel trace) that typically poll
 returns in under 0.1ms, yet sometimes it blocked for several ms, seemingly
 randomly.
 poll only looks like cpu hog. I think the reason is that OS has to do
 some background processing, like tcp stack, disk buffers, VM handling.
 When OS is called poll, it doesn't have to return asap. poll is a perfect
 place to do some background processing. The problem is that OS may
 decide to handle background tasks related to squid in the time allocated
 to squid. This results in that poll spends measurable time in OS, and
 this OS processing time is accounted to squid's system time. For this
 it appears that poll has high cpu overhead. It definitely has quite
 some overhead, I don't argue that, but I think that this overhead is
 somewhat exaggerated.

 Generally, I believe that we cannot rely on poll to return as soon as
 some FD is ready for io. It may do so in most cases, but when OS needs
 to do something, it will block poll and do it. On SMP systems this is
 especially unfortunate, because the other CPU is not used for that, but
 cpu time is stealed from squid main thread.
 We could solve this by using async notification from kernel, or we could
 also try to move poll into a separate kernel thread. Then main thread
 in squid would be free to proceed.

 After we resolv poll "overhead", we'll see the same effect with
 read/write calls. For eg. after I applied my patches related to DNS
 timing defaults and increasing io sizes, my cachebox system time now
 dominates in read/write calls, and only a fraction in poll.

 We moved disk io to threads because each io call could block.
 It doesn't make sense to call async-io for each network io event, but
 may make sense to move a bunch of io events into queue and let a thread
 to process them in one shot.

 I imagine that if we have a way to parse ready FD's, buildup a list of
 io ops needed and then let a thread to fulfill them, and in callbacks
 let main thread to handle the data processing, we could get more
 efficient squid on SMP.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 Delfi Online
 Tel: 6501 731, Fax: 6501 708
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Tue Sep 19 2000 - 03:50:38 MDT