Re: async-calls squid3/src comm.cc,1.81.4.16,1.81.4.17

Alex Rousskov wrote:
> On Tue, 2008-01-08 at 16:19 +1300, Amos Jeffries wrote:
>
>> As you say this is headed toward SMP/threading and that may cause any of
>> the scheduled steps to be run simultaneously or in a different order.
>
> We will cross that bridge when/if we have to. For now, the order of
> async events firing is guaranteed to be the same as their scheduling
> order. A lot of Squid code depends on that guarantee.

You seem to have slightly missed my point.
That this code is currently depending on that implicit guarantee when it
should not be. This change is the perfect time to drop that implicit
dependency and make it clean (bug free!). Having one precursor call
schedule its successor is the cleanest and fastest way to do that.
Run-time speed optimizations can wait, but this is a stability issue.

>
>> IMO in order to keep code flow same as before the async it needs to be
>> written something like this:
>>
>> Using async calls the above become:
>> 1) fd marked as closing, read and write handlers with COMM_ERR_CLOSING
>> parameter scheduled for execution
>>
>> after some time ...
>> - read/write handlers called with COMM_ERR_CLOSING
>> 2) comm_close handlers scheduled for execution
>>
>> after some more time ...
>> - comm_close handlers called
>> 3) closing fd and initializing the fdc_table[fd] operations scheduled
>> for execution
>>
>> And after even more time...
>> - The fd closed and fdc_table[fd] initialized.
>>
>> Each step in the process scheduling the next as possible to keep them in
>> sequence. With no single step scheduling the entire lot as seperate
>> (possible async-reversable!) events.
>
> I do not know what async-reversable is, but I think what you propose is
> practically equivalent to Christos code because async events never get
> out of order.

async-reversable is when things happening asynchronously CAN get out of
order. And in wort case occur in reverse order of scheduling.

Async generally means the thread/call/job is completely independant on
any other currently scheduled. Any dependency make the dependent jobs
synchronous regarding each other.

Squid is fast heading towards a state where the async queue might easily
be split between processors. That means several fast finishing calls
scheduled after a single slow-finishing might be run before the
slow-finishing is done.

>
> The only difference I can imagine is that in your scheme, a handler
> called earlier (in step 2) can schedule another/new async call that will
> be executed before handlers called in step 3. This should not make a
> difference for correctly written handlers.

The case which started this thread is a case in which the execution
timing of (3) *does* effect other calls/handlers.

Maybe I'm getting confused but your argument was that each async call is
self-complete in state yes?

>
> Cheers,
>
> Alex.
>
>
>>> Henrik Nordström wrote:
>>>> And why is this needed? The sendComplete callback should be invalidated
>>>> if the HTTP connection state is no longer there.
>>>>
>>>> Adding this reshedule adds a noticeable overhead an delay, plus makes
>>>> tracing of the code flow more difficult.
>>> The hope is that we are gaining in stability and less complex code
>>> in upper levels.
>>>
>>
>

Keep state-automata in a mult-dimensional threaded environment etc. in
mind during the current designs and they won't need re-designing next
year when its implemented.

The calls for SMP support and threading in squid should not be as huge
an overhaul as IPv6 was. It should be possible after async is merged to
link some threading library and incrementally turn each async call into
an independent thread.

Amos

-- 
Please use Squid 2.6STABLE17 or 3.0STABLE1.
There are serious security advisories out on all earlier releases.