Re: client_side and comm_close

> On Sun, 20 Apr 2008 22:44:22 -0600, Alex Rousskov wrote:
>
> > What if we let the close callback to determine how it should be called?
> > Old code gets an "immediate" close callback (the default in v3.1). The
> > code that was written to handle async calls uses asynchronous callbacks.
> > This approach may allow for a smoother migration to 100% asynchronous
> > calls without introducing new complex inter-call dependencies. I believe
> > this approach is also easy to implement without changing much of the
> > existing code (old or new).
>
> Thats fine for me. In fact a very good migration plan.

I have implemented something very similar to the above and,
unfortunately, it does not work well. The plan would have worked
beautifully if all comm_close users (old and new) were independent from
each other. In real world, there are some natural dependencies.

What happens is that an asynchronous write handler call for "new" code
is scheduled first but may be dialed _after_ the synchronous close
handler call for the "old" code. When "old" and "new" code are managing
related state (e.g., client-side HTTP and REQMOD transactions), the
out-of-order execution of callbacks creates problems.

The good news is that I think I found another simple workaround, thanks
to Christos' earlier work on bug #2253. Instead of mixing sync and async
calls, Comm now makes sure that _queued_ Comm callbacks that were
scheduled before comm_close get their flags reset to COMM_ERR_CLOSING
before being dialed.

The above hack preserves callback order and prevents situations where
Comm user receiving a successful I/O callback starts modifying
Comm-related state and gets an assertion because the descriptor is
already in a "closing" state.

Even if this hack solves a few current problems, it is only a temporary
solution because modifying queued callbacks is ugly, not possible with
threads, and should not be necessary in a well-written code. The Comm
notes I posted earlier today document the plans to remove the
COMM_ERR_CLOSING flag and, eventually, this hack.

It is possible that ongoing testing will reveal problems with this
approach and we will have to go back to the drawing board, of course.

Stay tuned,

Alex.
Received on Fri Sep 05 2008 - 06:03:21 MDT