Re: ideas

On Mon, Jun 12, 2000, Henrik Nordstrom wrote:
> Adrian Chadd wrote:
>
> > Yup. I was just trying to understand exactly what we are both getting at.
> > I'll move to your discussion since it dictates the request forwarding
> > protocol somewhat.
>
> Not really. The only thing I dictated was that the storage manager must
> be able to tap data from the request forwarding, and that the request
> forwarding must be able to tell the storage manager new facts about
> already cached objects like "this object SHOULD no longer be cached", or
> "this object is still fresh according to the next hop". Not a word on
> how this is to be done.

I had a solution to this in my proposal but its in thepaper notebook
stage. I'll throw it in my document tomorrow.

> > With my let system, it shouldn't be too hard to make it work with
> > SMP if you assume a couple of limitiations - specifically that 'lets'
> > won't cross thread boundaries. They can miagrate between threads but
> > not cross threads.
>
> Can you please describe the difference between migrating and moving
> between threads?

I don't think there's a difference. What I was trying to get at is that
if the code is written to enforce all 'lets' in a request/reply on
one CPU/thread then you don't have to worry about cross-thread
communication and locking, which would make things a lot faster.
How this would perform in real life is unknown to me right now.
I am thinking that at sufficiently high loads the requests will be
spread across the IO threads and will 'balance' out.

> > This doesn't make sense at low load, but I'm thinking that at
> > higher loads it will be better than trying to handle inter-thread
> > communication.
>
> Inter-thread communication is expensive, and should be avoided if
> possible. Bound by mostly the same issues as inter-process communication
> except that the threads shares more resources like memory and
> filedescriptors.

I'm more worried about inter-thread locking. You really can't have
much mutex locking in the system I've described, because the main loop
has to be non-blocking. For cross-thread communication I can only
think of a form of message queue which would work OK, but might
be expensive..

> > The storage manager poses an interesting problem in SMP. Do you try to
> > make it totally SMP happy or do you do much like the above - all
> > clientlets talking to servlets/storage manager'lets' will be in one
> > thread, eliminating the need for a lot of inter-thread communication?
> > How will that perform at low and high loads?
>
> Here our two approaches differs a lot. If you go for a multithreaded
> design then there will be one thread doing storage maintenance, while
> the actual storing of objects (or FS communication) should be fully
> distributed.

I don't know that it needs to be fully distributed. For example,
if you have two CPUs, and you are swapping in an object on CPU A,
you then have a new request come in on CPU B and the request requires
this same object, you migrate the request across to CPU A and then
it can hook into the existing storge manager session.

Hmm. How this is actually cleanly implemented isn't known.

> > I'm not an SMP/threads person so the only way for me to get my head
> > around the issues is just to code it up and play.
>
> I prefer to think about it first. The principles are quite simple:
>
> a) Make sure you have enought parallellism to keep all parts working
>
> c) Avoid communication between execution threads if possible.
>
> The hard part is to find the correct tradeoff point between the
> communication and the parallellism.

Yup. Thats in my mind right now.

> > Now, how much further do we want to break it down? Breaking out
> > (say) the DNS code into a thread might make sense, but do you
> > think it will be worth it?
>
> Probably not, except for possibly a thread responsible for managing the
> cache contents and trowing out stuff which is not longer interesing.
>
> However, there might be a good idea to make the DNS cache a two layer
> design, with a small cache in each thread using DNS data (i.e. mainly
> the thread making outgoing connections) and a larger shared cache.

Ok. I'll summarise some more of this tomorrow and then when we
agree, I'll commit it to the document.

Can someone else comment on this please?

Adrian

-- 
Adrian Chadd			Build a man a fire, and he's warm for the
<adrian@creative.net.au>	rest of the evening. Set a man on fire and
				he's warm for the rest of his life.