• =?UTF-8?Q?Christian_K=c3=b6stlin?= (10/10) Dec 07 2016 I really like std.concurrency for message passing style coding.
• =?UTF-8?Q?Ali_=c3=87ehreli?= (11/21) Dec 07 2016 The simplest idea is to have a single dispatcher thread that distributes...
• =?UTF-8?Q?Christian_K=c3=b6stlin?= (18/34) Dec 07 2016 Hi Ali,

=?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:

I really like std.concurrency for message passing style coding.

Today I thought about a scenario where I need a multiple producer,
multiple consumer pattern.
The multiple producer is easily covered by std.concurrency, because
all producers just can send to one Tid.
The tricky part is the multiple consumer part. At the moment I do not
see a way for several Tid's to share the same mailbox. Did I miss
something, or how would you implement this paradigm?

thanks in advance,
christian

=?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:

On 12/07/2016 12:08 PM, Christian Köstlin wrote:
 I really like std.concurrency for message passing style coding.

 Today I thought about a scenario where I need a multiple producer,
 multiple consumer pattern.
 The multiple producer is easily covered by std.concurrency, because
 all producers just can send to one Tid.
 The tricky part is the multiple consumer part. At the moment I do not
 see a way for several Tid's to share the same mailbox. Did I miss
 something, or how would you implement this paradigm?

 thanks in advance,
 christian

The simplest idea is to have a single dispatcher thread that distributes 
to consumers. However, both that and other shared mailbox designs are 
inherently slow due to contention on this single mailbox.

Sean Parent's "Better Code: Concurrency" presentation does talk about 
that topic and tells how "task stealing" is a solution.

There are other concurrency models out there like the Disruptor:

   https://lmax-exchange.github.io/disruptor/

It's a very interesting read but I don't know how it can be done with 
Phobos. It would be awesome if D had that solution.

Ali

=?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:

Hi Ali,

Thanks for the input, will read about this.
About the slowness. I think it also depends on the situation.
Sure, every message from all producers/consumers has to go through one
MessageBox, but, that is a small critical section, if the work for
producing and consuming takes long enough, the contention is not so
important (for small desktop sized problems). I would still be able to
stress all my cores :)

About a single dispatcher: Why not. But the dispatcher would have to
keep track (manually) of tasks to do and unoccupied workers. While this
is for sure possible, simply sharing the MessageBox would be a great
solutions I think. Looking into std.concurrency, it seems that it should
be possible to provide another spawn function, that takes an additional
MessageBox parameter that is then used for creating the Tid (given that
get and put on MessageBox are threadsafe).

What do you think?

Christian

On 08/12/2016 00:06, Ali Çehreli wrote:
 The simplest idea is to have a single dispatcher thread that distributes
 to consumers. However, both that and other shared mailbox designs are
 inherently slow due to contention on this single mailbox.
 
 Sean Parent's "Better Code: Concurrency" presentation does talk about
 that topic and tells how "task stealing" is a solution.
 
 There are other concurrency models out there like the Disruptor:
 
   https://lmax-exchange.github.io/disruptor/
 
 It's a very interesting read but I don't know how it can be done with
 Phobos. It would be awesome if D had that solution.
 
 Ali