www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Concurrency

reply Graham St Jack <Graham.StJack internode.on.net> writes:
I have been using std.concurrency for a while, and have been very 
impressed by it. In particular, it was easy to get a complex 
multi-threaded application going with std.concurrency. The 
thread-local-storage behaviour of D is really cool too.

However, I have had some problems with std.concurrency (most of which 
I'm sure are already being addressed by Sean). In particular, I wanted a 
way to select on multiple file-descriptors so that I could (say) send 
commands to a thread that also reads from a blocking file-descriptor 
like a serial port, UDP socket or GTK main loop. I also wanted to 
experiment with the amazing template capabilities of D that I found out 
about in the book, and see if I could generate boiler-plate 
message-passing code that doesn't rely on "magic" like Variant.

The attached file is where I am at so far. It works, but still needs a 
lot of work to bring it up to scratch. The reason for posting it here is 
to stimulate some discussion about how D's standard concurrency module 
should behave (and maybe get a few helpful tips).

Please take a look and let me know what you think.

-- 
Graham St Jack
Jul 25 2010
parent reply Sean Kelly <sean invisibleduck.org> writes:
Graham St Jack Wrote:


 However, I have had some problems with std.concurrency (most of which 
 I'm sure are already being addressed by Sean). In particular, I wanted a 
 way to select on multiple file-descriptors so that I could (say) send 
 commands to a thread that also reads from a blocking file-descriptor 
 like a serial port, UDP socket or GTK main loop.


Hm... this is essentially what libev and libevent do.  The trick is mostly
making this work in a consistent and performant manner on both Posix and
Windows.  On Posix, the read event simply tells you data is available, while on
Windows the read event hands the data to you directly.  The former works quite
well with messaging while the latter... well, it's kind of weird.

I like that you're playing with Channels, by the way.  I chose the messaging
API in std.concurrency because it can have a range of more structured models
built on top of it, so I think it's more likely to see broad use done this way.
 Maybe channels will be the first :-)
Jul 25 2010
parent Graham St Jack <Graham.StJack internode.on.net> writes:
On 26/07/10 13:48, Sean Kelly wrote:

 Graham St Jack Wrote:

    

 However, I have had some problems with std.concurrency (most of which
 I'm sure are already being addressed by Sean). In particular, I wanted a
 way to select on multiple file-descriptors so that I could (say) send
 commands to a thread that also reads from a blocking file-descriptor
 like a serial port, UDP socket or GTK main loop.
      


 Hm... this is essentially what libev and libevent do.  The trick is mostly
making this work in a consistent and performant manner on both Posix and
Windows.  On Posix, the read event simply tells you data is available, while on
Windows the read event hands the data to you directly.  The former works quite
well with messaging while the latter... well, it's kind of weird.
    


I don't know much about Windows programming, but I was expecting this to 
be the sticking point. So do we have a solution in D for this? For me, 
it is the key issue with std.concurrency as it stands.

Using the Pipe approach as I did in Channel, maybe a Windows version 
could allow a byte to be removed from the Pipe for each message, and 
another byte immediately put on to replace it if the Channel was not 
then empty.

 I like that you're playing with Channels, by the way.  I chose the messaging
API in std.concurrency because it can have a range of more structured models
built on top of it, so I think it's more likely to see broad use done this way.
 Maybe channels will be the first :-)
    


The Channels I proposed were really just a first try at doing code 
generation, but they do work really well. I particularly like being able 
to print out the generated code to see what you are getting. I must 
admit that I find Variant perplexing, and had issues with the message 
size limitation and no prohibition on aliased message parameters.

In working with both std.concurrency and alt.concurrency, I have found 
that alt.concurrency has some advantages such as:

Creating Channels first and providing them as arguments to spawn() is 
simpler than spawning threads and passing Tids in messages.

Being able to use the same set of parameter types in two different 
messages, and having names for the messages is clearer.

Being able to have multiple threads reading messages from the same 
Channel - something I need (well, want because it is much easier than 
double-handling all the messages through an intermediary) on my current 
project.

The benefit of compile-time checking.


Of course std.concurrency has lots of bonuses too. What I am after is a 
discussion that leads to a std.concurrency that is better still.


By the way, how are you getting on with rolling out "shared" through 
core.sync? I'm very keen to find out if shared can at last be used as 
intended - such as for implementing a Channel. I could lend a hand if 
you need one.

-- 
Graham St Jack
Jul 25 2010