www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Concurrency in D?

reply "Kris" <fu bar.com> writes:
Here's just one article identifying what many of us already know ~ to more 
easily take advantage of the growing multicore trend, one needs tools with a 
bent towards such designs: http://theinq.net/?article=27247

Multicore is happening. Take a look here 
http://arstechnica.com/news.ars/post/20041106-4377.html, here 
http://www.realworldtech.com/page.cfm?ArticleID=RWT021005084318, here 
http://www.realworldtech.com/page.cfm?ArticleID=RWT102405055354 and possibly 
here http://arstechnica.com/articles/paedia/cpu/xbox360-2.ars/7.

We had a (very) brief discussion about 18 months ago as to what D might do 
to support such environments, and to perhaps leapfrog other contemporary 
languages in this regard. I say 'contemporary' since Occam 
http://en.wikipedia.org/wiki/Occam_programming_language was designed from 
the ground-up to support a model of distributed parallel computation (it 
failed to prosper due partly to a tight association with the Transputer)

What do you think D might do in this regard? Are there perhaps some notions 
from Occam (and/or Ada), beyond simple contention-controls, worth pursuing? 
Is there something at the language-level that might make notable differences 
vis-a-vis debugging parallel systems?

- Kris
Oct 26 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <djp91p$g2r$1 digitaldaemon.com>, Kris says...
We had a (very) brief discussion about 18 months ago as to what D might do 
to support such environments, and to perhaps leapfrog other contemporary 
languages in this regard. I say 'contemporary' since Occam 
http://en.wikipedia.org/wiki/Occam_programming_language was designed from 
the ground-up to support a model of distributed parallel computation (it 
failed to prosper due partly to a tight association with the Transputer)

What do you think D might do in this regard? Are there perhaps some notions 
from Occam (and/or Ada), beyond simple contention-controls, worth pursuing? 
Is there something at the language-level that might make notable differences 
vis-a-vis debugging parallel systems?

There was a rather lengthy thread on this in comp.arch a few months ago: http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4 Cilk may be worth looking at as well, as it's a parallel version of C: http://supertech.lcs.mit.edu/cilk/ I think D is already very well suited for common parallel programming techniques as it supports inline assembler, language-level threads, the volatile statement, and it has garbage collection. These features allow for safe, language complaint two phase update, lock free, and other populalr techniques... and things will be even better once we have stack-based classes. Beyond that, I'll have to give it some thought. I think Cilk may be a good reference for desirable features, but I haven't gotten further with it than skimming the docs :-) Sean
Oct 26 2005
next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
 Cilk may be worth looking at as well, as it's a parallel version of C:
 
 http://supertech.lcs.mit.edu/cilk/

and corn: http://corn.telefonia.pl/ i like the {# // <-- threaded func1(); // thread1 func2(); // thread2 } syntax very much
Oct 27 2005
parent Sean Kelly <sean f4.ca> writes:
In article <djrl39$1l0m$1 digitaldaemon.com>, dennis luehring says...
 Cilk may be worth looking at as well, as it's a parallel version of C:
 
 http://supertech.lcs.mit.edu/cilk/

and corn: http://corn.telefonia.pl/ i like the {# // <-- threaded func1(); // thread1 func2(); // thread2 } syntax very much

Ah, that's like "parbegin" and "parend" that I was introduced to in an OS design class, though IIRC those were for process-level concurrency--they certainly beat the heck out of fork() though :-) Sean
Oct 27 2005
prev sibling parent reply "Kris" <fu bar.com> writes:
Thanks for the links, Sean. That forum had some interesting comments ~ 
especially regarding transactable execution.

D certainly does has a good range of primitives for thread-support, though 
some might conceivably be exposed in a somewhat simpler (and perhaps 
smarter) fashion than they currently are. But, that aside, I was kinda' 
thinking more along the lines of debugging assistance ~ a known bugbear of 
multithreading development.

For example, how about the compiler supporting some kind of read/write 
coloring, to indicate potential read/write conflicts between threads? That 
would be incredibly useful, and not hard for the compiler to identify, 
assuming it had some kind of PAR support to recognize threaded sections ... 
that is ~ once the compiler has some basic knowledge of what the programmer 
is attempting, it can provide seriously powerful diagnostics.

I suspect there's relatively few of us who have notable experience with 
multi-threading, as this article asserts (MS architect says "developers 
ill-prepared for new CPUs") 
http://arstechnica.com/news.ars/post/20051027-5492.html.

Is that the case? How about an informal poll of the D community:

===============

1) Do you have reasonable experience with the problems associated with MT, 
and feel reasonably comfortable with that development model?

2) Assuming it were financially attractive, could you make what you feel to 
be educated & valid assertions about how to successfully migrate a 
'traditional' application to take full advantage of the new breed of CPU's?

3) Do you know anyone else who could handle #2? How many others?





"Sean Kelly" <sean f4.ca> wrote in message 
news:djppmg$12om$1 digitaldaemon.com...
 In article <djp91p$g2r$1 digitaldaemon.com>, Kris says...
We had a (very) brief discussion about 18 months ago as to what D might do
to support such environments, and to perhaps leapfrog other contemporary
languages in this regard. I say 'contemporary' since Occam
http://en.wikipedia.org/wiki/Occam_programming_language was designed from
the ground-up to support a model of distributed parallel computation (it
failed to prosper due partly to a tight association with the Transputer)

What do you think D might do in this regard? Are there perhaps some 
notions
from Occam (and/or Ada), beyond simple contention-controls, worth 
pursuing?
Is there something at the language-level that might make notable 
differences
vis-a-vis debugging parallel systems?

There was a rather lengthy thread on this in comp.arch a few months ago: http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4 Cilk may be worth looking at as well, as it's a parallel version of C: http://supertech.lcs.mit.edu/cilk/ I think D is already very well suited for common parallel programming techniques as it supports inline assembler, language-level threads, the volatile statement, and it has garbage collection. These features allow for safe, language complaint two phase update, lock free, and other populalr techniques... and things will be even better once we have stack-based classes. Beyond that, I'll have to give it some thought. I think Cilk may be a good reference for desirable features, but I haven't gotten further with it than skimming the docs :-) Sean

Oct 30 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Sun, 30 Oct 2005 13:57:01 -0800, Kris <fu bar.com> wrote:
 1) Do you have reasonable experience with the problems associated with  
 MT, and feel reasonably comfortable with that development model?

Yes. 4+ years developing a multi-threaded email server for windows, linux, freebsd, solaris (x86 and sparc), mac osx, etc. 7+ years in that area (web/email server) of development i.e. (single-threaded) web cgi support products and modules for use with the email server. 2+ years doing GUI style work (inherently multi-threaded), also support products for the email server.
 2) Assuming it were financially attractive, could you make what you feel  
 to be educated & valid assertions about how to successfully migrate a
 'traditional' application to take full advantage of the new breed of  
 CPU's?

I believe so.
 3) Do you know anyone else who could handle #2? How many others?

My boss and 4 or 5 (ex-)workmates. Regan
 "Sean Kelly" <sean f4.ca> wrote in message
 news:djppmg$12om$1 digitaldaemon.com...
 In article <djp91p$g2r$1 digitaldaemon.com>, Kris says...
 We had a (very) brief discussion about 18 months ago as to what D  
 might do
 to support such environments, and to perhaps leapfrog other  
 contemporary
 languages in this regard. I say 'contemporary' since Occam
 http://en.wikipedia.org/wiki/Occam_programming_language was designed  
 from
 the ground-up to support a model of distributed parallel computation  
 (it
 failed to prosper due partly to a tight association with the  
 Transputer)

 What do you think D might do in this regard? Are there perhaps some
 notions
 from Occam (and/or Ada), beyond simple contention-controls, worth
 pursuing?
 Is there something at the language-level that might make notable
 differences
 vis-a-vis debugging parallel systems?

There was a rather lengthy thread on this in comp.arch a few months ago: http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4 Cilk may be worth looking at as well, as it's a parallel version of C: http://supertech.lcs.mit.edu/cilk/ I think D is already very well suited for common parallel programming techniques as it supports inline assembler, language-level threads, the volatile statement, and it has garbage collection. These features allow for safe, language complaint two phase update, lock free, and other populalr techniques... and things will be even better once we have stack-based classes. Beyond that, I'll have to give it some thought. I think Cilk may be a good reference for desirable features, but I haven't gotten further with it than skimming the docs :-) Sean


Oct 30 2005
prev sibling next sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
(PS: the tone here might appear aggressive, but wasn't meant as such.)

Kris wrote:
 1) Do you have reasonable experience with the problems associated 
 with MT, and feel reasonably comfortable with that development model?
 

is not hard. As long as you manage to stay on top of what you yourself are doing. Case in point: somewhere in the '90s, I organized a seminar for top-level Java programmers, in Lapland. In my spare time, I attended some of the more interesting lectures. One of the points I never will forget was, that an object instance, with its current state, may have several simultaneous threads executing its code -- within its current state. I didn't sleep the next night. But ever since, I've felt comfortable with MT.
 2) Assuming it were financially attractive, could you make what you 
 feel to be educated & valid assertions about how to successfully 
 migrate a 'traditional' application to take full advantage of the new
  breed of CPU's?

No. But that's mainly because I have a hard time understanding what one vs. two (or more) processors have to do with this. On the conceptual level, that is. Of course (if this were a job interview) I could start about spreading the work load between a "ui process" and a "controller process" or some such BS. But from a programmer's point of view, you either have MT or you don't. D has it right in the language, and even if it weren't, you could rather easily implement cooperative MT yourself, in D. Of course, this isn't preemptive multitasking, but that was not asked for. Then again, preemptive multitasking, as presented by the operating system, is heavy compared with (self-made) cooperative multitasking. Therefore, they should be used in dissimilar contexts. --- As to "migrate a 'traditional' application to take full advantage of the new breed of CPUs", I'd like to be as arrogant as to say, the issue is way broader than can be asked in one sentence. It's not like there's a known (or even unknown) receipe on how to do it. The 'traditional' applications are diverse enough to preclude any single method of conversion. (Examples: A traditional app to count vehicles on the Bay Bridge, in real time, getting input only from an under-road inductance loop, and differentiating between regular cars and utility vehicles. A bank back-room application monitoring input and output of cashiers' bought and sold notes in a currency exchange office. Any old mainframe app designed to talk with hundreds of VT-100 terminals, converted to be used with Internet browsers instead. (Or was it a new in-between server, serving the browsers and appearing to the mainframe as hundreds of VT terminals?))
 3) Do you know anyone else who could handle #2? How many others?

Disregarding my own answer to #2, and assuming I guess what you were asking, my answer would be 3-6 people I know. But then, they aren't exactly average programmers either. ---- So where does this leave us? I think you might rephrase the question. First of all, to more specific, and secondly, more to what you're after. For example, early in your post, you mention "transactable execution". This naturally biases the answers towards database related thinking -- right? Which again raises the question: are we charting things that should be in the language, or are we charting what should "sort of" be included in "the eventual standard distribution of D"?
Oct 30 2005
next sibling parent Sean Kelly <sean f4.ca> writes:
Georg Wrede wrote:
 (PS: the tone here might appear aggressive, but wasn't meant as such.)
 
 Kris wrote:
 
 1) Do you have reasonable experience with the problems associated with 
 MT, and feel reasonably comfortable with that development model?

is not hard. As long as you manage to stay on top of what you yourself are doing.

I'd add deadlock resistance to the list of difficult MT problems. This by itself is one of the strongest arguments for lock-free concurrency IMO.
 2) Assuming it were financially attractive, could you make what you 
 feel to be educated & valid assertions about how to successfully 
 migrate a 'traditional' application to take full advantage of the new
  breed of CPU's?

No. But that's mainly because I have a hard time understanding what one vs. two (or more) processors have to do with this.

The hardware layer doesn't matter as such. The primary reason this is a concern with respect to new CPUs is that clock speed is beginning to plateau, so performance scaling is moving away from clock speed towards parallelization. This has been a growing concern among the C++ and MT communities for a while now.
 But from a programmer's point of view, you either have MT or you don't.
 D has it right in the language, and even if it weren't, you could rather
 easily implement cooperative MT yourself, in D. Of course, this isn't
 preemptive multitasking, but that was not asked for.

I agree with Kris that it's worth considering other language features to support or facilitate concurrent programming. While concurrency is a fairly well understood problem by some, the support for it in popular procedural languages is pretty weak. Java is better off than most, and it doesn't even go much further than "synchronized" and a memory model definition. D may do quite well to have some additional concurrent programming features available (similar to those in Cilk, for example).
 As to "migrate a 'traditional' application to take full advantage of the
 new breed of CPUs", I'd like to be as arrogant as to say, the issue is
 way broader than can be asked in one sentence. It's not like there's a
 known (or even unknown) receipe on how to do it. The 'traditional'
 applications are diverse enough to preclude any single method of
 conversion.

It would be nice if there were a fairly systematic way for doing this, but I agree that there may not be. The Intel compiler, for example, can parallelize some code blocks automatically, but this amounts to a very small portion of the code (only loops IIRC).
 Which again raises the question: are we charting things that should be
 in the language, or are we charting what should "sort of" be included in
 "the eventual standard distribution of D"?

A bit of both, I think. Sean
Oct 30 2005
prev sibling parent "Kris" <fu bar.com> writes:
Good points. Some comments inline:

"Georg Wrede" <georg.wrede nospam.org> ...
 (PS: the tone here might appear aggressive, but wasn't meant as such.)

 Kris wrote:
 1) Do you have reasonable experience with the problems associated with 
 MT, and feel reasonably comfortable with that development model?

is not hard. As long as you manage to stay on top of what you yourself are doing. <snip>

That's a large part of what the linked article was getting at. If a compiler (or other tool) could help more in this regard it could be a big win. The Concur project at MS is apparently hoping to do just that. The write-conflict coloring I mentioned earlier would be an example of such a win: it would identify all possible read/write & write/write conflicts, thereby supporting a developer in their quest to "stay on top of things" (as you put it so well).
 2) Assuming it were financially attractive, could you make what you feel 
 to be educated & valid assertions about how to successfully migrate a 
 'traditional' application to take full advantage of the new
  breed of CPU's?

No. But that's mainly because I have a hard time understanding what one vs. two (or more) processors have to do with this.

Only that to take full advantage of the lastest CPUs, one likely has to become familiar with MT. Whereas before, that was rarely a concern for a large swathe of application types (and programmers). <snip>
 As to "migrate a 'traditional' application to take full advantage of the
 new breed of CPUs", I'd like to be as arrogant as to say, the issue is
 way broader than can be asked in one sentence. It's not like there's a
 known (or even unknown) receipe on how to do it. The 'traditional'
 applications are diverse enough to preclude any single method of
 conversion.

Yes; you are quite right. I should have been more specific, and narrow the scope to those applications that stood to gain most from an ever-increasing single-core CPU throughput (MHz is one vector). Such apps rarely deal with MT issues directly (except for the occasional "execute later" API present in certain GUIs). Now that the performance gains are slanted more towards multi-core, developers will perhaps /have/ to face the MT hurdle. Exactly what that linked article was getting at. On the other hand, I wanted to leave the question open to interpretation. There's many ways to skin the MT cat, as you note; but only those with prior experience would likely be familiar with the concern vectors.
 3) Do you know anyone else who could handle #2? How many others?

Disregarding my own answer to #2, and assuming I guess what you were asking, my answer would be 3-6 people I know. But then, they aren't exactly average programmers either.

I think that last sentence identifies an important point. If D could help those 'average' programmers succeed by, for example, pointing out in the code where there might be trouble, then it would stand head-and-shoulders above any other language in this respect (and I'm guessing that this kind of thing is an aspect of the Concur project?). <snip>
 I think you might rephrase the question. First of all, to more specific,
 and secondly, more to what you're after. For example, early in your
 post, you mention "transactable execution". This naturally biases the
 answers towards database related thinking -- right?

By "transactable execution" I was reflecting the ability to roll-back the effect of execution ~ any execution ~ within certain contraints. There's been a lot of thought in that area over the last 30 years (or whatever) since it would allow parallel /speculative/ execution at a much higher/large-grained level than is currently possible at the die/branch level. Traditionally, that kind of thing has been built into DBs to gain ACID properties (atomic, consistent, isolated, durable), but it has other advantages too. Still, that's all rather speculative at this point, and doesn't really have much bearing on the diagnostic type of tools I'd like to see. I mentioned it only as reference to the links posted by Sean.
 Which again raises the question: are we charting things that should be
 in the language, or are we charting what should "sort of" be included in
 "the eventual standard distribution of D"?

Specifically, I was hoping to concentrate on what language attributes (as opposed to library attributes) could help developers in the MT world. Being able to extract features from the code (such as parallel execution) can lead to the coloring example I mentioned. Again, this is just speculation. I have several other half-baked notions in mind, but they don't really belong in this "poll" thread so I'll just briefly mention a couple of them: a) A little more than the simple (yet powerful) 'synchronized' keyword; perhaps some of the CS constructs that Doug Lea implements & documents so well? Having some related extractable features at the language level would likely aid diagnostics in a similar manner to a PAR type construct, since they tend to go hand-in-hand. b) A multi-instance GC. Single-pool allocators, and a GC in particular, can be a serious bottleneck for multi-threaded apps. More so than single-threaded since there's now contention and stalling in the allocator, and often severe contention in the collector (i.e. halts all running threads). I tend to avoid allocations at all cost within a server; precisely for this reason (e.g. the Mango HTTP and Servlet servers do not allocate any per-request memory, once they have warmed up). Thanks for the feedback, Georg.
Oct 30 2005
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Kris wrote:
 Thanks for the links, Sean. That forum had some interesting comments ~ 
 especially regarding transactable execution.

I think so too. For anyone interested, the post that made me think of the thread in the first place is here (154. Andy Glew): http://groups.google.com/group/comp.arch/browse_frm/thread/3d2139bf82f6298e/92c0dbef8aaac3b4#92c0dbef8aaac3b4 It does a rather good job of summing up most of the issues. Now on to the poll!
 1) Do you have reasonable experience with the problems associated with MT, 
 and feel reasonably comfortable with that development model?

Yes.
 2) Assuming it were financially attractive, could you make what you feel to 
 be educated & valid assertions about how to successfully migrate a 
 'traditional' application to take full advantage of the new breed of CPU's?

I believe so, though I'd have to think about whether I could suggest a mechanical approach for doing so without being overly general.
 3) Do you know anyone else who could handle #2? How many others?

Of my current coworkers, I can think of of one person. I haven't worked with a terribly large number of people who have much MT programming experience or who know architecture issues well enough to compensate. Sean
Oct 31 2005
parent Georg Wrede <georg.wrede nospam.org> writes:
I took a peek at Cilk. The academic research behind it seems quite 
solid, and the additions to C are kept to a minimum.

I wonder, has any of you guys looked at the preprocessor? Would it be 
hard to convert it to work on D code?
Oct 31 2005