www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Short article on std.parallism

reply "Jeremy Wright" <jeremy codestrokes.com> writes:
I implemented bucket sort in D to demonstrate how easy it is to use 
std.parallelism.  I welcome any feedback.

http://www.codestrokes.com/archives/116

Jeremy Wright 
May 29 2011
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Very cool article. :)

Btw, you can omit 'auto' when you use 'immutable' declarations.
May 29 2011
prev sibling next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Sun, 29 May 2011 18:18:14 -0700, Jeremy Wright wrote:

 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.
 
 http://www.codestrokes.com/archives/116

Nice article. :) A tip to make the code even more terse: You can replace taskPool.parallel () with parallel(). The latter just forwards to the former. (See the last item of the std.parallelism docs.) -Lars
May 30 2011
prev sibling next sibling parent Johann MacDonagh <johann.macdonagh..no spam..gmail.com> writes:
On 5/29/2011 9:18 PM, Jeremy Wright wrote:
 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism. I welcome any feedback.

 http://www.codestrokes.com/archives/116

 Jeremy Wright

Nice. Just a few nit-picks: 1. End of first paragraph: "makes writing parallel code, correct." -> "makes writing parallel code correct." 2. Middle of second paragraph: "The tools are designed to find deadlocks, analyze timing". You wrote that line in the paragraph above. Seemed a little clunky for me. 3. The light green used in the chart and the D code is difficult to see on the white background. 4. Maybe a little comment just before the version(MultiThreaded) line to show readers that this is where the magic happens. Helps for readers new to D so they're not stumbling over the initial bucket creation code. 5. Maybe a comment at the end talking about std.parallel_algorithm. I suppose this wouldn't be a good idea if the idea gets dropped or radically changed, but even a little comment mentioning how std.parallelism will be wrapped for easy algorithm usage in the future would be a good selling point. Anyway, great article!
May 30 2011
prev sibling next sibling parent Graham Fawcett <fawcett uwindsor.ca> writes:
On Sun, 29 May 2011 18:18:14 -0700, Jeremy Wright wrote:

 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.
 
 http://www.codestrokes.com/archives/116

Haven't read it yet, but: "like many faucets" --> "like many facets" Best, Graham
May 30 2011
prev sibling next sibling parent "Jeremy Wright" <jeremy codestrokes.com> writes:
Wow! Thank you for your feedback. I'll work through your comments. I 
appreciate you all taking the time to read my article.

Jeremy Wright

"Jeremy Wright"  wrote in message news:irurgr$1g65$1 digitalmars.com...

I implemented bucket sort in D to demonstrate how easy it is to use
std.parallelism.  I welcome any feedback.

http://www.codestrokes.com/archives/116

Jeremy Wright 
May 30 2011
prev sibling next sibling parent "Jeremy Wright" <jeremy codestrokes.com> writes:
Wow! Thank you for your feedback. I'll work through your comments. I 
appreciate you all taking the time to read my article.

Jeremy Wright

"Jeremy Wright"  wrote in message news:irurgr$1g65$1 digitalmars.com...

I implemented bucket sort in D to demonstrate how easy it is to use
std.parallelism.  I welcome any feedback.

http://www.codestrokes.com/archives/116

Jeremy Wright 
May 30 2011
prev sibling next sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Mon, 30 May 2011 04:18:14 +0300, Jeremy Wright <jeremy codestrokes.com>  
wrote:

 I implemented bucket sort in D to demonstrate how easy it is to use  
 std.parallelism.  I welcome any feedback.

One thing: I would suggest to avoid using ~= in a tight loop, as it is rather slow. Using std.array.appender for the first loop and std.array.join for the second one should be much faster. -- Best regards, Vladimir mailto:vladimir thecybershadow.net
May 30 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/30/11, Vladimir Panteleev <vladimir thecybershadow.net> wrote:
 On Mon, 30 May 2011 04:18:14 +0300, Jeremy Wright <jeremy codestrokes.com>
 wrote:

 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.

One thing: I would suggest to avoid using ~= in a tight loop, as it is rather slow. Using std.array.appender for the first loop and std.array.join for the second one should be much faster. -- Best regards, Vladimir mailto:vladimir thecybershadow.net

I wonder why we even have this operator in the language if we're supposed to avoid it all the time.
May 30 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-30 22:52, Andrej Mitrovic wrote:
 On 5/30/11, Vladimir Panteleev <vladimir thecybershadow.net> wrote:
 On Mon, 30 May 2011 04:18:14 +0300, Jeremy Wright
 <jeremy codestrokes.com>
 
 wrote:
 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.

One thing: I would suggest to avoid using ~= in a tight loop, as it is rather slow. Using std.array.appender for the first loop and std.array.join for the second one should be much faster. -- Best regards, Vladimir mailto:vladimir thecybershadow.net

I wonder why we even have this operator in the language if we're supposed to avoid it all the time.

There's no problem with using it, generally-speaking. And it's actually faster than it used to be, so it's less of an issue. The problem really only lines with a tight loop. Extra checks have to be made with ~= to verify that reallocation isn't needed which don't have to be made with Appender, because it's able to make assumptions that ~= can't make. So, if you're in a loop that appends over and over for a large number of iterations, it's going to be faster to use Appender. Other than that, ~= is plenty fast enough. - Jonathan M Davis
May 30 2011
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Why doesn't Appender overload opCatAssign? It would be almost trivial
to replace usage of existing arrays with Appender, instead of having
to replace all calls with var.put().

And why doesn't it overload toString? You can't print its contents to
stdout like you can with slices.

And why can't you slice an Appender?

I see a lot of drawbacks with the only benefit being performance. Now,
if Appender had some syntax sugar to make it appear as if it were a
simple dynamic array (well, slice..), that would be sweet and would
encourage its use, at least with me. Otherwise it just looks ugly
compared to the sexy D arrays. It looks like everything is becoming a
template in a library these days..
May 30 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-30 23:21, Andrej Mitrovic wrote:
 Why doesn't Appender overload opCatAssign? It would be almost trivial
 to replace usage of existing arrays with Appender, instead of having
 to replace all calls with var.put().
 
 And why doesn't it overload toString? You can't print its contents to
 stdout like you can with slices.
 
 And why can't you slice an Appender?
 
 I see a lot of drawbacks with the only benefit being performance. Now,
 if Appender had some syntax sugar to make it appear as if it were a
 simple dynamic array (well, slice..), that would be sweet and would
 encourage its use, at least with me. Otherwise it just looks ugly
 compared to the sexy D arrays. It looks like everything is becoming a
 template in a library these days..

If you could slice an Appender, it would lose all of its guarantees. It can do what it does in part because it knows that it hasn't been sliced. If you were slicing it, then you'd have to worry about possible reallocations which loses all of the benefit of Appender in the first place. Appender only works because it's creating an array. If you tried to use it as a dynamic array in general, it wouldn't work. It's purely an optimization for creating arrays. ~= can't be made to do what Appender does, and if you tried to make Appender do what ~= does, then Appender couldn't do what it currently does. Appender has a very specific purpose and no one should be trying to replace dynamic arrays with it. It's just that if you want to get your code to be as fast as possible, there are cases where using Appender is a good idea. Beyond that, you shouldn't be using Appender. - Jonathan M Davis
May 30 2011
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 31 May 2011 02:21:13 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:

 Why doesn't Appender overload opCatAssign? It would be almost trivial
 to replace usage of existing arrays with Appender, instead of having
 to replace all calls with var.put().

It should, there might already be an enhancement filed for it.
 And why doesn't it overload toString? You can't print its contents to
 stdout like you can with slices.

 And why can't you slice an Appender?

writeln(app.data); auto slice = app.data[2...6]; -Steve
May 31 2011
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 31 May 2011 02:21:13 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:
 Why doesn't Appender overload opCatAssign? It would be almost trivial
 to replace usage of existing arrays with Appender, instead of having
 to replace all calls with var.put().

I've submitted a patch for an improved appender implementation. I didn't include '~=' as at the time, I thought it unnecessarily complicated dependent patch to std.range.put. ( See: http://d.puremagic.com/issues/show_bug.cgi?id=5813 and http://d.puremagic.com/issues/show_bug.cgi?id=5233 ) On review, I think it should be a trivial fix. And, as there were two enhancement requests in bugzilla for appender + "~=" (See Issue 4287), I will update my patches.
 And why doesn't it overload toString? You can't print its contents to
 stdout like you can with slices.

In the current appender? No reason. In my patched appender? Less trivial but doable. I've put it on my todo list.
 And why can't you slice an Appender?

What do you mean by a slice? An appender slice is non-nonsensical, since it can't append. If you mean slicing an appender to get a T[]; there is no technical reason why it can't be done now. But it's bad design. Supporting slicing, indexing and even length all expose too much of the underlying implementation. And array building utilities should be allowed to change their internal implementations. For example, in .Net 4.0, StringBuilder changed from a dynamic array to a linked list. And a linked list can't efficiently index nor slice. Regarding my patched appender, O(1) indexing and slicing is doable, but would require some additional memory overhead and would not be as fast as built-in arrays.
 I see a lot of drawbacks with the only benefit being performance.

It also lets you build immutable and const arrays like they were mutable. Oh, and for my patch that performance gain, in terms of free memory and performance is massive for large arrays.
 Now,
 if Appender had some syntax sugar to make it appear as if it were a
 simple dynamic array (well, slice..), that would be sweet and would
 encourage its use, at least with me. Otherwise it just looks ugly
 compared to the sexy D arrays. It looks like everything is becoming a
 template in a library these days..

A good array builder _isn't_ a simple array (dynamic or otherwise) and it shouldn't be used as such. Appender has one job and needs to do it well. Supporting other functions that have very poor performance is a sure way to whittle away all the performance gains using Appender gets you in the first place.
Jun 08 2011