www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Why is amap implemented as a member function of TaskPool?

reply "Atila Neves" <atila.neves gmail.com> writes:
Or what I really want to ask: why can't I call amap from 
std.parallelism with a lambda? I assume it's because it's a 
member function but I'm not 100% sure.

I hardly ever call map with a named function (named local 
functions don't work with TaskPool.amap either), it's always a 
closure. Not just in D, pretty much in any language I write code 
in.

I had to roll my own parallel map today, but at least I did get a 
nice 3x speedup.

Atila
Sep 18 2014
next sibling parent reply "Jared" <jared economicmodeling.com> writes:
On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
 Or what I really want to ask: why can't I call amap from 
 std.parallelism with a lambda? I assume it's because it's a 
 member function but I'm not 100% sure.

 Atila
You have to tell DMD that the lambda is not in fact a delegate. import std.stdio; import std.range; import std.parallelism; void main() { auto w = iota(0,1_000_000); int[] foo; // Not OK, dmd can't infer lambda isn't a delegate // foo = taskPool().amap!(a => a + 1)(w); // OK: foo = taskPool().amap!`a+1`(w); // string lambdas, yeah! foo = taskPool().amap!(function int(int a) => a + 1)(w); static int func(int a) { return a + 1; } foo = taskPool().amap!func(w); }
Sep 18 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
The point is I _want_ a delegate.

Atila

On Thursday, 18 September 2014 at 20:51:30 UTC, Jared wrote:
 On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves 
 wrote:
 Or what I really want to ask: why can't I call amap from 
 std.parallelism with a lambda? I assume it's because it's a 
 member function but I'm not 100% sure.

 Atila
You have to tell DMD that the lambda is not in fact a delegate. import std.stdio; import std.range; import std.parallelism; void main() { auto w = iota(0,1_000_000); int[] foo; // Not OK, dmd can't infer lambda isn't a delegate // foo = taskPool().amap!(a => a + 1)(w); // OK: foo = taskPool().amap!`a+1`(w); // string lambdas, yeah! foo = taskPool().amap!(function int(int a) => a + 1)(w); static int func(int a) { return a + 1; } foo = taskPool().amap!func(w); }
Sep 19 2014
parent "Jared" <jared economicmodeling.com> writes:
On Friday, 19 September 2014 at 07:17:50 UTC, Atila Neves wrote:
 The point is I _want_ a delegate.

 Atila

 On Thursday, 18 September 2014 at 20:51:30 UTC, Jared wrote:
 On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves 
 wrote:
 Or what I really want to ask: why can't I call amap from 
 std.parallelism with a lambda? I assume it's because it's a 
 member function but I'm not 100% sure.
Check out this: https://issues.dlang.org/show_bug.cgi?id=5710 It seems that the compiler doesn't know what to do with non-global templates and delegates since the delegate would end up with two context pointers. Looks like there are efforts to fix it.
Sep 19 2014
prev sibling parent reply =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
 I had to roll my own parallel map today, but at least I did get 
 a nice 3x speedup.
Is your own parallel map public somewhere? It would be interesting to see it.
Sep 19 2014
next sibling parent reply Russel Winder via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:
On Sat, 2014-09-20 at 06:46 +0000, "Nordlöw" via Digitalmars-d-learn
wrote:
 On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves wrote:
 I had to roll my own parallel map today, but at least I did get 
 a nice 3x speedup.
How many cores? Is the problem a data parallel one and hence should show linear speedup?
 Is your own parallel map public somewhere? It would be 
 interesting to see it.
Particularly if it can be used to improved the code in std.parallelism. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Sep 20 2014
parent "Atila Neves" <atila.neves gmail.com> writes:
On Saturday, 20 September 2014 at 07:25:45 UTC, Russel Winder via 
Digitalmars-d-learn wrote:
 On Sat, 2014-09-20 at 06:46 +0000, "Nordlöw" via 
 Digitalmars-d-learn
 wrote:
 On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves 
 wrote:
 I had to roll my own parallel map today, but at least I did 
 get a nice 3x speedup.
How many cores? Is the problem a data parallel one and hence should show linear speedup?
It depends on the data. I was running analysis on a build dependency graph. It'd depend on the dependency tree topology. I have 4 cores with hyperthreading. 3x speedup seems good to me, especially since the implementation is recursive.
 Is your own parallel map public somewhere? It would be 
 interesting to see it.
Particularly if it can be used to improved the code in std.parallelism.
Now that I looked at the bug id referenced above, I don't think it'd help. I can see all sorts of problems with guaranteeing no mutable references are captured by the delegate. Atila
Sep 22 2014
prev sibling parent "Atila Neves" <atila.neves gmail.com> writes:
On Saturday, 20 September 2014 at 06:46:43 UTC, Nordlöw wrote:
 On Thursday, 18 September 2014 at 19:49:00 UTC, Atila Neves 
 wrote:
 I had to roll my own parallel map today, but at least I did 
 get a nice 3x speedup.
Is your own parallel map public somewhere? It would be interesting to see it.
I just did the simplest, stupidest thing that would work, so it's probably buggy. It works where I used it (and was faster) so that's all I needed to know. To even think of releasing this I'd use atomics instead of the mutex and try to break it in all sorts of ways. But here it is anyway: private auto pmap(alias fun, R)(R range) if(isInputRange!R) { import std.parallelism; import core.sync.mutex; static __gshared Mutex mutex; if(mutex is null) mutex = new Mutex; typeof(fun(range.front))[] values; foreach(i, value; range.parallel) { auto newValue = fun(value); synchronized(mutex) { if(values.length < i + 1) values.length = i + 1; values[i] = newValue; } } return values; } Oh, and the reason I don't just append to `values` is that I need to preserve the original order. Atila
Sep 22 2014