digitalmars.D.learn - Idiomatic way of writing nested loops?
- Anton Fediushin (30/30) Jul 17 2017 Hello! What is the best way of rewriting this code in idiomatic D
- Sebastiaan Koppe (2/4) Jul 17 2017 https://dlang.org/phobos/std_algorithm_setops.html#.cartesianProduct
- Anton Fediushin (3/7) Jul 17 2017 Thank you! I knew it is in the library! So, `parallel` will work
- Sebastiaan Koppe (2/4) Jul 17 2017 Yes
- Nicholas Wilson (4/35) Jul 17 2017 With regards to parallel, only use it on the outermost loop.
- Anton Fediushin (3/7) Jul 18 2017 Thank you! Yes, `parallel` runs only 4 threads on my machine, so
- Russel Winder via Digitalmars-d-learn (19/27) Jul 18 2017 But how many processors, cache sizes, memory speed, etc, etc, etc. The o...
- Russel Winder via Digitalmars-d-learn (32/57) Jul 18 2017 It is important to separate threads and tasks carefully here: as far as ...
Hello! What is the best way of rewriting this code in idiomatic D
manner?
------
foreach(a; ["foo", "bar"]) {
foreach(b; ["baz", "foz", "bof"]) {
foreach(c; ["FOO", "BAR"]) {
// Some operations on a, b and c
}
}
}
------
Every array has at least 1 element, and adding/removing new
"nested loops" should be as easy as possible.
Also, I have a question about running this in parallel: if I want
to use nested loops with `parallel` from `std.parallelism`,
should I add `parallel` to every loop like this?
------
foreach(a; ["foo", "bar"].parallel) {
foreach(b; ["baz", "foz", "bof"].parallel) {
foreach(c; ["FOO", "BAR"].parallel) {
// Some operations on a, b and c
}
}
}
------
I am worried about running thousands of threads, because in this
case first `parallel` runs 2 tasks, every task runs 3 tasks and
every task runned inside a task runs 2 more tasks.
So, how to write this in idiomatic D manner and run it _if
possible_ in parallel?
Jul 17 2017
On Monday, 17 July 2017 at 11:07:35 UTC, Anton Fediushin wrote:Hello! What is the best way of rewriting this code in idiomatic D manner?https://dlang.org/phobos/std_algorithm_setops.html#.cartesianProduct
Jul 17 2017
On Monday, 17 July 2017 at 11:32:45 UTC, Sebastiaan Koppe wrote:On Monday, 17 July 2017 at 11:07:35 UTC, Anton Fediushin wrote:Thank you! I knew it is in the library! So, `parallel` will work just fine with this function, isn't it?Hello! What is the best way of rewriting this code in idiomatic D manner?https://dlang.org/phobos/std_algorithm_setops.html#.cartesianProduct
Jul 17 2017
Sebastiaan Koppe <mail skoppe.eu> On Monday, 17 July 2017 at 11:55:47 UTC, Anton Fediushin wrote:Thank you! I knew it is in the library! So, `parallel` will work just fine with this function, isn't it?Yes
Jul 17 2017
On Monday, 17 July 2017 at 11:07:35 UTC, Anton Fediushin wrote:
Hello! What is the best way of rewriting this code in idiomatic
D manner?
------
foreach(a; ["foo", "bar"]) {
foreach(b; ["baz", "foz", "bof"]) {
foreach(c; ["FOO", "BAR"]) {
// Some operations on a, b and c
}
}
}
------
Every array has at least 1 element, and adding/removing new
"nested loops" should be as easy as possible.
Also, I have a question about running this in parallel: if I
want to use nested loops with `parallel` from
`std.parallelism`, should I add `parallel` to every loop like
this?
------
foreach(a; ["foo", "bar"].parallel) {
foreach(b; ["baz", "foz", "bof"].parallel) {
foreach(c; ["FOO", "BAR"].parallel) {
// Some operations on a, b and c
}
}
}
------
I am worried about running thousands of threads, because in
this case first `parallel` runs 2 tasks, every task runs 3
tasks and every task runned inside a task runs 2 more tasks.
So, how to write this in idiomatic D manner and run it _if
possible_ in parallel?
With regards to parallel, only use it on the outermost loop.
Assuming you have more items in the outermost loop than you do
threads parallelising more than one loop won't net you any speed.
Jul 17 2017
On Tuesday, 18 July 2017 at 03:36:04 UTC, Nicholas Wilson wrote:With regards to parallel, only use it on the outermost loop. Assuming you have more items in the outermost loop than you do threads parallelising more than one loop won't net you any speed.Thank you! Yes, `parallel` runs only 4 threads on my machine, so there is no reason to use it in nested loops.
Jul 18 2017
On Tue, 2017-07-18 at 08:41 +0000, Anton Fediushin via Digitalmars-d-learn wrote:On Tuesday, 18 July 2017 at 03:36:04 UTC, Nicholas Wilson wrote:But how many processors, cache sizes, memory speed, etc, etc, etc. The only way of knowing what the fastest performance is is to try some things and ge= t some performance data. Even then the result only applies to that data on that computer. This is the sort of situation where philosophising about performance often ends up with totally the wrong code. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t:+44 20 7585 2200 voip:sip: russel.winder ekiga.net 41 Buckmaster Road m:+44 7770 465 077 xmpp:russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype:russel_winderWith regards to parallel, only use it on the outermost loop.=20 Assuming you have more items in the outermost loop than you do=20 threads parallelising more than one loop won't net you any=20 speed.=20 Thank you! Yes, `parallel` runs only 4 threads on my machine, so=20 there is no reason to use it in nested loops.
Jul 18 2017
On Tue, 2017-07-18 at 03:36 +0000, Nicholas Wilson via Digitalmars-d-learn wrote:On Monday, 17 July 2017 at 11:07:35 UTC, Anton Fediushin wrote:It is important to separate threads and tasks carefully here: as far as I a= m aware the .parallel creates tasks not threads. The only threads are the one= s in the thread pool animatng the tasks. This having the thousands of tasks i= s not a problem per se, since these are not threads. The question of what the best decomposition for parallelism is has to be determined by benchmarking =E2=80=93 guesswork usually gets it wrong. My prejudice here though is that for a loop structure such as this, unless the computation at the centre is a biggy, you probably don't want the .parallel on the inner loop. But I repeat only benchmarking will tell what the best parallelism decomposition is.[=E2=80=A6] =20 Also, I have a question about running this in parallel: if I=20 want to use nested loops with `parallel` from=20 `std.parallelism`, should I add `parallel` to every loop like=20 this? ------ foreach(a; ["foo", "bar"].parallel) { foreach(b; ["baz", "foz", "bof"].parallel) { foreach(c; ["FOO", "BAR"].parallel) { // Some operations on a, b and c } } } ------ I am worried about running thousands of threads, because in=20 this case first `parallel` runs 2 tasks, every task runs 3=20 tasks and every task runned inside a task runs 2 more tasks.I am not convinced by this "idiom" of only the outer loop. It may be true for some cases, but certtainly not all. This is task and thread pool based parallelism here, not vector parallelism. Without knowing the actual computational structure of the statements at the centre, there can be no known best parallelism structure. Experimentation on medium sized data sets before moving to the real ones is required to get the likely best performance. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t:+44 20 7585 2200 voip:sip: russel.winder ekiga.net 41 Buckmaster Road m:+44 7770 465 077 xmpp:russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype:russel_winderSo, how to write this in idiomatic D manner and run it _if=20 possible_ in parallel?=20 With regards to parallel, only use it on the outermost loop.=20 Assuming you have more items in the outermost loop than you do=20 threads parallelising more than one loop won't net you any speed.
Jul 18 2017