digitalmars.D - Bug? taskPool.map() with bufSize and writeln() gets stuck
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> Feb 10 2012
- "MattCodr" <matheus_nab hotmail.com> Feb 10 2012
- "Martin Nowak" <dawg dawgfoto.de> Feb 11 2012
- "Martin Nowak" <dawg dawgfoto.de> Feb 11 2012
Sorry for the double-post; I have asked the same question on D.learn
earlier but I think this is more of a question to this forum.
Tested on Ubuntu 11.10 64-bit dmd.
The following program gets stuck during the writeln() call.
- Note that the foo() call alone works fine.
- Also note that the program works fine when there is no writeln() call
nor foo() call. All elements get processed in that case and the results
are ignored.
Am I using taskPool.map incorrectly or is this a bug? Can you help
identify where the problem may be? How is writeln() using the range
differently than foo() to cause this behavior?
import std.stdio;
import std.parallelism;
import core.thread;
int func(int i)
{
writeln("processing ", i);
return i;
}
void main()
{
auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
writeln(results); // <-- Gets stuck HERE
foo(results); // this works fine
}
void foo(R)(R range)
{
for ( ; !range.empty; range.popFront()) {
writeln(range.front);
}
}
Thank you,
Ali
Feb 10 2012
"MattCodr" <matheus_nab hotmail.com> On Saturday, 11 February 2012 at 01:31:29 UTC, Ali Çehreli wrote:Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum. Tested on Ubuntu 11.10 64-bit dmd. The following program gets stuck during the writeln() call. - Note that the foo() call alone works fine. - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored. Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior? import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); writeln(results); // <-- Gets stuck HERE foo(results); // this works fine } void foo(R)(R range) { for ( ; !range.empty; range.popFront()) { writeln(range.front); } } Thank you, Ali
For what I see, maybe that's because it's a "non-random access ranges". So to access "results", you need access from it's aggregate... try this inside your main: foreach(int r; results) writeln(r); // foo(results); // this works fin
Feb 10 2012
On Sat, 11 Feb 2012 02:31:29 +0100, Ali =C3=87ehreli <acehreli yahoo.com=wrote:
Sorry for the double-post; I have asked the same question on D.learn =
earlier but I think this is more of a question to this forum. Tested on Ubuntu 11.10 64-bit dmd. The following program gets stuck during the writeln() call. - Note that the foo() call alone works fine. - Also note that the program works fine when there is no writeln() cal=
nor foo() call. All elements get processed in that case and the result=
are ignored. Am I using taskPool.map incorrectly or is this a bug? Can you help =
identify where the problem may be? How is writeln() using the range =
differently than foo() to cause this behavior? import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); return i; } void main() { auto results =3D taskPool.map!func([1,2,3,4,5,6,7,8], 2); writeln(results); // <-- Gets stuck HERE foo(results); // this works fine } void foo(R)(R range) { for ( ; !range.empty; range.popFront()) { writeln(range.front); } } Thank you, Ali
Yeah, you have a deadlock in there, it's somewhat hidden though. The issue is that writeln will take lock on stdout once. This will deadl= ock with the lazy processing of the map range.
Feb 11 2012
On 02/11/2012 12:56 AM, Martin Nowak wrote:On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli yahoo.com>
Sorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum. Tested on Ubuntu 11.10 64-bit dmd. The following program gets stuck during the writeln() call. - Note that the foo() call alone works fine. - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored. Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior? import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); writeln(results); // <-- Gets stuck HERE foo(results); // this works fine } void foo(R)(R range) { for ( ; !range.empty; range.popFront()) { writeln(range.front); } } Thank you, Ali
Yeah, you have a deadlock in there, it's somewhat hidden though. The issue is that writeln will take lock on stdout once. This will
with the lazy processing of the map range.
Thank you. I was trying to visualize the semi-lazy nature of taskPool.map. Now I get what I want when the writeln() call in main() is changed to be on stderr: import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); Thread.sleep(dur!"seconds"(1)); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); stderr.writeln(results); // <-- now on stderr } Now the output hints at how taskPool.map is semi-lazy: processing 1 processing 2 [processing 3 1, 2processing 4 , 3processing 5 , 4processing 6 , 5, 6processing 7 processing 8 , 7, 8] Good. :) Ali
Feb 11 2012
On Sat, 11 Feb 2012 17:18:21 +0100, Ali =C3=87ehreli <acehreli yahoo.com=wrote:
On 02/11/2012 12:56 AM, Martin Nowak wrote: > On Sat, 11 Feb 2012 02:31:29 +0100, Ali =C3=87ehreli <acehreli yaho=
wrote: > >> Sorry for the double-post; I have asked the same question on D.lea=
>> earlier but I think this is more of a question to this forum. >> >> Tested on Ubuntu 11.10 64-bit dmd. >> >> The following program gets stuck during the writeln() call. >> >> - Note that the foo() call alone works fine. >> >> - Also note that the program works fine when there is no writeln()=
>> call nor foo() call. All elements get processed in that case and t=
>> results are ignored. >> >> Am I using taskPool.map incorrectly or is this a bug? Can you help=
>> identify where the problem may be? How is writeln() using the rang=
>> differently than foo() to cause this behavior? >> >> import std.stdio; >> import std.parallelism; >> import core.thread; >> >> int func(int i) >> { >> writeln("processing ", i); >> return i; >> } >> >> void main() >> { >> auto results =3D taskPool.map!func([1,2,3,4,5,6,7,8], 2); >> >> writeln(results); // <-- Gets stuck HERE >> >> foo(results); // this works fine >> } >> >> void foo(R)(R range) >> { >> for ( ; !range.empty; range.popFront()) { >> writeln(range.front); >> } >> } >> >> Thank you, >> Ali > > Yeah, you have a deadlock in there, it's somewhat hidden though. > The issue is that writeln will take lock on stdout once. This will =
deadlock > with the lazy processing of the map range. Thank you. I was trying to visualize the semi-lazy nature of taskPool.map. Now I =
get what I want when the writeln() call in main() is changed to be on =
stderr: import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); Thread.sleep(dur!"seconds"(1)); return i; } void main() { auto results =3D taskPool.map!func([1,2,3,4,5,6,7,8], 2); stderr.writeln(results); // <-- now on stderr } Now the output hints at how taskPool.map is semi-lazy: processing 1 processing 2 [processing 3 1, 2processing 4 , 3processing 5 , 4processing 6 , 5, 6processing 7 processing 8 , 7, 8] Good. :) Ali
right output.
Feb 11 2012