www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Bug? taskPool.map() with bufSize and writeln() gets stuck

reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
Ubuntu 11.10 64-bit dmd.

The following program gets stuck during the writeln() call.

- The foo() call alone works fine.

- The program works fine when there is no writeln() call nor foo() call. 
All elements get processed in that case and the results are ignored.

Am I using taskPool.map incorrectly or is this a bug?

import std.stdio;
import std.parallelism;
import core.thread;

int func(int i)
{
     writeln("processing ", i);
     return i;
}

void main()
{
     auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);

     writeln(results);  // <-- Gets stuck

     foo(results); // this works fine
}

void foo(R)(R range)
{
     for ( ; !range.empty; range.popFront()) {
         writeln(range.front);
     }
}

Thank you,
Ali
Feb 10 2012
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/10/2012 10:28 AM, Ali Çehreli wrote:
 Ubuntu 11.10 64-bit dmd.

 The following program gets stuck during the writeln() call.

 - The foo() call alone works fine.

 - The program works fine when there is no writeln() call nor foo() call.
 All elements get processed in that case and the results are ignored.

 Am I using taskPool.map incorrectly or is this a bug?

 import std.stdio;
 import std.parallelism;
 import core.thread;

 int func(int i)
 {
 writeln("processing ", i);
 return i;
 }

 void main()
 {
 auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);

 writeln(results); // <-- Gets stuck

 foo(results); // this works fine
 }

 void foo(R)(R range)
 {
 for ( ; !range.empty; range.popFront()) {
 writeln(range.front);
 }
 }

 Thank you,
 Ali

I have asked the same question on the main D newsgroup. It turns out, since the writeln() calls in func() and main() share the same resource (namely stdout), the main thread and the task threads get dead-locked on an output synchronization lock. Changing the call in main to be on stderr fixes the issue and I get what I want. Now I can see how taskPool.map works semi-lazily: import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); Thread.sleep(dur!"seconds"(1)); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); stderr.writeln(results); // <-- now on stderr } The output: processing 1 processing 2 [1, 2processing 3 processing 4 , 3, 4processing 5 processing 6 , 5, 6processing 7 processing 8 , 7, 8] std.parallelism is only for when the tasks are truly independent from each other. I had forgotten that using stdout would violate that condition. Ali
Feb 11 2012