www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Are spawn'ed threads waited automatically?

reply =?ISO-8859-1?Q?Ali_=C7ehreli?= <acehreli yahoo.com> writes:
First, the answer may be as simple as "use core.thread.thread_joinAll". 
Is that the proper way of waiting for all threads?

Second, my question may not be a valid example as starting a thread 
without communicating with it may be under the umbrella of 
parallelization. Maybe in concurrency, threads communicate with each 
other so that the following situation should not occur in practice.

Third is my question: :) If I spawn a single thread in main, the single 
thread seems to run to completion.

import std.stdio;
import std.concurrency;
import core.thread;

void foo()
{
     foreach (i; 0 .. 5) {
         Thread.sleep(dur!"msecs"(500));
         writeln(i, " foo");
     }
}

void main()
{
     spawn(&foo);
     writeln("main done");
}

I get all of foo's output after "main done":

main done
0 foo
1 foo
2 foo
3 foo
4 foo

If I introduce an intermediate thread that spawns the foo thread, now 
foo sometimes terminates early:

import std.stdio;
import std.concurrency;
import core.thread;

void foo()
{
     foreach (i; 0 .. 5) {
         Thread.sleep(dur!"msecs"(500));
         writeln(i, " foo");
     }
}

void intermediate()
{
     spawn(&foo);
     writeln("intermediate done");
}

void main()
{
     spawn(&intermediate);
     writeln("main done");
}

The output is inconsistent. Sometimes there is nothing from foo:

main done
intermediate done

Sometimes it runs fully:

main done
intermediate done
0 foo
1 foo
2 foo
3 foo
4 foo

Is the inconsistency a bug or a natural consequence of something? :) 
(Perhaps even the first example that seems to run correctly just has a 
higher probability of showing this behavior.)

I am aware of thread_joinAll(). Is that the recommended way of waiting 
for all threads?

Thank you,
Ali
Jun 06 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli yahoo.com> wrote:

 First, the answer may be as simple as "use core.thread.thread_joinAll".  
 Is that the proper way of waiting for all threads?

main (the C main, not D main) does this already: https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L512 But note that "daemonized" threads will not be included: http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon -Steve
Jun 06 2011
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/06/2011 12:07 PM, Steven Schveighoffer wrote:
 On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli yahoo.com> wrote:

 First, the answer may be as simple as "use
 core.thread.thread_joinAll". Is that the proper way of waiting for all
 threads?

main (the C main, not D main) does this already: https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dmain2.d#L512 But note that "daemonized" threads will not be included: http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon -Steve

Thank you but it doesn't explain the inconsistent behavior. It seems like thread_joinAll() has a different idea at different times. Now I also print the result of isDaemon(): import std.stdio; import std.concurrency; import core.thread; void tell_daemon_state(string name) { writeln(name, " isDaemon: ", Thread.getThis.isDaemon); } void foo() { tell_daemon_state("foo"); foreach (i; 0 .. 5) { Thread.sleep(dur!"msecs"(500)); writeln(i, " foo"); } } void intermediate() { tell_daemon_state("intermediate"); spawn(&foo); writeln("intermediate done"); } void main() { tell_daemon_state("main"); spawn(&intermediate); writeln("main done"); } I see that only the main thread is a daemon: main isDaemon: true main done intermediate isDaemon: false intermediate done foo isDaemon: false 0 foo 1 foo 2 foo 3 foo 4 foo That makes sense. There is a race condition: Just because I added the printing of the isDaemon state, now foo() runs to completion seemingly everytime I start the program. When I remove the printing AND run the program under 'time', I get inconsistent behavior. The following are two consecutive runs: $ time ./deneme main done intermediate done 0 foo <--- foo()'s output is present 1 foo 2 foo 3 foo 4 foo real 0m2.504s user 0m0.000s sys 0m0.000s $ time ./deneme main done intermediate done <--- foo()'s output is missing real 0m0.003s user 0m0.000s sys 0m0.000s As if thread_joinAll() misses the fact that there is still the non-daemon foo() thread. Note that it's not failing to flush stdout either. The program runs shorter in the case where foo()'s output is missing. Thank you, Ali
Jun 06 2011
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/06/2011 03:52 PM, Jonathan M Davis wrote:

 At this point, I don't trust spawn at all (on Linux at least). I've 

 many problems with it.

Thank you. I've spawned threads from within threads and now received ThreadException and segmentation faults as well: http://d.puremagic.com/issues/show_bug.cgi?id=6116 Ali
Jun 06 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On 2011-06-06 14:37, Ali Çehreli wrote:
 On 06/06/2011 12:07 PM, Steven Schveighoffer wrote:
 On Mon, 06 Jun 2011 14:09:25 -0400, Ali Çehreli <acehreli yahoo.com> 


 First, the answer may be as simple as "use
 core.thread.thread_joinAll". Is that the proper way of waiting for all
 threads?

main (the C main, not D main) does this already: https://github.com/D-Programming-Language/druntime/blob/master/src/rt/dma in2.d#L512 But note that "daemonized" threads will not be included: http://www.digitalmars.com/d/2.0/phobos/core_thread.html#isDaemon -Steve

Thank you but it doesn't explain the inconsistent behavior. It seems like thread_joinAll() has a different idea at different times. Now I also print the result of isDaemon(): import std.stdio; import std.concurrency; import core.thread; void tell_daemon_state(string name) { writeln(name, " isDaemon: ", Thread.getThis.isDaemon); } void foo() { tell_daemon_state("foo"); foreach (i; 0 .. 5) { Thread.sleep(dur!"msecs"(500)); writeln(i, " foo"); } } void intermediate() { tell_daemon_state("intermediate"); spawn(&foo); writeln("intermediate done"); } void main() { tell_daemon_state("main"); spawn(&intermediate); writeln("main done"); } I see that only the main thread is a daemon: main isDaemon: true main done intermediate isDaemon: false intermediate done foo isDaemon: false 0 foo 1 foo 2 foo 3 foo 4 foo That makes sense. There is a race condition: Just because I added the printing of the isDaemon state, now foo() runs to completion seemingly everytime I start the program. When I remove the printing AND run the program under 'time', I get inconsistent behavior. The following are two consecutive runs: $ time ./deneme main done intermediate done 0 foo <--- foo()'s output is present 1 foo 2 foo 3 foo 4 foo real 0m2.504s user 0m0.000s sys 0m0.000s $ time ./deneme main done intermediate done <--- foo()'s output is missing real 0m0.003s user 0m0.000s sys 0m0.000s As if thread_joinAll() misses the fact that there is still the non-daemon foo() thread. Note that it's not failing to flush stdout either. The program runs shorter in the case where foo()'s output is missing.

Unless the code has changed (and Sean was working on it a couple of months back, so I'm not sure what the current state is), on Linux, none of the spawned threads ever get joined, and they're all joinable - which causes problems. As I understand it, they should all be detached (as in the pthread concept of detached, not detached from the GC like core.Thread talks about) rather than joinable. At this point, I don't trust spawn at all (on Linux at least). I've had too many problems with it. But I don't know what the current state is, because Sean was at least working on improving the situation. It's possible that the joinable issues and whatnot were worked out, but I don't know and kind of doubt it. Regardless, spawned threads aren't intended to be joined by you. They should run until they're done doing whatever they're doing and then exit. And the program should wait for them all to exit, even if main finishes. If that's not happening, then there are bugs that need to be fixed. You shouldn't ever have to worry about joining spawned threads. - Jonathan M Davis
Jun 06 2011