www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - LinkTerminated without linked thread?

reply "Stephan" <stephan_schiffels mac.com> writes:
Hi,

I am getting totally confused by Exceptions and Multi-threading 
with std.concurrency.
I have a simple setup:
     - the main thread spawns several child-threads
     - each child-thread performs a task
     - each child-thread sends their result back to the main 
thread and finishes.
     - the main thread uses receiveOnly to get the message.

To spawn the child-threads I use spawn, not spawnLinked. 
Nevertheless, in all runs, at some point the program finishes 
with an uncaught LinkTerminated exception!
I am confused, because I thought, that LinkTerminated can only 
happen if the child-threads are started with spawnLinked. In 
small test programs, I don't get LinkTerminated exceptions if a 
child thread crashes. So my first question is: Where does the 
LinkTerminated Exception come from, when the only functions from 
std.concurrency that I use are "spawn" and "receiveOnly".

My second question: Given that spawn works as documented and 
expected: How can I report an Error or an Exception thrown in a 
child-thread? Should I wrap the whole function that is spawned in 
a try-catch statement to print any exception manually?

Thanks for any clarification!

Stephan
Jan 24 2013
next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:

 Hi,

 I am getting totally confused by Exceptions and Multi-threading 
 with std.concurrency.
 I have a simple setup:
     - the main thread spawns several child-threads
     - each child-thread performs a task
     - each child-thread sends their result back to the main 
 thread and finishes.
     - the main thread uses receiveOnly to get the message.

 To spawn the child-threads I use spawn, not spawnLinked. 
 Nevertheless, in all runs, at some point the program finishes 
 with an uncaught LinkTerminated exception!
 I am confused, because I thought, that LinkTerminated can only 
 happen if the child-threads are started with spawnLinked. In 
 small test programs, I don't get LinkTerminated exceptions if a 
 child thread crashes. So my first question is: Where does the 
 LinkTerminated Exception come from, when the only functions 
 from std.concurrency that I use are "spawn" and "receiveOnly".
 Thanks for any clarification!


I have had the same problem, especially when creating large 
numbers of threads (several hundred)
Jan 24 2013
prev sibling next sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:

 So my first question is: Where does the LinkTerminated 
 Exception come from, when the only functions from 
 std.concurrency that I use are "spawn" and "receiveOnly".


To track this down, what about looking at the std.concurrency 
source and placing a breakpoint at the appropriate line in 
MessageBox.get()?

David
Jan 24 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger 
wrote:

 On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:

 So my first question is: Where does the LinkTerminated 
 Exception come from, when the only functions from 
 std.concurrency that I use are "spawn" and "receiveOnly".


 To track this down, what about looking at the std.concurrency 
 source and placing a breakpoint at the appropriate line in 
 MessageBox.get()?

 David


Yeah, I've been playing with std.concurrency too lately, and 
getting random bugs. I'll try to formalize them into formal 
requests.

BTW: Question: Is "LinkTerminated" a priority message? I've been 
avoiding using SpawnLinked because I've been unable to 
effectively end my workers. Maybe I just suck though...
Jan 24 2013
parent reply Sean Kelly <sean invisibleduck.org> writes:
On Jan 24, 2013, at 5:58 AM, "monarch_dodra" <monarchdodra gmail.com> wrote:=



 On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger wrote:

 On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:

 So my first question is: Where does the LinkTerminated Exception come fr=






om, when the only functions from std.concurrency that I use are "spawn" and "=
receiveOnly".

=20
 To track this down, what about looking at the std.concurrency source and p=




lacing a breakpoint at the appropriate line in MessageBox.get()?

=20
 David


=20
 Yeah, I've been playing with std.concurrency too lately, and getting rando=


m bugs. I'll try to formalize them into formal requests.

=20
 BTW: Question: Is "LinkTerminated" a priority message? I've been avoiding u=


sing SpawnLinked because I've been unable to effectively end my workers. May=
be I just suck though...

It's a control message. These are messages generated automatically by std.co=
ncurrency rather than sent by the user. They live in the normal message queu=
e and so are processed when receive doesn't find a match earlier in the queu=
e. The idea is that if an owner sends a spawned thread a bunch of messages, t=
he spawned thread should have an opportunity to process those messages befor=
e receiving an OwnerTerminated message.  Making them priority messages would=
 make designing predictable algorithms difficult.=20=
Jan 24 2013
next sibling parent reply "Stephan" <stephan_schiffels mac.com> writes:
On Thursday, 24 January 2013 at 17:12:49 UTC, Sean Kelly wrote:

 On Jan 24, 2013, at 5:58 AM, "monarch_dodra" 
 <monarchdodra gmail.com> wrote:


 On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger 
 wrote:

 On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:

 So my first question is: Where does the LinkTerminated 
 Exception come from, when the only functions from 
 std.concurrency that I use are "spawn" and "receiveOnly".


 
 To track this down, what about looking at the std.concurrency 
 source and placing a breakpoint at the appropriate line in 
 MessageBox.get()?
 
 David


 
 Yeah, I've been playing with std.concurrency too lately, and 
 getting random bugs. I'll try to formalize them into formal 
 requests.
 
 BTW: Question: Is "LinkTerminated" a priority message? I've 
 been avoiding using SpawnLinked because I've been unable to 
 effectively end my workers. Maybe I just suck though...


 It's a control message. These are messages generated 
 automatically by std.concurrency rather than sent by the user. 
 They live in the normal message queue and so are processed when 
 receive doesn't find a match earlier in the queue. The idea is 
 that if an owner sends a spawned thread a bunch of messages, 
 the spawned thread should have an opportunity to process those 
 messages before receiving an OwnerTerminated message.  Making 
 them priority messages would make designing predictable 
 algorithms difficult.


But still, LinkTerminated needs to be thrown somewhere. And 
without spawnLinked I can't see why it should be thrown anywhere.
I am trying to use a different setup now, with a fewer number of 
threads. Maybe that'll work.

Stephan
Jan 24 2013
parent Sean Kelly <sean invisibleduck.org> writes:
On Jan 24, 2013, at 9:58 AM, Stephan <stephan_schiffels mac.com> wrote:


 On Thursday, 24 January 2013 at 17:12:49 UTC, Sean Kelly wrote:

 On Jan 24, 2013, at 5:58 AM, "monarch_dodra" <monarchdodra gmail.com> =




wrote:

=20

 On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger wrote:

 On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:

 So my first question is: Where does the LinkTerminated Exception =










come from, when the only functions from std.concurrency that I use are =
"spawn" and "receiveOnly".

 To track this down, what about looking at the std.concurrency =








source and placing a breakpoint at the appropriate line in =
MessageBox.get()?

 David


 Yeah, I've been playing with std.concurrency too lately, and getting =






random bugs. I'll try to formalize them into formal requests.

 BTW: Question: Is "LinkTerminated" a priority message? I've been =






avoiding using SpawnLinked because I've been unable to effectively end =
my workers. Maybe I just suck though...

=20
 It's a control message. These are messages generated automatically by =




std.concurrency rather than sent by the user. They live in the normal =
message queue and so are processed when receive doesn't find a match =
earlier in the queue. The idea is that if an owner sends a spawned =
thread a bunch of messages, the spawned thread should have an =
opportunity to process those messages before receiving an =
OwnerTerminated message.  Making them priority messages would make =
designing predictable algorithms difficult.

=20
 But still, LinkTerminated needs to be thrown somewhere. And without =


spawnLinked I can't see why it should be thrown anywhere.

 I am trying to use a different setup now, with a fewer number of =


threads. Maybe that'll work.

It shouldn't be thrown unless you spawn with spawnLinked.  The code is =
pretty straightforward.  If you can come up with a repro of unexpected =
behavior, please file a bug report.=
Jan 24 2013
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Thursday, 24 January 2013 at 17:12:49 UTC, Sean Kelly wrote:

 On Jan 24, 2013, at 5:58 AM, "monarch_dodra" 
 <monarchdodra gmail.com> wrote:


 On Thursday, 24 January 2013 at 13:45:18 UTC, David Nadlinger 
 wrote:

 On Thursday, 24 January 2013 at 10:57:26 UTC, Stephan wrote:

 So my first question is: Where does the LinkTerminated 
 Exception come from, when the only functions from 
 std.concurrency that I use are "spawn" and "receiveOnly".


 
 To track this down, what about looking at the std.concurrency 
 source and placing a breakpoint at the appropriate line in 
 MessageBox.get()?
 
 David


 
 Yeah, I've been playing with std.concurrency too lately, and 
 getting random bugs. I'll try to formalize them into formal 
 requests.
 
 BTW: Question: Is "LinkTerminated" a priority message? I've 
 been avoiding using SpawnLinked because I've been unable to 
 effectively end my workers. Maybe I just suck though...


 It's a control message. These are messages generated 
 automatically by std.concurrency rather than sent by the user. 
 They live in the normal message queue and so are processed when 
 receive doesn't find a match earlier in the queue. The idea is 
 that if an owner sends a spawned thread a bunch of messages, 
 the spawned thread should have an opportunity to process those 
 messages before receiving an OwnerTerminated message.  Making 
 them priority messages would make designing predictable 
 algorithms difficult.


OK. TY.

I'll try to reproduce, but I'm 90% sure my reduced code was this:

//----
void worker(Tid owner)
{
   owner.send(1);
}

void main()
{
     spawnLinked(&worker, thisTid);
     receive(
         (int a){}
     );
}
//----
And it (50% of the time) terminated in a LinkTerminated exception 
being thrown.

Any quick thoughts? I'll investigate on my end depending on your 
explanation.
Jan 24 2013
parent reply "Stephan" <stephan_schiffels mac.com> writes:
On Friday, 25 January 2013 at 01:13:59 UTC, monarch_dodra wrote:

 I'll try to reproduce, but I'm 90% sure my reduced code was 
 this:

 //----
 void worker(Tid owner)
 {
   owner.send(1);
 }

 void main()
 {
     spawnLinked(&worker, thisTid);
     receive(
         (int a){}
     );
 }
 //----
 And it (50% of the time) terminated in a LinkTerminated 
 exception being thrown.

 Any quick thoughts? I'll investigate on my end depending on 
 your explanation.



Nope. That code works 100% for me.

Stephan
Jan 24 2013
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 25 January 2013 at 07:33:57 UTC, Stephan wrote:

 On Friday, 25 January 2013 at 01:13:59 UTC, monarch_dodra wrote:

 [SNIP]



 Nope. That code works 100% for me.

 Stephan


OK. I'm getting a better hang at the semantics of how message 
passing works.

I think I found I've identified the one of the issues I was 
running into though. I don't have much experience with MPI, so I 
don't know if this is a bug, or a user issue. In any case, I'm 
getting inconsistent behavior.

The basic premise of my problem is that one of my workers had a 
full mail box, while being set to "block". If it dies *while* the 
owner is blocked for sending, then the sender is blocked... 
forever.

//----
import std.concurrency, std.stdio;

void worker()
{
     thisTid.setMaxMailboxSize(10, OnCrowding.block);
     Thread.sleep(msecs(1000));
}
void main()
{
     auto wid = spawn(&worker);
     foreach ( i ; 0 .. 20 )
     {
         writeln(i);
         wid.send(i);
     }
}
//----

Basically, the worker sleeps for a second, the mailbox fills. The 
owner waits. The worker then dies, but the owner never wakes :(

The *inconsistent* behavior I'm getting though is that if I 
comment the "writeln(i)", then the program doesn't hang.

Say... what exactly happens when a message is sent to a dead 
thread? Seems like it just silently disappears...
Jan 25 2013
prev sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 01/24/2013 02:57 AM, Stephan wrote:


 How can I report an Error or an Exception thrown in a child-thread?
 Should I wrap the whole function that is spawned in a try-catch
 statement to print any exception manually?


You can send the exception as a message as well:

// at the worker:
     try {
         // ...

     } catch (shared(Exception) exc) {
         owner.send(exc);
     }},

// at the owner:
     receive(
         // ...

         (shared(Exception) exc) {
             // ...
         });

Ali
Jan 25 2013