www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Threading bugs

reply Tristam MacDonald <swiftcoder gmail.com> writes:
Content-Type: text/plain

So, I am sure that I must be missing something simple, that is causing my
multi-threading code in D fail miserably. I have cut it down to the smallest
code sample that exhibits the problem, and I would be grateful if someone could
have a look and let me know why...

import std.thread;
import std.stdio;

class Main
{
	this() {
		writefln("starting");

		Thread worker = new Thread(&workerMain);
		worker.start();
	}
	~this() {
		writefln("ending");
	}
	
	int workerMain()
	{
// -- disabled in case of deadlock in std.format, but never reached anyway
//		writefln("In Thread");
		return 0;
	}
	
	Thread worker;
}

int main()
{
	Main m = new Main();
		
	return 0;
}
Jun 26 2007
next sibling parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
And score 10 for forgetting to explain the problem :)

Anyway, this code sample should print "starting", star a thread that exits
immediately, and finally print "ending" as the destructor is called at program
termination.
Instead, it prints "starting" and then does nothing - or more precisely hogs
the entirety of only one of my 2 cores - and never exits.

BTW, this is using gdc on a Mac (latest release of gdcmac), so could be a
pthread specific problem perhaps?

Tristam MacDonald Wrote:

 So, I am sure that I must be missing something simple, that is causing my
multi-threading code in D fail miserably. I have cut it down to the smallest
code sample that exhibits the problem, and I would be grateful if someone could
have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 
 class Main
 {
 	this() {
 		writefln("starting");
 
 		Thread worker = new Thread(&workerMain);
 		worker.start();
 	}
 	~this() {
 		writefln("ending");
 	}
 	
 	int workerMain()
 	{
 // -- disabled in case of deadlock in std.format, but never reached anyway
 //		writefln("In Thread");
 		return 0;
 	}
 	
 	Thread worker;
 }
 
 int main()
 {
 	Main m = new Main();
 		
 	return 0;
 }
 
 
 

Jun 26 2007
parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:

 And score 10 for forgetting to explain the problem :)
 
 Anyway, this code sample should print "starting", star a thread that exits
 immediately, and finally print "ending" as the destructor is called at
 program termination. Instead, it prints "starting" and then does nothing -
 or more precisely hogs the entirety of only one of my 2 cores - and never
 exits.
 
 BTW, this is using gdc on a Mac (latest release of gdcmac), so could be a
 pthread specific problem perhaps?
 
 Tristam MacDonald Wrote:
 
 So, I am sure that I must be missing something simple, that is causing my
 multi-threading code in D fail miserably. I have cut it down to the
 smallest code sample that exhibits the problem, and I would be grateful
 if someone could have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 // -- disabled in case of deadlock in std.format, but never reached
 anyway
 //           writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 return 0;
 }
 
 


If you change the main method to this the thread will be run. int main() { Main m = new Main(); foreach(t;Thread.getAll()) if(!t.isSelf()) t.wait(); return 0; } The current behavior seams to be that all threads exits when main returns (or when std.c.stdlib.exit is run). The constructor is not run because the class is not deleted by you or the garbage collector, there was talk about this behavior some time ago (maybe last summer) and this is by design, search the newsgroup or changelog if you want to follow the discussion.
Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
Content-Type: text/plain

Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).

I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?

The point I don't understand, is why is this only the case when I am using
threads? And I think the thread implementation may be a little buggy here
anyway, why on earth would the assert statement below cause a 'Bus Error'?

class Main
{
	this() {
		writefln("starting");

		Thread worker = new Thread(&workerMain);
		worker.start();
	}
	~this() {
		writefln("ending");
	}
	
	int workerMain()
	{
		writefln("In Thread");
		return 0;
	}
	
	Thread worker;
}

int main()
{
	Main m = new Main();

	assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not done!");
		
	return 0;
}

Johan Granberg Wrote:

 Tristam MacDonald wrote:
 
 And score 10 for forgetting to explain the problem :)
 
 Anyway, this code sample should print "starting", star a thread that exits
 immediately, and finally print "ending" as the destructor is called at
 program termination. Instead, it prints "starting" and then does nothing -
 or more precisely hogs the entirety of only one of my 2 cores - and never
 exits.
 
 BTW, this is using gdc on a Mac (latest release of gdcmac), so could be a
 pthread specific problem perhaps?
 
 Tristam MacDonald Wrote:
 
 So, I am sure that I must be missing something simple, that is causing my
 multi-threading code in D fail miserably. I have cut it down to the
 smallest code sample that exhibits the problem, and I would be grateful
 if someone could have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 // -- disabled in case of deadlock in std.format, but never reached
 anyway
 //           writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 return 0;
 }
 
 


If you change the main method to this the thread will be run. int main() { Main m = new Main(); foreach(t;Thread.getAll()) if(!t.isSelf()) t.wait(); return 0; } The current behavior seams to be that all threads exits when main returns (or when std.c.stdlib.exit is run). The constructor is not run because the class is not deleted by you or the garbage collector, there was talk about this behavior some time ago (maybe last summer) and this is by design, search the newsgroup or changelog if you want to follow the discussion.

Jun 26 2007
next sibling parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:

 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the object
 had a non trivial destructor (dispose of shared memory, flush an iostream,
 etc.)?
 
 The point I don't understand, is why is this only the case when I am using
 threads? And I think the thread implementation may be a little buggy here
 anyway, why on earth would the assert statement below cause a 'Bus Error'?

Because you are redeclaring worker in the Main constructor, change this line Thread worker = new Thread(&workerMain); to this worker = new Thread(&workerMain); and it will probably work.
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not done!");
 
 return 0;
 }

Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
Oh cripes, right, with that fixed your wait solution works fine, but I still
can't find why destructors are not  called at termination with threads?

Johan Granberg Wrote:

 Tristam MacDonald wrote:
 
 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the object
 had a non trivial destructor (dispose of shared memory, flush an iostream,
 etc.)?
 
 The point I don't understand, is why is this only the case when I am using
 threads? And I think the thread implementation may be a little buggy here
 anyway, why on earth would the assert statement below cause a 'Bus Error'?

Because you are redeclaring worker in the Main constructor, change this line Thread worker = new Thread(&workerMain); to this worker = new Thread(&workerMain); and it will probably work.
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not done!");
 
 return 0;
 }


Jun 26 2007
parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:

 Oh cripes, right, with that fixed your wait solution works fine, but I
 still can't find why destructors are not  called at termination with
 threads?

An old newsgroup post by Walter http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=11455 i don't like this behavior myself but that's it.
 Johan Granberg Wrote:
 
 Tristam MacDonald wrote:
 
 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the
 object had a non trivial destructor (dispose of shared memory, flush an
 iostream, etc.)?
 
 The point I don't understand, is why is this only the case when I am
 using threads? And I think the thread implementation may be a little
 buggy here anyway, why on earth would the assert statement below cause
 a 'Bus Error'?

Because you are redeclaring worker in the Main constructor, change this line Thread worker = new Thread(&workerMain); to this worker = new Thread(&workerMain); and it will probably work.
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not
 done!");
 
 return 0;
 }



Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
Would explicitly running a full collect cycle at the end of main (and wrapping
logic in an inner function work around this? More likely some would still be
left though.

Johan Granberg Wrote:

 Tristam MacDonald wrote:
 
 Oh cripes, right, with that fixed your wait solution works fine, but I
 still can't find why destructors are not  called at termination with
 threads?

An old newsgroup post by Walter http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=11455 i don't like this behavior myself but that's it.
 Johan Granberg Wrote:
 
 Tristam MacDonald wrote:
 
 Hmm, I don't see anything relevant in either the changelog or the news
 group (haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
 destructors called when the program exits? What would happen if the
 object had a non trivial destructor (dispose of shared memory, flush an
 iostream, etc.)?
 
 The point I don't understand, is why is this only the case when I am
 using threads? And I think the thread implementation may be a little
 buggy here anyway, why on earth would the assert statement below cause
 a 'Bus Error'?

Because you are redeclaring worker in the Main constructor, change this line Thread worker = new Thread(&workerMain); to this worker = new Thread(&workerMain); and it will probably work.
 class Main
 {
 this() {
 writefln("starting");
 
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 
 int workerMain()
 {
 writefln("In Thread");
 return 0;
 }
 
 Thread worker;
 }
 
 int main()
 {
 Main m = new Main();
 
 assert(m.worker.getState() == Thread.TS.TERMINATED, "Thread not
 done!");
 
 return 0;
 }




Jun 26 2007
next sibling parent Johan Granberg <lijat.meREM OVEgmail.com> writes:
Tristam MacDonald wrote:

 Would explicitly running a full collect cycle at the end of main (and
 wrapping logic in an inner function work around this? More likely some
 would still be left though.

I think it would work if you ensure that no valid references is left in scope when you does so. (set all globals and the local references in main to null)
Jun 26 2007
prev sibling parent Sean Kelly <sean f4.ca> writes:
Tristam MacDonald wrote:
 Would explicitly running a full collect cycle at the end of main (and wrapping
logic in an inner function work around this? More likely some would still be
left though.

You can call fullCollectNoStack() to collect everything whether there is a valid reference to it or not. Sean
Jun 26 2007
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Tristam MacDonald wrote:
 Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?

Running the dtors of all objects on exit is problematic. Should they run before or after the module dtors? What if they are run after the module dtors but the object in question relied on the module's dtor not yet having been run? In Tango, uncollected objects not specifically cleaned up in a module dtor are not guaranteed to be collected for this reason. An alternative would be to run a collection after main() exits as a part of the cleanup process. This would get your Main object below, but it would slow the shutdown process for the sake of collecting only a very few objects, and I'm not sure it's worthwhile to do so.
 The point I don't understand, is why is this only the case when I am using
threads? And I think the thread implementation may be a little buggy here
anyway, why on earth would the assert statement below cause a 'Bus Error'?

I have no idea. On Tango, your program outputs: starting In Thread Here's the converted code: import tango.core.Thread; import tango.stdc.stdio; class Main { this() { printf("starting\n"); Thread worker = new Thread(&workerMain); worker.start(); } ~this() { printf("ending\n"); } void workerMain() { printf("In Thread\n"); } Thread worker; } int main() { Main m = new Main(); return 0; } Sean
Jun 26 2007
parent reply Tristam MacDonald <swiftcoder gmail.com> writes:
I guess coming from a C++ background (i.e. no GC), I am having trouble with the
whole idea of destructors-as-finalizers, meaning only used to free memory,
rather than to manage resources.

To me this seems a big hole in the language. I know 'scope' is supposed to be
used for RAII, but it practice it falls short, due to the inability to return
or copy scoped classes in a useful manner. This leads to lots of C-style
explicit reference counting (obj.retain(), obj.release(), etc.), without even
the C++ convinience of wrapping it in 'fake' pointers.

AFAIK, no GC'd language has come up with a good solution, and there obviously
isn't a simple offhand fix. Maybe constructors/destructors/copying of structs
would fill this hole by allowing high-level value types, which would solve RAII
by dint of residing on the stack. 

Sean Kelly Wrote:
 Tristam MacDonald wrote:
 Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).
 
 I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?

Running the dtors of all objects on exit is problematic. Should they run before or after the module dtors? What if they are run after the module dtors but the object in question relied on the module's dtor not yet having been run? In Tango, uncollected objects not specifically cleaned up in a module dtor are not guaranteed to be collected for this reason. An alternative would be to run a collection after main() exits as a part of the cleanup process. This would get your Main object below, but it would slow the shutdown process for the sake of collecting only a very few objects, and I'm not sure it's worthwhile to do so.

Jun 27 2007
next sibling parent Sean Kelly <sean f4.ca> writes:
Tristam MacDonald wrote:
 I guess coming from a C++ background (i.e. no GC), I am having trouble with
the whole idea of destructors-as-finalizers, meaning only used to free memory,
rather than to manage resources.

They can do both, but if it's important that those resources are explicitly released then you should take steps to manage the object's lifetime. I could still be convinced that all lingering objects should be collected on shutdown, but I haven't come up with an approach I'm entirely happy with (the most obvious being to simply collect everything and if an exception is thrown then give up, but continue the rest of the shutdown process).
 To me this seems a big hole in the language. I know 'scope' is supposed to be
used for RAII, but it practice it falls short, due to the inability to return
or copy scoped classes in a useful manner. This leads to lots of C-style
explicit reference counting (obj.retain(), obj.release(), etc.), without even
the C++ convinience of wrapping it in 'fake' pointers.

Yup. It doesn't help that there is no way to implement smart pointers in D, given the lack of copy semantics in structs.
 AFAIK, no GC'd language has come up with a good solution, and there obviously
isn't a simple offhand fix. Maybe constructors/destructors/copying of structs
would fill this hole by allowing high-level value types, which would solve RAII
by dint of residing on the stack. 

FWIW, Tango has a means of hooking the collection routine, which may be used as a means of detecting resource 'leaks'. Sean
Jun 27 2007
prev sibling parent Myron Alexander <someone somewhere.com> writes:
Tristam MacDonald wrote:
 I guess coming from a C++ background (i.e. no GC), I am having trouble with
the whole idea of destructors-as-finalizers, meaning only used to free memory,
rather than to manage resources.
 
 To me this seems a big hole in the language. I know 'scope' is supposed to be
used for RAII, but it practice it falls short, due to the inability to return
or copy scoped classes in a useful manner. This leads to lots of C-style
explicit reference counting (obj.retain(), obj.release(), etc.), without even
the C++ convinience of wrapping it in 'fake' pointers.
 
 AFAIK, no GC'd language has come up with a good solution, and there obviously
isn't a simple offhand fix. Maybe constructors/destructors/copying of structs
would fill this hole by allowing high-level value types, which would solve RAII
by dint of residing on the stack. 
 
 Sean Kelly Wrote:
 Tristam MacDonald wrote:
 Hmm, I don't see anything relevant in either the changelog or the news group
(haven't finished searching the latter though).

 I am not sure I understand, shouldn't all remaining objects have their
destructors called when the program exits? What would happen if the object had
a non trivial destructor (dispose of shared memory, flush an iostream, etc.)?

run before or after the module dtors? What if they are run after the module dtors but the object in question relied on the module's dtor not yet having been run? In Tango, uncollected objects not specifically cleaned up in a module dtor are not guaranteed to be collected for this reason. An alternative would be to run a collection after main() exits as a part of the cleanup process. This would get your Main object below, but it would slow the shutdown process for the sake of collecting only a very few objects, and I'm not sure it's worthwhile to do so.


I'm not quite sure how D is different from C++ in this instance. C++ on heap: main... { SomeObject so = new SomeObject (); ... } End of main, program exits and the destructor of SomeObject is not executed. You have to 'delete so;'. C++ on stack: main... { SomeObject so(); ... } End of main, stack unwound, so::~SomeObject() is executed. If 'so' were in a function, and you wanted to return 'so', then you need a copy constructor; 'so' is destructed at end of function. D on heap: main... { SomeObject so = new SomeObject (); ... } D on stack: main... { scope SomeObject so = new SomeObject (); ... } I have not tried it yet, but you can write a class copy constructor for D and, afaik, constructors are coming to structs. I have no idea how to solve the raised gc issues. Regards, Myron.
Jun 29 2007
prev sibling parent reply BCS <ao pathlink.com> writes:
Reply to Tristam,

 So, I am sure that I must be missing something simple, that is causing
 my multi-threading code in D fail miserably. I have cut it down to the
 smallest code sample that exhibits the problem, and I would be
 grateful if someone could have a look and let me know why...
 
 import std.thread;
 import std.stdio;
 class Main
 {
 this() {
 writefln("starting");
 Thread worker = new Thread(&workerMain);
 worker.start();
 }
 ~this() {
 writefln("ending");
 }
 int workerMain()
 {
 // -- disabled in case of deadlock in std.format, but never reached
 anyway
 //		writefln("In Thread");
 return 0;
 }
 Thread worker;
 }
 int main()
 {
 Main m = new Main();
 return 0;
 }

If I had to guess, I'd say that the program is exiting before the thread starts. But then I'm also guessing what the problem is.
Jun 26 2007
parent BCS <ao pathlink.com> writes:
Reply to Benjamin,

  But then I'm also guessing what the problem is.
 

scrath that
Jun 26 2007