digitalmars.D.bugs - [Issue 4307] New: spawn()'ed thread doesn't terminate
- d-bugmail puremagic.com (40/40) Jun 13 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- Sean Kelly (1/1) Jun 15 2010 The main thread is somewhat special in D--it doesn't actually terminate ...
- Sean Kelly (1/1) Jun 16 2010 Never mind, I'll fix it. This will require adding an atexit() sort of r...
- d-bugmail puremagic.com (55/55) Nov 01 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (10/10) Nov 01 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (20/20) Nov 01 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (8/8) Nov 01 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (36/36) Nov 02 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (19/19) Nov 02 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (8/8) Nov 02 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (17/17) Nov 02 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (8/8) Nov 02 2010 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (15/15) Jan 24 2011 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (13/13) Jan 25 2011 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (15/15) Jan 25 2011 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (11/11) Feb 03 2011 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (12/12) Feb 03 2011 http://d.puremagic.com/issues/show_bug.cgi?id=4307
- d-bugmail puremagic.com (13/13) May 21 2013 http://d.puremagic.com/issues/show_bug.cgi?id=4307
http://d.puremagic.com/issues/show_bug.cgi?id=4307 Summary: spawn()'ed thread doesn't terminate Product: D Version: D2 Platform: x86 OS/Version: Windows Status: NEW Severity: normal Priority: P2 Component: DMD AssignedTo: nobody puremagic.com ReportedBy: torhu yahoo.com Using DMD 2.047. This example hangs after printing '9'. From reading Andrei's book, my understanding is that the spawned thread should terminate automatically when its owner thread terminates. But that doesn't happen here. --- import std.concurrency; import std.stdio; void f() { for (;;) { int i = receiveOnly!int(); writeln(i); } } void main() { Tid tid = spawn(&f); foreach (int i; 0..10) { send(tid, i); } } --- -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jun 13 2010
The main thread is somewhat special in D--it doesn't actually terminate until all other threads have terminated. And because the "OwnerTerminated" message is sent in a static dtor, the ordering is wrong for having spawned threads receive this message. I'll leave it up to Andrei to decide whether this is correct behavior or if the OwnerTerminated message should be sent when D main() exits (I'm leaning towards this latter behavior myself). If a change is necessary I'll have to add a stack for onMainExit callbacks to be executed in druntime.
Jun 15 2010
Never mind, I'll fix it. This will require adding an atexit() sort of routine to druntime, which I believe was requested in another ticket anyway.
Jun 16 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 Jonathan M Davis <jmdavisProg gmx.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jmdavisProg gmx.com PDT --- I don't know if this is the same problem or not, but the most threads that I seem to be able to create in a program is 505 or 506. import std.concurrency; import std.stdio; void main() { int currThreads = 0; enum maxThreads = 6; size_t totalThreads = 0; auto recProc = (Tid tid) { writeln(++totalThreads); }; for(size_t i = 0; i < 1_000; ++i) { if(currThreads < maxThreads) receiveTimeout(1, recProc); else receive(recProc); spawn(&threadFunc, thisTid); } while(currThreads > 0) receive(recProc); } void threadFunc(Tid parentTid) { send(parentTid, thisTid); } prints out to either 505 or 506 and then throws an exception: core.thread.ThreadException: Error creating thread ---------------- ./d(void core.thread.Thread.start()) [0x80902b0] ./d(_D3std11concurrency34__T6_spawnTS3std11concurrency3TidZ6_spawnFbPFS3std11concurrency3TidZvS3std11concurrency3TidZS3std11concurrency3Tid+0x7f) [0x808c3c7] ./d(_D3std11concurrency33__T5spawnTS3std11concurrency3TidZ5spawnFPFS3std11concurrency3TidZvS3std11concurrency3TidZS3std11concurrency3Tid+0x10) [0x808c344] ./d(_Dmain+0x6b) [0x8087d0f] ./d(extern (C) int rt.dmain2.main(int, char**)) [0x8091a56] ./d(extern (C) int rt.dmain2.main(int, char**)) [0x80919b0] ./d(extern (C) int rt.dmain2.main(int, char**)) [0x8091a9a] ./d(extern (C) int rt.dmain2.main(int, char**)) [0x80919b0] ./d(main+0x96) [0x8091956] /usr/lib32/libc.so.6(__libc_start_main+0xe6) [0xf756ec76] ./d() [0x8087bf1] My best guess is that the threads aren't really terminating, so the OS is running out of threads. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 01 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PDT --- If I put a print statement at the end of exec() in std.concurrency._spawn(), it appears to print out just fine every time that a spawned thread is supposed to be terminating, so whatever the problem is, I think that it pretty much has to be in core.Thread (or maybe dmd itself, depending on what's causing the issue) rather than std.concurrency. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 01 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PDT --- I think what's happening is that pthread_join() never gets called on a thread started with spawn(), and IIRC, if pthread_join() never gets called on a thread, it never actually terminates. The only place that I see pthread_join() getting called is in Thread.join(), and that never gets called for threads started with spawn(). What we really want here, I think, is for threads which succesfully terminate to just join to their parent thread themselves without the parent thread having to call join() on them, but I'm not sure that you can really do that with pthreads. Assuming that the parent thread has to join() on all threads created with pthread_create(), we're going to need to find a way to get the parent thread to call join() on its spawned threads. About all I can think of is to have a thread whose entire job is to create threads and and make sure that they join. But it's been too long since I had to deal with pthreads for me to remember all of the details. In any case, I believe that the problem stems from the fact that the spawned threads are never actually joined. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 01 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PDT --- Actually, what I think needs to happen is for there to be a way to start threads as detached rather than joinable and have spawn() start detached threads rather than joinable threads. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 01 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PDT --- Actually, the program I posted earlier was buggy (that's what I get for simplifying code and not studying the simplified version enough, I guess). Here's the corrected version: import std.concurrency; import std.stdio; void main() { int currThreads = 0; enum maxThreads = 6; size_t totalThreads = 0; auto recProc = (Tid tid) { writeln(++totalThreads); --currThreads; }; for(size_t i = 0; i < 1_000; ++i) { if(currThreads < maxThreads) receiveTimeout(1, recProc); else receive(recProc); spawn(&threadFunc, thisTid); ++currThreads; } while(currThreads > 0) receive(recProc); } void threadFunc(Tid parentTid) { send(parentTid, thisTid); } -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 02 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PDT --- Created an attachment (id=797) Patch for core.thread for linux with changes based off of dmd 2.050. Here's a patch for core.thread which appears to solve the problem on Linux. Essentially, I made it so that you can give start() a bool to tell it to start the thread as detached (as in pthread_detach(), not detached from the D runtime), which I chose to call joinable (to distinguish from being detached from the D runtime). So, spawn() can call thread.start(false), and then the thread should terminate properly without needing to be joined. The changes are essentially Posix only, so if this problem is also on Windows, then other changes may be required for Windows and/or MacOS X, but I know essentially nothing about threads on either of those systems. So, I don't know if this patch is the best overall solution, but it does appear to fix the problem on Linux, so even if it's not enough of a fix, it should at least help solve the problem. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 02 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PDT --- Created an attachment (id=798) Patch for std.concurrency for linux with changes based off of dmd 2.050. The patch for std.concurrency to go with the patch for core.thread. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 02 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 Jonathan M Davis <jmdavisProg gmx.com> changed: What |Removed |Added ---------------------------------------------------------------------------- obsolete| | PDT --- Created an attachment (id=799) Patch for core.thread for linux with changes based off of dmd 2.050. Apparently, I made a couple of errors in my patch, so here's update version. Unfortunately, while these changes seem to fix all of the simple cases that I've tried, my more complicated programs still fail to terminate properly, so there appear to be issues beyond spawned threads being joinable instead of detached. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 02 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PDT --- Ah, it looks like the problems with my more complex applications have to do with passing functions with incorrect signatures to receive(), so my patch does seem to do the trick (on Linux at least). -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 02 2010
http://d.puremagic.com/issues/show_bug.cgi?id=4307 Jonathan M Davis <jmdavisProg gmx.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|DMD |druntime OS/Version|Windows |All Severity|normal |major PST --- Actually, I'm going to bump this up to major, since spawn is pretty much useless in my experience without this being fixed. Also, I'm 99.99999999999999% sure it's a druntime bug, so I'm moving it to druntime (which I probably would have done before had I noticed that it was marked as dmd). -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jan 24 2011
http://d.puremagic.com/issues/show_bug.cgi?id=4307 Sean Kelly <sean invisibleduck.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED CC| |sean invisibleduck.org PST --- I don't think threads can start as detached, because the GC needs to interact with them. I'll give this another look though. I also fixed a major timing-related bug since 2.051 was released, which may help here as wells. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jan 25 2011
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PST --- I don't know exactly what the GC requires with regards to threads, but when I was talking about starting a thread as detached, I meant detached in the pthreads sense, not the GC sense, like core.thread generally talks about with functions like thread_detachThis(). Spawned threads obviously need to be attached to the GC. The problem is that they can't have to be joined unless there's somehow a thread somewhere which cleans them up. Spawned threads are essentially supposed to clean themselves up and go away when they terminate, and that essentially means that they need to be detached in the pthread sense, since the programmer isn't going to have the parent thread join it when it's done. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jan 25 2011
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PST --- Fixed in the current revision. The threads exit with a MessageMismatch exception though, which seems like a QOI issue. They should probably exit with an OwnerTerminated message instead (which is the message that triggered the mismatch). I've just changed the behavior so LinkTerminated and OwnerTerminated messages are still thrown as-is instead of being considered a MessageMismatch. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Feb 03 2011
http://d.puremagic.com/issues/show_bug.cgi?id=4307 PST --- What about joining the spawned threads? From what I recall, there was no place that join was called on them, so if I understand correctly, they'll continue to exist until the program terminates. And I believe that there are a finite number of threads allowed at one time, so it would be a problem if spawned threads continued to exist after they're done executing. If spawned threads are joinable, they need to be joined or they'll never actually terminate (with pthreads anyway) - unless I'm misunderstanding something. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Feb 03 2011
http://d.puremagic.com/issues/show_bug.cgi?id=4307 Jonathan M Davis <jmdavisProg gmx.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED PDT --- I believe that this has been long since fixed - certainly all of the programs here seem to work properly. If someone runs into similar problems, please open a new bug for them. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
May 21 2013