www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 11378] New: implicit runtime initialization/finalization is broken

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378

           Summary: implicit runtime initialization/finalization is broken
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Keywords: dll
          Severity: regression
          Priority: P2
         Component: druntime
        AssignedTo: nobody puremagic.com
        ReportedBy: code dawg.eu


--- Comment #0 from Martin Nowak <code dawg.eu> 2013-10-29 07:04:31 PDT ---
This change was introduced with
https://github.com/D-Programming-Language/druntime/pull/590 (commit
https://github.com/dawgfoto/druntime/commit/68eee299e28ee51421aac8fd5920edeb814c5b54).

It was added to perform automatic initialization/finalization when loading
shared libraries.
The change of the initialization order introduced some regressions (Bug 11149,
Bug 10976 and Bug 11309).
Furthermore the runtime initialization takes ownership of the main thread
(calls thread_attachThis) which may be unintended in a C executable that uses a
D library.

There are basically two ways to fix this.

A. Remove all implicit initialization/finalization.

   All linked D libraries (including druntime) are initialized/finalized
   by _d_run_main or by calling rt_init/rt_term for C executable.

   Dynamically loaded shared libraries would need to be initialized after
   loading and finalized before unloading. This required a ctor/dtor API in
   druntime that can be called from within a shared library (might use the
   return address to determine the calling DSO) to support D libraries in
   C plugin frameworks that only allow predefined init/fini hooks.
   We should also add an API that takes a library handle and runs ctors/dtors.
   When moving the ctor/dtor calls out of libc's _d_dso_registry the calls need
   to be synchronized.

B. Remove only implicit initialization/finalization of linked libraries.

   All linked D libraries (including druntime) are initialized/finalized
   by _d_run_main or by calling rt_init/rt_term for C executable.

   Dynamically loaded shared libraries would still be initialized implicitly
   (this includes calling rt_init/rt_term if it hasn't been called before and
   implicitly attaching threads that load a D library).
   The main difficulty here is to detect in _d_dso_registry whether the calling
   DSO was linked or dynamically loaded.


I'm currently working on approach B because it avoids complicating the usage of
shared libraries.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 29 2013
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #1 from Martin Nowak <code dawg.eu> 2013-10-29 12:54:51 PDT ---
C. Perform implicit init/fini only after rt_init was called.

   All linked D libraries (including druntime) are initialized/finalized
   by _d_run_main or by calling rt_init/rt_term for C executable.

   All dynamic libraries loaded before calling rt_init are only initialized
   when calling rt_init. Likewise calling rt_term will terminate all
   currently loaded libraries.

This provides init control for C executables with a simple/existing API while
solving the init order issues.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 29 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull


--- Comment #2 from Martin Nowak <code dawg.eu> 2013-10-29 16:47:12 PDT ---
https://github.com/D-Programming-Language/druntime/pull/649

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 29 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #3 from github-bugzilla puremagic.com 2013-10-30 00:13:11 PDT ---
Commits pushed to master at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/dc559c3ef2916102c6f295d70c3941644e545bf2
make runtime initialization/finalization explicit

- fix Issue 11378 - implicit runtime initialization/finalization
  is broken

- C programs need to call rt_init in order
  to initialize the runtime and all shared
  libraries.

- After rt_init was called shared libraries are
  initialized on loading and finalized on unloading.

- C programs need to call rt_term in order
  to finalize the runtime and all shared
  libraries.

https://github.com/D-Programming-Language/druntime/commit/9ab0b70dc957f6f40cf9683888fe1e779b110e49
Merge pull request #649 from dawgfoto/fix11378

fix Issue 11378 - implicit runtime initialization/finalization is broken

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #4 from github-bugzilla puremagic.com 2013-10-30 00:21:40 PDT ---
Commit pushed to 2.064 at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/7c396ef4080737928e68bdb2550e403b87fe379f
Merge pull request #649 from dawgfoto/fix11378

fix Issue 11378 - implicit runtime initialization/finalization is broken

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugzilla digitalmars.com
         Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378


Martin Krejcirik <mk krej.cz> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |mk krej.cz
         Resolution|FIXED                       |


--- Comment #5 from Martin Krejcirik <mk krej.cz> 2013-10-30 18:34:26 CET ---
I've noticed the following program now segfaults on exit. In case calling
exit() is now not possible, please document the correct way how to do it (I
haven't found any).

---------------------
import std.concurrency, std.c.stdlib;

int main(string[] args)
{
    spawn(&thread);
    return 0;
}

void thread()
{
    while(1)
    {
        receive(
            (OwnerTerminated o) { exit(EXIT_SUCCESS); } /* segfault on exit */
        );
    }
}
---------------------

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378


kai redstar.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kai redstar.de


--- Comment #6 from kai redstar.de 2013-10-30 13:52:11 PDT ---
After merge of pull request #649 I can't compile druntime.

Error message is:

src\rt\dmain2.d(442): Error: function object.Throwable.toString () is not
callab
le using argument types (void delegate(const(char)[] buf) nothrow)
src\rt\dmain2.d(450): Error: function object.Throwable.toString () is not
callab
le using argument types (void delegate(const(char)[] buf) nothrow)

The new overloaded method toString() is missing ib object.di / object_.d

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #7 from Martin Nowak <code dawg.eu> 2013-10-30 13:54:00 PDT ---
(In reply to comment #5)
 I've noticed the following program now segfaults on exit. In case calling
 exit() is now not possible, please document the correct way how to do it (I
 haven't found any).
 
You have a race condition because both the main thread and the spawned thread run the process destruction. You shouldn't call exit in your thread but simply break the infinite loop. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #8 from Martin Nowak <code dawg.eu> 2013-10-30 13:59:57 PDT ---
(In reply to comment #6)
 After merge of pull request #649 I can't compile druntime.
 
I see, that overload is only available on master but is missing on 2.064. So cherry-picking didn't work for this pull. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #9 from Martin Krejcirik <mk krej.cz> 2013-10-30 22:23:11 CET ---
(In reply to comment #7)
 You shouldn't call exit in your thread but simply break the infinite loop.
OK, that's probably workable, would be nice if there was some function to correctly exit the thread at any moment, though. Case closed for me. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #10 from Walter Bright <bugzilla digitalmars.com> 2013-10-30
20:19:42 PDT ---
Not sure - is this fixed or not for 2.064?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |FIXED


--- Comment #11 from Martin Nowak <code dawg.eu> 2013-10-30 23:10:27 PDT ---
(In reply to comment #9)
 OK, that's probably workable, would be nice if there was some function to
 correctly exit the thread at any moment, though. Case closed for me.
Exiting a thread is different from exiting the process and even that would require more for a correct shutdown than C's exit function. The only mechanism that we have which could correctly unwind a stack and free all resources is throwing an Exception. What's your use-case for this? -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 30 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378


Jacob Carlborg <doob me.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |doob me.com


--- Comment #12 from Jacob Carlborg <doob me.com> 2013-10-31 00:29:11 PDT ---
(In reply to comment #11)

 Exiting a thread is different from exiting the process and even that would
 require more for a correct shutdown than C's exit function.
 
 The only mechanism that we have which could correctly unwind a stack and free
 all resources is throwing an Exception. What's your use-case for this?
Can't we setup a callback using "atexit" which correctly terminates the runtime? -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 31 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=11378



--- Comment #13 from Martin Nowak <code dawg.eu> 2013-10-31 07:50:45 PDT ---
(In reply to comment #12)
 Can't we setup a callback using "atexit" which correctly terminates the
 runtime?
No, because then we couldn't clean up the stack. I still don't understand the need for this. I don't even know if a C program can be terminated by a call to exit from another thread. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Oct 31 2013