www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Call to Runtime.unloadLibrary corrupts stdout and stderr

reply Markus Pursche <pursche01 gmail.com> writes:
Hi, I'm trying to hot reload a DLL while the program is running 
to allow for rapid iteration in a game engine I am working on. I 
started reading up on how DLLs work between D code here: 
https://wiki.dlang.org/Win32_DLLs_in_D#D_code_calling_D_code_in_DLLs

I followed that example, however when I try to do the following I 
crash:
1. Load DLL
2. Unload DLL
3. Load DLL again

The problem I am getting is a Bad File Descriptor exception right 
after step 2, on my very next writeln to be exact.

Here is the exception I am getting: http://imgur.com/a/b9XBm
I spent a couple of hours trying to debug it, googling all sorts 
of combinations of "dlang unloadlibrary bad file descriptor", 
until I got the idea to try "dlang freelibrary bad file 
descriptor" and found this old bug report from 2004:

http://forum.dlang.org/post/cg5hpf$kpv$1 digitaldaemon.com

This sounds absolutely spot on to the problems I am having. I 
tried to save stdout and reset it after unloading my library but 
I still crashed on the next writeln, does anyone have an idea for 
a workaround on this?

I'm hoping that someone like Walter or Andrei will see this and 
decide that it's time for this 2004 issue to finally die. :P
Sep 02
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 02/09/2017 10:38 PM, Markus Pursche wrote:
 Hi, I'm trying to hot reload a DLL while the program is running to allow 
 for rapid iteration in a game engine I am working on. I started reading 
 up on how DLLs work between D code here: 
 https://wiki.dlang.org/Win32_DLLs_in_D#D_code_calling_D_code_in_DLLs
 
 I followed that example, however when I try to do the following I crash:
 1. Load DLL
 2. Unload DLL
 3. Load DLL again
 
 The problem I am getting is a Bad File Descriptor exception right after 
 step 2, on my very next writeln to be exact.
 
 Here is the exception I am getting: http://imgur.com/a/b9XBm
 I spent a couple of hours trying to debug it, googling all sorts of 
 combinations of "dlang unloadlibrary bad file descriptor", until I got 
 the idea to try "dlang freelibrary bad file descriptor" and found this 
 old bug report from 2004:
 
 http://forum.dlang.org/post/cg5hpf$kpv$1 digitaldaemon.com
 
 This sounds absolutely spot on to the problems I am having. I tried to 
 save stdout and reset it after unloading my library but I still crashed 
 on the next writeln, does anyone have an idea for a workaround on this?
 
 I'm hoping that someone like Walter or Andrei will see this and decide 
 that it's time for this 2004 issue to finally die. :P
*sigh* 2018H1 plan I am listing shared library support. Even better still, I will argue it as the ONLY language/druntime feature we need in it! Seriously, this is the biggest thing holding back D atm.
Sep 02
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 02.09.2017 23:38, Markus Pursche wrote:
 Hi, I'm trying to hot reload a DLL while the program is running to allow 
 for rapid iteration in a game engine I am working on. I started reading 
 up on how DLLs work between D code here: 
 https://wiki.dlang.org/Win32_DLLs_in_D#D_code_calling_D_code_in_DLLs
Unfortunately, this section only applies to simple programs (no multi-threading, no sharing of other resources than GC-memory, not even malloc'ed memory). I think there should be a big red warning at the beginning of that section. Proper DLL support for Windows (with a shared phobos library) is slowly being worked on, see http://dconf.org/2016/talks/thaut.html In the mean time, I prefer keeping DLL resources completely separate, as shown in https://wiki.dlang.org/Win32_DLLs_in_D#DLLs_with_a_C_Interface
 
 I followed that example, however when I try to do the following I crash:
 1. Load DLL
 2. Unload DLL
 3. Load DLL again
 
 The problem I am getting is a Bad File Descriptor exception right after 
 step 2, on my very next writeln to be exact.
 
 Here is the exception I am getting: http://imgur.com/a/b9XBm
 I spent a couple of hours trying to debug it, googling all sorts of 
 combinations of "dlang unloadlibrary bad file descriptor", until I got 
 the idea to try "dlang freelibrary bad file descriptor" and found this 
 old bug report from 2004:
 
 http://forum.dlang.org/post/cg5hpf$kpv$1 digitaldaemon.com
 
 This sounds absolutely spot on to the problems I am having. I tried to 
 save stdout and reset it after unloading my library but I still crashed 
 on the next writeln, does anyone have an idea for a workaround on this?
 
 I'm hoping that someone like Walter or Andrei will see this and decide 
 that it's time for this 2004 issue to finally die. :P
You are probably hitting https://issues.dlang.org/show_bug.cgi?id=1550, so try the workaround: import std.stdio and add "_fcloseallp = null;" somewhere in your DllMain.
Sep 03
parent reply Markus Pursche <pursche01 gmail.com> writes:
On Sunday, 3 September 2017 at 13:41:33 UTC, Rainer Schuetze 
wrote:
 On 02.09.2017 23:38, Markus Pursche wrote:
 [...]
Unfortunately, this section only applies to simple programs (no multi-threading, no sharing of other resources than GC-memory, not even malloc'ed memory). I think there should be a big red warning at the beginning of that section. Proper DLL support for Windows (with a shared phobos library) is slowly being worked on, see http://dconf.org/2016/talks/thaut.html In the mean time, I prefer keeping DLL resources completely separate, as shown in https://wiki.dlang.org/Win32_DLLs_in_D#DLLs_with_a_C_Interface
 [...]
You are probably hitting https://issues.dlang.org/show_bug.cgi?id=1550, so try the workaround: import std.stdio and add "_fcloseallp = null;" somewhere in your DllMain.
This worked! So, is there any chance that you could tell me why this isn't the default behavior from the start? Like is there a reason for it not to be there automatically, like a performance loss or a leak or something?
Sep 03
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 03.09.2017 20:21, Markus Pursche wrote:
 On Sunday, 3 September 2017 at 13:41:33 UTC, Rainer Schuetze wrote:
 On 02.09.2017 23:38, Markus Pursche wrote:
 [...]
Unfortunately, this section only applies to simple programs (no multi-threading, no sharing of other resources than GC-memory, not even malloc'ed memory). I think there should be a big red warning at the beginning of that section. Proper DLL support for Windows (with a shared phobos library) is slowly being worked on, see http://dconf.org/2016/talks/thaut.html In the mean time, I prefer keeping DLL resources completely separate, as shown in https://wiki.dlang.org/Win32_DLLs_in_D#DLLs_with_a_C_Interface
 [...]
You are probably hitting https://issues.dlang.org/show_bug.cgi?id=1550, so try the workaround: import std.stdio and add "_fcloseallp = null;" somewhere in your DllMain.
This worked! So, is there any chance that you could tell me why this isn't the default behavior from the start? Like is there a reason for it not to be there automatically, like a performance loss or a leak or something?
This workaround has side effects, i.e. it doesn't automatically close any file still open by the DLLs' instance of the C runtime, so it might cause incomplete files if someone relies on these being automatically flushed and closed. It's a bug in the dmc runtime. Only Walter can fix it as it is closed source.
Sep 03
parent reply Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 9/3/2017 1:07 PM, Rainer Schuetze via Digitalmars-d wrote:
 This workaround has side effects, i.e. it doesn't automatically close 
 any file still open by the DLLs' instance of the C runtime, so it 
 might cause incomplete files if someone relies on these being 
 automatically flushed and closed.

 It's a bug in the dmc runtime. Only Walter can fix it as it is closed 
 source.
The source is on github, but it's in a private repo.  I'm sure Walter would be happy to give access to it should someone who wants to fix it step forward and ask him.
Sep 03
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 9/4/17 2:20 AM, Brad Roberts via Digitalmars-d wrote:
 On 9/3/2017 1:07 PM, Rainer Schuetze via Digitalmars-d wrote:
 This workaround has side effects, i.e. it doesn't automatically close 
 any file still open by the DLLs' instance of the C runtime, so it 
 might cause incomplete files if someone relies on these being 
 automatically flushed and closed.

 It's a bug in the dmc runtime. Only Walter can fix it as it is closed 
 source.
The source is on github, but it's in a private repo.  I'm sure Walter would be happy to give access to it should someone who wants to fix it step forward and ask him.
To that point, he did give me access, and I submitted a PR to fix an issue with pipes and std.process a few years back. However, I was never able to completely build DMC's library from source, so in order to test, I had to replace the affected .obj file in the library with my modified one. I was missing a MASM specific version. -Steve
Sep 12