www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - State of windows DLL support

reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
What is the state of DLL support on windows?

I ask because I have a project coming up very soon which will 
require interacting with DLLs (I think it is a C interface) and I 
would much rather do it in D than C given the opportunity.

  I don't think the choice of language matters, users and 
maintainers can read neither C nor D, and it has to be able to 
communicate with LabVIEW (shudder,gag), best way would be some 
kind of File based bidirectional pipe, or some kind of localhost 
server? I would be writing the server.
Apr 03 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 03/04/2016 11:36 PM, Nicholas Wilson wrote:

 What is the state of DLL support on windows?

 I ask because I have a project coming up very soon which will require
 interacting with DLLs (I think it is a C interface) and I would much
 rather do it in D than C given the opportunity.

   I don't think the choice of language matters, users and maintainers
 can read neither C nor D, and it has to be able to communicate with
 LabVIEW (shudder,gag), best way would be some kind of File based
 bidirectional pipe, or some kind of localhost server? I would be writing
 the server.


Assumption: you need a shared library loaded into LabVIEW.

What I would suggest is if you can throw together a simple c library 
that manages:

- starting/stopping/restarting a process
- communicating with said process

That way you can alter during runtime the program and still use D, or 
any language really.
Apr 03 2016
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 3 April 2016 at 11:46:24 UTC, rikki cattermole wrote:

 On 03/04/2016 11:36 PM, Nicholas Wilson wrote:

 What is the state of DLL support on windows?

 I ask because I have a project coming up very soon which will 
 require
 interacting with DLLs (I think it is a C interface) and I 
 would much
 rather do it in D than C given the opportunity.

   I don't think the choice of language matters, users and 
 maintainers
 can read neither C nor D, and it has to be able to communicate 
 with
 LabVIEW (shudder,gag), best way would be some kind of File 
 based
 bidirectional pipe, or some kind of localhost server? I would 
 be writing
 the server.


 Assumption: you need a shared library loaded into LabVIEW.

 What I would suggest is if you can throw together a simple c 
 library that manages:

 - starting/stopping/restarting a process
 - communicating with said process

 That way you can alter during runtime the program and still use 
 D, or any language really.


I would much rather have nothing to do with LabVIEW for a large 
number of reasons, not the least of which is that it will be much 
simpler to explore/debug with just the server. That and IMO 
LabVIEW is the scum of the earth and I would rather program in 
INTERCAL, that program in LabVIEW.
Apr 03 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 04/04/2016 12:16 AM, Nicholas Wilson wrote:

 On Sunday, 3 April 2016 at 11:46:24 UTC, rikki cattermole wrote:

 On 03/04/2016 11:36 PM, Nicholas Wilson wrote:

 What is the state of DLL support on windows?

 I ask because I have a project coming up very soon which will require
 interacting with DLLs (I think it is a C interface) and I would much
 rather do it in D than C given the opportunity.

   I don't think the choice of language matters, users and maintainers
 can read neither C nor D, and it has to be able to communicate with
 LabVIEW (shudder,gag), best way would be some kind of File based
 bidirectional pipe, or some kind of localhost server? I would be writing
 the server.


 Assumption: you need a shared library loaded into LabVIEW.

 What I would suggest is if you can throw together a simple c library
 that manages:

 - starting/stopping/restarting a process
 - communicating with said process

 That way you can alter during runtime the program and still use D, or
 any language really.


 I would much rather have nothing to do with LabVIEW for a large number
 of reasons, not the least of which is that it will be much simpler to
 explore/debug with just the server. That and IMO LabVIEW is the scum of
 the earth and I would rather program in INTERCAL, that program in LabVIEW.


I'm just guessing context here.
Apr 03 2016
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 3 April 2016 at 12:20:33 UTC, rikki cattermole wrote:

 I'm just guessing context here.


Oh. Needed functionality is in DLL. Need it in LV. Can't / don't 
know how to in LV. setting up a server for that functionality in 
D ( I/O to some power inverters DAQ ). set up a pipe /local host 
server to transfer data to LV.
Apr 03 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 04/04/2016 12:55 AM, Nicholas Wilson wrote:

 On Sunday, 3 April 2016 at 12:20:33 UTC, rikki cattermole wrote:

 I'm just guessing context here.


 Oh. Needed functionality is in DLL. Need it in LV. Can't / don't know
 how to in LV. setting up a server for that functionality in D ( I/O to
 some power inverters DAQ ). set up a pipe /local host server to transfer
 data to LV.


Okay now I'm more awake, lets see what we can do.

The problem you're hitting is there is no real load/unload hooks by the 
looks of things.

Okay assuming DllMain is in fact being called correctly, you can 
initialize and unload D's runtime.
So you can build reloadable platform fairly easily there.

http://www.ni.com/white-paper/3056/en/

You'll need to test that you can in fact fully load/unload D's runtime 
more than once.
Unfortunately I don't think you can modify LabVIEW from D side.

I've done a bit of reading on how to make it so GetProcAddress returns 
whatever you want (runtime adding of symbols).
Simply put, do-able!

Example code: 
http://www.codeproject.com/script/Articles/ViewDownloads.aspx?aid=116253
Apr 03 2016
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 3 April 2016 at 13:50:20 UTC, rikki cattermole wrote:

 On 04/04/2016 12:55 AM, Nicholas Wilson wrote:

 On Sunday, 3 April 2016 at 12:20:33 UTC, rikki cattermole 
 wrote:

 I'm just guessing context here.


 Oh. Needed functionality is in DLL. Need it in LV. Can't / 
 don't know
 how to in LV. setting up a server for that functionality in D 
 ( I/O to
 some power inverters DAQ ). set up a pipe /local host server 
 to transfer
 data to LV.


 Okay now I'm more awake, lets see what we can do.

 The problem you're hitting is there is no real load/unload 
 hooks by the looks of things.

 Okay assuming DllMain is in fact being called correctly, you 
 can initialize and unload D's runtime.
 So you can build reloadable platform fairly easily there.

 http://www.ni.com/white-paper/3056/en/

 You'll need to test that you can in fact fully load/unload D's 
 runtime more than once.
 Unfortunately I don't think you can modify LabVIEW from D side.

 I've done a bit of reading on how to make it so GetProcAddress 
 returns whatever you want (runtime adding of symbols).
 Simply put, do-able!

 Example code: 
 http://www.codeproject.com/script/Articles/ViewDownloads.aspx?aid=116253


I must be missing some context because I have no idea how this 
post (especially the last link) is an answer to the OP's question.

Nicholas, the answer to your original question (how well DLLs 
work with D) depends on the exact situation, namely:

1. If the .exe is non-D and you want to write a D DLL, you should 
not encounter any problems. I assume you will use a C API to 
communicate. The D DLL will have its own self-contained D runtime 
and manage its own memory. The only main thing you have to be 
careful of is when passing the only reference to a GC-allocated 
block of memory to the main program: the D GC can't know whether 
the main program still holds a reference to it, and may free it 
under the main program's nose. The API should specify the exact 
lifetime of objects passed indirectly; if in doubt, copy them to 
unmanaged (core.stdc.stdlib.malloc'd) memory.

2. If the main program is in D and you want to use a C DLL, then 
it is no different to how D already uses the Windows API runtime 
or any other C library. You will need to find, create or convert 
an import library in order to link against the C DLL. The same 
warning about memory and lifetime as above applies.

3. If you want to write D DLLs to use in D programs, you have two 
situations, depending on whether you want the D runtime to be 
shared. You could restrict the program to a C API with 
well-specified lifetimes, taking care of the same issues as 
above. It might also be possible to share the D runtime between 
the D EXE and DLL, so that there is one GC which knows when 
objects are reachable by either the EXE or DLLs, but I don't know 
what's the current state of that on Windows.
Apr 03 2016
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 04/04/2016 1:59 AM, Vladimir Panteleev wrote:

 On Sunday, 3 April 2016 at 13:50:20 UTC, rikki cattermole wrote:

 On 04/04/2016 12:55 AM, Nicholas Wilson wrote:

 On Sunday, 3 April 2016 at 12:20:33 UTC, rikki cattermole wrote:

 I'm just guessing context here.


 Oh. Needed functionality is in DLL. Need it in LV. Can't / don't know
 how to in LV. setting up a server for that functionality in D ( I/O to
 some power inverters DAQ ). set up a pipe /local host server to transfer
 data to LV.


 Okay now I'm more awake, lets see what we can do.

 The problem you're hitting is there is no real load/unload hooks by
 the looks of things.

 Okay assuming DllMain is in fact being called correctly, you can
 initialize and unload D's runtime.
 So you can build reloadable platform fairly easily there.

 http://www.ni.com/white-paper/3056/en/

 You'll need to test that you can in fact fully load/unload D's runtime
 more than once.
 Unfortunately I don't think you can modify LabVIEW from D side.

 I've done a bit of reading on how to make it so GetProcAddress returns
 whatever you want (runtime adding of symbols).
 Simply put, do-able!

 Example code:
 http://www.codeproject.com/script/Articles/ViewDownloads.aspx?aid=116253


 I must be missing some context because I have no idea how this post
 (especially the last link) is an answer to the OP's question.

 Nicholas, the answer to your original question (how well DLLs work with
 D) depends on the exact situation, namely:

 1. If the .exe is non-D and you want to write a D DLL, you should not
 encounter any problems. I assume you will use a C API to communicate.
 The D DLL will have its own self-contained D runtime and manage its own
 memory. The only main thing you have to be careful of is when passing
 the only reference to a GC-allocated block of memory to the main
 program: the D GC can't know whether the main program still holds a
 reference to it, and may free it under the main program's nose. The API
 should specify the exact lifetime of objects passed indirectly; if in
 doubt, copy them to unmanaged (core.stdc.stdlib.malloc'd) memory.

 2. If the main program is in D and you want to use a C DLL, then it is
 no different to how D already uses the Windows API runtime or any other
 C library. You will need to find, create or convert an import library in
 order to link against the C DLL. The same warning about memory and
 lifetime as above applies.

 3. If you want to write D DLLs to use in D programs, you have two
 situations, depending on whether you want the D runtime to be shared.
 You could restrict the program to a C API with well-specified lifetimes,
 taking care of the same issues as above. It might also be possible to
 share the D runtime between the D EXE and DLL, so that there is one GC
 which knows when objects are reachable by either the EXE or DLLs, but I
 don't know what's the current state of that on Windows.


The link is there as a backup plan. I made the assumption that it may 
not be possible to have more than one D shared lib loaded during the 
lifetime.
The idea is simple. Have a D shared lib that acts as a dynamic dispatch 
to the appropriate child process who will route the call to the function 
in question.
The hooking into GetProcAddress allows having symbols that are not 
exported by the shared library that was loaded.

Anyway, at worse this architecture would allow quite fast reloading!
Apr 03 2016
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 3 April 2016 at 14:19:17 UTC, rikki cattermole wrote:

 The link is there as a backup plan. I made the assumption that 
 it may not be possible to have more than one D shared lib 
 loaded during the lifetime.
 The idea is simple. Have a D shared lib that acts as a dynamic 
 dispatch to the appropriate child process who will route the 
 call to the function in question.
 The hooking into GetProcAddress allows having symbols that are 
 not exported by the shared library that was loaded.

 Anyway, at worse this architecture would allow quite fast 
 reloading!


Have you actually tried doing this in practice and getting it to 
work?

Even with correct function signatures, you'd need more than just 
the types to correctly marshall the data between processes. RPC 
is generally done with a subset of types with well-defined 
serializations.

How would you even get GetProcAddress hooking working in this 
scenario? You wouldn't even know the number of arguments, much 
less their types!
Apr 03 2016
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 04/04/2016 2:34 AM, Vladimir Panteleev wrote:

 On Sunday, 3 April 2016 at 14:19:17 UTC, rikki cattermole wrote:

 The link is there as a backup plan. I made the assumption that it may
 not be possible to have more than one D shared lib loaded during the
 lifetime.
 The idea is simple. Have a D shared lib that acts as a dynamic
 dispatch to the appropriate child process who will route the call to
 the function in question.
 The hooking into GetProcAddress allows having symbols that are not
 exported by the shared library that was loaded.

 Anyway, at worse this architecture would allow quite fast reloading!


 Have you actually tried doing this in practice and getting it to work?

 Even with correct function signatures, you'd need more than just the
 types to correctly marshall the data between processes. RPC is generally
 done with a subset of types with well-defined serializations.

 How would you even get GetProcAddress hooking working in this scenario?
 You wouldn't even know the number of arguments, much less their types!


LabVIEW is the one that calls the functions. You declare the signature 
there. Nothing fancy pretty much limited to c here.
 From what I've ready anyway.

So hooking GetProcAddress just tells LabVIEW where to find the function, 
not much beyond that.
Apr 03 2016
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 3 April 2016 at 15:32:00 UTC, rikki cattermole wrote:

 On 04/04/2016 2:34 AM, Vladimir Panteleev wrote:

 [...]


 LabVIEW is the one that calls the functions. You declare the 
 signature there. Nothing fancy pretty much limited to c here.
 From what I've ready anyway.

 So hooking GetProcAddress just tells LabVIEW where to find the 
 function, not much beyond that.


I still have no idea how all this can possibly make any sense or 
work at all, possibly because I don't know what LabVIEW is.
Apr 03 2016
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 3 April 2016 at 15:34:33 UTC, Vladimir Panteleev wrote:

 On Sunday, 3 April 2016 at 15:32:00 UTC, rikki cattermole wrote:

 On 04/04/2016 2:34 AM, Vladimir Panteleev wrote:

 [...]


 LabVIEW is the one that calls the functions. You declare the 
 signature there. Nothing fancy pretty much limited to c here.
 From what I've ready anyway.

 So hooking GetProcAddress just tells LabVIEW where to find the 
 function, not much beyond that.


 I still have no idea how all this can possibly make any sense 
 or work at all, possibly because I don't know what LabVIEW is.


Ah, ignorance is bliss my friend. A graphical programming 
environment, enough said. Their libraries are good i'll give them 
that, but the interface is tedious beyond comparison. Also it's 
not text,so you can't diff it, which implies you can use any 
version control.
Apr 03 2016
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 3 April 2016 at 14:34:07 UTC, Vladimir Panteleev wrote:

 On Sunday, 3 April 2016 at 14:19:17 UTC, rikki cattermole wrote:

 [...]


 Have you actually tried doing this in practice and getting it 
 to work?

 Even with correct function signatures, you'd need more than 
 just the types to correctly marshall the data between 
 processes. RPC is generally done with a subset of types with 
 well-defined serializations.

 How would you even get GetProcAddress hooking working in this 
 scenario? You wouldn't even know the number of arguments, much 
 less their types!


Hence why i wanted to do it in separate processes communicating 
via a (whatever the windows version is of a) pipe or on localhost
Apr 03 2016
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 3 April 2016 at 13:59:29 UTC, Vladimir Panteleev wrote:

 2. If the main program is in D and you want to use a C DLL, 
 then it is no different to how D already uses the Windows API 
 runtime or any other C library. You will need to find, create 
 or convert an import library in order to link against the C 
 DLL. The same warning about memory and lifetime as above 
 applies.


thanks. In theory all the DLL will do is talk to a device driver 
that talks to the power inverters, so just copying the useful 
data should work, possibly freeing the old data.
I'm not quite sure how easy output will be but oh well.
Apr 03 2016