www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Status of Win32 C++ interop

reply "Szymon Gatner" <noemail gmail.com> writes:
Hi,

what is the current status of:

- Win x86/32bit/coff32 interop with C++?

- improvements for general C++ interop that were suppose to come 
with 2.068
Sep 04 2015
parent reply "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Friday, 4 September 2015 at 08:53:27 UTC, Szymon Gatner wrote:

 Hi,

 what is the current status of:

 - Win x86/32bit/coff32 interop with C++?

 - improvements for general C++ interop that were suppose to 
 come with 2.068


If you use either the -m64 or -mscoff32 interop should be pretty 
good. I'm currently using it heavily and its working quite nicely.
Sep 04 2015
parent reply "Szymon Gatner" <noemail gmail.com> writes:
On Friday, 4 September 2015 at 08:58:41 UTC, Benjamin Thaut wrote:

 On Friday, 4 September 2015 at 08:53:27 UTC, Szymon Gatner 
 wrote:

 Hi,

 what is the current status of:

 - Win x86/32bit/coff32 interop with C++?

 - improvements for general C++ interop that were suppose to 
 come with 2.068


 If you use either the -m64 or -mscoff32 interop should be 
 pretty good. I'm currently using it heavily and its working 
 quite nicely.


What about 32bit phobos? Last time I checked (2.067) only x64 was 
distributed.
Sep 04 2015
parent reply "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Friday, 4 September 2015 at 09:07:39 UTC, Szymon Gatner wrote:

 What about 32bit phobos? Last time I checked (2.067) only x64 
 was distributed.


You have to compile it yourself. Use the win64 makefile and 
replace the arch=64 with
arch=32mscoff.

For more details see here:
http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_get_32mscoff_libraries_for_phobos_73980.html
Sep 04 2015
parent reply "Szymon Gatner" <noemail gmail.com> writes:
On Friday, 4 September 2015 at 09:27:14 UTC, Benjamin Thaut wrote:

 On Friday, 4 September 2015 at 09:07:39 UTC, Szymon Gatner 
 wrote:

 What about 32bit phobos? Last time I checked (2.067) only x64 
 was distributed.


 You have to compile it yourself. Use the win64 makefile and 
 replace the arch=64 with
 arch=32mscoff.

 For more details see here:
 http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_get_32mscoff_libraries_for_phobos_73980.html


Ah so that didn't change. Did you try it? Do you run hybrid C++/D 
apps on Win (whether 32 or 64)?. Asking because last time I tried 
it (Win x64 tho) writeln() call from D side crashed app because 
stdio wasn't properly initialized even tho rt_init() was 
successful.
Sep 04 2015
next sibling parent "Kagamin" <spam here.lot> writes:
See https://issues.dlang.org/show_bug.cgi?id=13889
Sep 04 2015
prev sibling parent reply "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Friday, 4 September 2015 at 10:04:48 UTC, Szymon Gatner wrote:

 On Friday, 4 September 2015 at 09:27:14 UTC, Benjamin Thaut 
 wrote:

 On Friday, 4 September 2015 at 09:07:39 UTC, Szymon Gatner 
 wrote:

 What about 32bit phobos? Last time I checked (2.067) only x64 
 was distributed.


 You have to compile it yourself. Use the win64 makefile and 
 replace the arch=64 with
 arch=32mscoff.

 For more details see here:
 http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_get_32mscoff_libraries_for_phobos_73980.html


 Ah so that didn't change. Did you try it? Do you run hybrid 
 C++/D apps on Win (whether 32 or 64)?. Asking because last time 
 I tried it (Win x64 tho) writeln() call from D side crashed app 
 because stdio wasn't properly initialized even tho rt_init() 
 was successful.


I am running hybrid D/C++ apps. I found it to work best when you 
give D the control over the main method, e.g. the program entry 
point should be in D land. Then simply call a C++ function from 
there to initialize your c++ stuff.

I'm currently even running hybrid C++/D apps with dlls. E.g. 
multiple D dlls + multiple C++ dlls loaded by a c++ main program. 
But to do that I have heavy compiler + runtime modifications 
which are not ready yet to do a PR for them.
Sep 04 2015
next sibling parent reply "Szymon Gatner" <noemail gmail.com> writes:
On Friday, 4 September 2015 at 14:18:40 UTC, Benjamin Thaut wrote:

 On Friday, 4 September 2015 at 10:04:48 UTC, Szymon Gatner 
 wrote:

 On Friday, 4 September 2015 at 09:27:14 UTC, Benjamin Thaut 
 wrote:

 On Friday, 4 September 2015 at 09:07:39 UTC, Szymon Gatner 
 wrote:

 What about 32bit phobos? Last time I checked (2.067) only 
 x64 was distributed.


 You have to compile it yourself. Use the win64 makefile and 
 replace the arch=64 with
 arch=32mscoff.

 For more details see here:
 http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_get_32mscoff_libraries_for_phobos_73980.html


 Ah so that didn't change. Did you try it? Do you run hybrid 
 C++/D apps on Win (whether 32 or 64)?. Asking because last 
 time I tried it (Win x64 tho) writeln() call from D side 
 crashed app because stdio wasn't properly initialized even tho 
 rt_init() was successful.


 I am running hybrid D/C++ apps. I found it to work best when 
 you give D the control over the main method, e.g. the program 
 entry point should be in D land. Then simply call a C++ 
 function from there to initialize your c++ stuff.

 I'm currently even running hybrid C++/D apps with dlls. E.g. 
 multiple D dlls + multiple C++ dlls loaded by a c++ main 
 program. But to do that I have heavy compiler + runtime 
 modifications which are not ready yet to do a PR for them.


Hmm that is very good to hear, as is promising. In the mean time 
I tired making tiny Win x64 C++/D app as described by A.Ruppe in 
his book (I did that also when it was released and stdio was 
crashing apps) but now using phobos64.lib from 2.068 distribution 
does not even link properly with VC2015. Oh well, not just yet I 
suppose.

Thanks for your assistance!
Sep 04 2015
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 4 September 2015 at 15:43:44 UTC, Szymon Gatner wrote:

 but now using phobos64.lib from 2.068 distribution does not 
 even link properly with VC2015.


That's https://issues.dlang.org/show_bug.cgi?id=14849
Sep 04 2015
parent "Szymon Gatner" <noemail gmail.com> writes:
On Friday, 4 September 2015 at 16:26:51 UTC, Kagamin wrote:

 On Friday, 4 September 2015 at 15:43:44 UTC, Szymon Gatner 
 wrote:

 but now using phobos64.lib from 2.068 distribution does not 
 even link properly with VC2015.


 That's https://issues.dlang.org/show_bug.cgi?id=14849


yup, that looks like it
Sep 04 2015
prev sibling parent reply "Laeeth Isharc" <nospamlaeeth nospamlaeeth.com> writes:
On Friday, 4 September 2015 at 14:18:40 UTC, Benjamin Thaut wrote:

 On Friday, 4 September 2015 at 10:04:48 UTC, Szymon Gatner 
 wrote:

 On Friday, 4 September 2015 at 09:27:14 UTC, Benjamin Thaut 
 wrote:

 On Friday, 4 September 2015 at 09:07:39 UTC, Szymon Gatner 
 wrote:

 What about 32bit phobos? Last time I checked (2.067) only 
 x64 was distributed.


 You have to compile it yourself. Use the win64 makefile and 
 replace the arch=64 with
 arch=32mscoff.

 For more details see here:
 http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_get_32mscoff_libraries_for_phobos_73980.html


 Ah so that didn't change. Did you try it? Do you run hybrid 
 C++/D apps on Win (whether 32 or 64)?. Asking because last 
 time I tried it (Win x64 tho) writeln() call from D side 
 crashed app because stdio wasn't properly initialized even tho 
 rt_init() was successful.


 I am running hybrid D/C++ apps. I found it to work best when 
 you give D the control over the main method, e.g. the program 
 entry point should be in D land. Then simply call a C++ 
 function from there to initialize your c++ stuff.

 I'm currently even running hybrid C++/D apps with dlls. E.g. 
 multiple D dlls + multiple C++ dlls loaded by a c++ main 
 program. But to do that I have heavy compiler + runtime 
 modifications which are not ready yet to do a PR for them.


Hi Benjamin

Would you be able to give a little more colour on what the limits 
are of interoperability for C++ with DMD master or release ?  As 
I understand it destructors and constructors don't work, and 
obviously it will get tricky passing structures and classes from 
C++ libraries not ported to D.

Reason I ask is I am about to make a case to someone as to why we 
should use D, and I don't want to make any unfounded claims.  I 
don't really know C++ myself, although my noddy test worked fine.

Thanks - and we should bring the doc page IP to date.  At least 
say that more works than currently described, as I am sure this 
will be a factor in adoption.


Laeeth.
Sep 04 2015
parent reply "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Friday, 4 September 2015 at 16:19:49 UTC, Laeeth Isharc wrote:

 Hi Benjamin

 Would you be able to give a little more colour on what the 
 limits are of interoperability for C++ with DMD master or 
 release ?  As I understand it destructors and constructors 
 don't work, and obviously it will get tricky passing structures 
 and classes from C++ libraries not ported to D.



So, things that work really well are classes. Given that you 
construct them in the their "native environment". E.g. create D 
objects in D land and C++ objects in C++ land. I usually use 
factory methods for this. Manually destroying objects also works 
if you add some kind of "Destory" method (can be virtual, but 
doesn't have to be) to your class which simply does a "delete 
this" or similar. The only problems with interfacing C++ classes 
to D is if they have a virtual destructor. But there is a easy 
workaround like so:

C++:

class SomeBaseClass
{
public:
   virtual ~SomeBaseClass();
   virtual void SomeVirtualFunc();
}

D:
extern(C++) class SomeBaseClass
{
protected:
   abstract void __cppDtor(); // don't call

public:
   void SomeVirtualFunc(); // default virtual in D
}

Free functions, static functions, non virtual functions all work 
flawlessly.

I recommend that you create your own pointer type on the C++ 
side, e.g. DPtr which calls the GC.addRoot / GC.removeRoot 
methods of the D's gc interface in the apropriate places in case 
you want to pass D objects into C++ space. If you have a 
reference counting concept on the c++ side its also possible to 
build a small proxy object which does reference counting in c++ 
and calls GC.removeRoot when the last reference count drops.

If you want to bind more complex types, e.g. c++ templates you 
can either declare them extern(C++) in D and the compiler will do 
the correct mangling, or what I did to make them fully functional 
on both sides: I did a full implementation both in D and C++. The 
implementation ensures that the data layout is exactly the same 
for D and C++ but other than that the implementation is 
duplicated on both sides and even different in some cases. It 
still possible to pass this complex type from C++ to D because 
the data layout is the same. I did this for a custom hash map 
implementation and passing it around works flawlessly (by 
reference)

I had another complicated case where I wanted a complex value 
type that is implemented in C++ on the D side. It had a lot of 
members which types don't have a equivalent in D, so I did the 
following.

extern(C++)
{
   void constructComplexStruct(ComplexStruct* self);
   void destructComplexStruct(ComplexStruct* self);
}

struct DefaultCtor {}; // helper type
alias defaultCtor = DefaultCtor();

struct ComplexStruct
{
    disable this();
    disable this(this);
   this(DefaultCtor)
   {
     constructComplexStruct(&this);
   }

   ~this()
   {
     destructComplexStruct(~this);
   }

   extern(C++) void SomeMethodOfComplexStruct();

private:

   // must be sizeof(ComplexStruct) from c++, use unittest to 
ensure!
   void[288] m_data;
}

constructComplexStruct and destructComplexStruct just do a 
placement new / call the C++ dtor.

Usage then works like this:

void someDFunc()
{
   ComplexStruct myStruct(defaultCtor); // constructed using C++ 
default ctor
   myStruct.SomeMethodOfComplexStruct(); // call C++ implemented 
method
   // destructed using C++ dtor
}

So far I haven't found a situation where I couldn't make it work 
the way I wanted.  Its just some work to write the D headers for 
the C++ classes and vise versa, because you have to duplicate 
everything once more. An automated tool for this would be nice, 
but unfotunately there is currently none.

Kind Regards
Benjamin Thaut
Sep 07 2015
next sibling parent reply drug <drug2004 bk.ru> writes:
07.09.2015 21:37, Benjamin Thaut пишет:
snip

 So far I haven't found a situation where I couldn't make it work the way
 I wanted.  Its just some work to write the D headers for the C++ classes
 and vise versa, because you have to duplicate everything once more. An
 automated tool for this would be nice, but unfotunately there is
 currently none.

 Kind Regards
 Benjamin Thaut


It's great. But isn't it based on your custom modifications of compiler 
and the others?
Sep 07 2015
parent reply "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Monday, 7 September 2015 at 19:30:44 UTC, drug wrote:

 07.09.2015 21:37, Benjamin Thaut пишет:
 snip

 So far I haven't found a situation where I couldn't make it 
 work the way
 I wanted.  Its just some work to write the D headers for the 
 C++ classes
 and vise versa, because you have to duplicate everything once 
 more. An
 automated tool for this would be nice, but unfotunately there 
 is
 currently none.

 Kind Regards
 Benjamin Thaut


 It's great. But isn't it based on your custom modifications of 
 compiler and the others?


No, the compiler modifications are only for the Windows DLL 
support. All modifications I did to the c++ binding are already 
in 2.068
Sep 08 2015
parent drug <drug2004 bk.ru> writes:
On 08.09.2015 11:45, Benjamin Thaut wrote:

 On Monday, 7 September 2015 at 19:30:44 UTC, drug wrote:

 07.09.2015 21:37, Benjamin Thaut пишет:
 snip

 So far I haven't found a situation where I couldn't make it work the way
 I wanted.  Its just some work to write the D headers for the C++ classes
 and vise versa, because you have to duplicate everything once more. An
 automated tool for this would be nice, but unfotunately there is
 currently none.

 Kind Regards
 Benjamin Thaut


 It's great. But isn't it based on your custom modifications of
 compiler and the others?


 No, the compiler modifications are only for the Windows DLL support. All
 modifications I did to the c++ binding are already in 2.068


It's good. Thank you!
Sep 08 2015
prev sibling parent reply "Laeeth Isharc" <spamnolaeeth nospamlaeeth.com> writes:
On Monday, 7 September 2015 at 18:37:49 UTC, Benjamin Thaut wrote:

 On Friday, 4 September 2015 at 16:19:49 UTC, Laeeth Isharc 
 wrote:

 Hi Benjamin

 Would you be able to give a little more colour on what the 
 limits are of interoperability for C++ with DMD master or 
 release ?  As I understand it destructors and constructors 
 don't work, and obviously it will get tricky passing 
 structures and classes from C++ libraries not ported to D.



 So, things that work really well are classes...
 So far I haven't found a situation where I couldn't make it 
 work the way I wanted.  Its just some work to write the D 
 headers for the C++ classes and vise versa, because you have to 
 duplicate everything once more. An automated tool for this 
 would be nice, but unfotunately there is currently none.



This is really very clear and helpful, and I appreciate your 
taking the time.  I will place it on the wiki if that's okay.

Library support is surely one of the largest impediments to the 
adoption of D, and we ought to place some emphasis on updating 
the documentation here:
http://dlang.org/cpp_interface.html

How does it work when external APIs expect objects from the C++ 
standard library?  strings, and so on?  How about funny pointer 
types?  shared_ptr  etc?  std::vector, std::list?

Here is one C++ library used by many in finance (at least as a 
backup).  I think there might be a decent amount of value in 
making this usable from D.  (Trying to put my own interests 
aside!)  Paging Andy Smith ?

http://quantlib.org/reference/_bermudan_swaption_8cpp-example.html

Are there any well-known C++ libraries that you have interfaced 
to ?  Could you give some examples of how long it takes ?

Would you be able to drop me an email about something else?  No 
contact info on your blog, but my domain is 
kaleidicassociates.com and my user id is laeeth 


Laeeth.
Sep 08 2015
next sibling parent "Kagamin" <spam here.lot> writes:
On Tuesday, 8 September 2015 at 12:56:00 UTC, Laeeth Isharc wrote:

 How does it work when external APIs expect objects from the C++ 
 standard library?  strings, and so on?  How about funny pointer 
 types?  shared_ptr  etc?  std::vector, std::list?


No, in current state nothing smart is supported. See 
https://issues.dlang.org/show_bug.cgi?id=14086
Sep 08 2015
prev sibling parent "Benjamin Thaut" <code benjamin-thaut.de> writes:
On Tuesday, 8 September 2015 at 12:56:00 UTC, Laeeth Isharc wrote:

 This is really very clear and helpful, and I appreciate your 
 taking the time.  I will place it on the wiki if that's okay.


Thats ok.


 Library support is surely one of the largest impediments to the 
 adoption of D, and we ought to place some emphasis on updating 
 the documentation here:
 http://dlang.org/cpp_interface.html

 How does it work when external APIs expect objects from the C++ 
 standard library?  strings, and so on?  How about funny pointer 
 types?  shared_ptr  etc?  std::vector, std::list?


For templated types like std::vector, std::list and shared_ptr 
you have two options:
- Redo the complete implementation on the D side ensuring that 
the data layout is the same
- Or, expose helper functions which call placement new / 
desturctor of your needed std::vector<T> and do the hidden data 
trick I described above. The regular functions of std::vector 
just have to be declared and the linker will find them if and 
only if the c++ side instanciated the template.


 Here is one C++ library used by many in finance (at least as a 
 backup).  I think there might be a decent amount of value in 
 making this usable from D.  (Trying to put my own interests 
 aside!)  Paging Andy Smith ?

 http://quantlib.org/reference/_bermudan_swaption_8cpp-example.html


I think you have to make this usable from D yourself ;-)


 Are there any well-known C++ libraries that you have interfaced 
 to ?  Could you give some examples of how long it takes ?


I'm interfacing to a 3D engine I'm working on in my spare time 
with some collegues. So no well known C++ library.


 Would you be able to drop me an email about something else?  No 
 contact info on your blog, but my domain is 
 kaleidicassociates.com and my user id is laeeth 


Will do.

Kind Regards
Benjamin Thaut
Sep 10 2015