www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - C++ template name mangling

reply Walter Bright <newshound2 digitalmars.com> writes:
Currently, D supports:

1. C++ function name mangling
2. C++ namespace name mangling
3. C++ class field and vtbl[] layout
4. C++ function calling conventions

But what is missing is name mangling to match C++ templates. This makes it 
awkward and tedious to construct a D interface to a C++ template.

Andrei has proposed that D support, for templates declared within an
extern(C++) 
block, C++ name mangling. This should not be difficult, it should not break any 
existing D source code, and will enable limited interaction with C++ templates 
from D.

One nice side benefit is that no other language offers such support, and given 
the increasing ubiquity of C++ template use, it would give us a nice leg up.
Aug 15 2014
next sibling parent reply "Francesco Cattoglio" <francesco.cattoglio gmail.com> writes:
On Friday, 15 August 2014 at 19:53:28 UTC, Walter Bright wrote:

 Currently, D supports:

 1. C++ function name mangling
 2. C++ namespace name mangling
 3. C++ class field and vtbl[] layout
 4. C++ function calling conventions

 But what is missing is name mangling to match C++ templates. 
 This makes it awkward and tedious to construct a D interface to 
 a C++ template.

 Andrei has proposed that D support, for templates declared 
 within an extern(C++) block, C++ name mangling. This should not 
 be difficult, it should not break any existing D source code, 
 and will enable limited interaction with C++ templates from D.

 One nice side benefit is that no other language offers such 
 support, and given the increasing ubiquity of C++ template use, 
 it would give us a nice leg up.


Do different C++ compilers even mangle template names in the same 
way? I remember function and classes mangling being almost 
standard nowadays, does this apply to templates, too?
Sure this would add to the "Cool things you can do in D"; do you 
have any estimate of the amount of work needed/target release?
Aug 15 2014
parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 15.08.2014 22:27, schrieb Francesco Cattoglio:

 On Friday, 15 August 2014 at 19:53:28 UTC, Walter Bright wrote:

 Currently, D supports:

 1. C++ function name mangling
 2. C++ namespace name mangling
 3. C++ class field and vtbl[] layout
 4. C++ function calling conventions

 But what is missing is name mangling to match C++ templates. This
 makes it awkward and tedious to construct a D interface to a C++
 template.

 Andrei has proposed that D support, for templates declared within an
 extern(C++) block, C++ name mangling. This should not be difficult, it
 should not break any existing D source code, and will enable limited
 interaction with C++ templates from D.

 One nice side benefit is that no other language offers such support,
 and given the increasing ubiquity of C++ template use, it would give
 us a nice leg up.


 Do different C++ compilers even mangle template names in the same way? I
 remember function and classes mangling being almost standard nowadays,
 does this apply to templates, too?
 Sure this would add to the "Cool things you can do in D"; do you have
 any estimate of the amount of work needed/target release?


On Linux and OSX (and other primarily-GCC/clang-using-platforms) 
probably all (common) C++ compilers behave like GCC's g++.
Not sure about different windows compilers, though - but I'd be 
surprised if classnames, -fields, vtbl-layouts, function-names and 
namespaces were mangled the same but templates not - so either the 
problem already exists or there is none. Just guessing, though.

Cheers,
Daniel
Aug 15 2014
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 15, 2014 at 12:53:29PM -0700, Walter Bright via Digitalmars-d wrote:

 Currently, D supports:
 
 1. C++ function name mangling
 2. C++ namespace name mangling
 3. C++ class field and vtbl[] layout
 4. C++ function calling conventions
 
 But what is missing is name mangling to match C++ templates. This
 makes it awkward and tedious to construct a D interface to a C++
 template.
 
 Andrei has proposed that D support, for templates declared within an
 extern(C++) block, C++ name mangling. This should not be difficult, it
 should not break any existing D source code, and will enable limited
 interaction with C++ templates from D.
 
 One nice side benefit is that no other language offers such support,
 and given the increasing ubiquity of C++ template use, it would give
 us a nice leg up.


Sounds like an interesting idea.

What are the limits to this, though? Obviously, anything involving
D-only features can't possibly work, like alias parameters, typesafe
variadics, etc..


T

-- 
Customer support: the art of getting your clients to pay for your own
incompetence.
Aug 15 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/15/2014 1:53 PM, H. S. Teoh via Digitalmars-d wrote:

 What are the limits to this, though? Obviously, anything involving
 D-only features can't possibly work, like alias parameters, typesafe
 variadics, etc..


That's right, it'd have to be the subset of template types that C++ supports.
Aug 15 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 15, 2014 at 01:57:40PM -0700, Walter Bright via Digitalmars-d wrote:

 On 8/15/2014 1:53 PM, H. S. Teoh via Digitalmars-d wrote:

What are the limits to this, though? Obviously, anything involving
D-only features can't possibly work, like alias parameters, typesafe
variadics, etc..


 
 That's right, it'd have to be the subset of template types that C++
 supports.


What would interfacing with C++ templates be like in practice? C++
templates have SFINAE, but D doesn't; so how would one translate a set
of C++ templates that rely on SFINAE to the equivalent D declarations
(inside extern(C++) blocks)?

I hope we won't need to add SFINAE to extern(C++) blocks...


T

-- 
Study gravitation, it's a field with a lot of potential.
Aug 15 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/15/2014 2:02 PM, H. S. Teoh via Digitalmars-d wrote:

 What would interfacing with C++ templates be like in practice?


Don't really know. It's more like try it out and see what we can do. Clearly, 
for someone to use this effectively, they'd have to be comfortable with both
C++ 
templates and D templates (cough, Andrei, cough) but it could be a real enabler 
for such programmers.



 C++ templates have SFINAE, but D doesn't; so how would one translate a set
 of C++ templates that rely on SFINAE to the equivalent D declarations
 (inside extern(C++) blocks)?

 I hope we won't need to add SFINAE to extern(C++) blocks...


Yeah, SFINAE seems clearly out of the question.
Aug 15 2014
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 15, 2014 at 03:00:33PM -0700, Walter Bright via Digitalmars-d wrote:

 On 8/15/2014 2:02 PM, H. S. Teoh via Digitalmars-d wrote:

What would interfacing with C++ templates be like in practice?


 
 Don't really know. It's more like try it out and see what we can do.
 Clearly, for someone to use this effectively, they'd have to be comfortable
 with both C++ templates and D templates (cough, Andrei, cough) but it could
 be a real enabler for such programmers.
 
 

C++ templates have SFINAE, but D doesn't; so how would one translate a set
of C++ templates that rely on SFINAE to the equivalent D declarations
(inside extern(C++) blocks)?

I hope we won't need to add SFINAE to extern(C++) blocks...


 
 Yeah, SFINAE seems clearly out of the question.


On that note, opDispatch unfortunately appears to have SFINAE... or
behaves as though it has SFINAE:

	https://issues.dlang.org/show_bug.cgi?id=8387

This problem was reported back in 2012 and has had at least 4 duplicates
filed for it. It would be *really* nice if we could finally squash this
bug. (I wish I knew dmd internals well enough to do it myself... but I'm
still trying to find the time to work on the ambiguous inner scope
mangling problem.)


T

-- 
What do you call optometrist jokes? Vitreous humor.
Aug 15 2014
prev sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Walter Bright"  wrote in message news:lslvu0$2vrk$1 digitalmars.com...


 Don't really know. It's more like try it out and see what we can do. 
 Clearly, for someone to use this effectively, they'd have to be 
 comfortable with both C++ templates and D templates (cough, Andrei, cough) 
 but it could be a real enabler for such programmers.


Just don't try it out in the master branch.
Aug 15 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-08-15 23:02, H. S. Teoh via Digitalmars-d wrote:


 What would interfacing with C++ templates be like in practice? C++
 templates have SFINAE, but D doesn't; so how would one translate a set
 of C++ templates that rely on SFINAE to the equivalent D declarations
 (inside extern(C++) blocks)?

 I hope we won't need to add SFINAE to extern(C++) blocks...


Could template constraints help?

-- 
/Jacob Carlborg
Aug 16 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/16/14, 11:55 AM, Jacob Carlborg wrote:

 On 2014-08-15 23:02, H. S. Teoh via Digitalmars-d wrote:


 What would interfacing with C++ templates be like in practice? C++
 templates have SFINAE, but D doesn't; so how would one translate a set
 of C++ templates that rely on SFINAE to the equivalent D declarations
 (inside extern(C++) blocks)?

 I hope we won't need to add SFINAE to extern(C++) blocks...


 Could template constraints help?


Like until now, extern(C++) code obeys D lookup rules but generates 
calls to C++ functions. -- Andrei
Aug 16 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/15/14, 1:53 PM, H. S. Teoh via Digitalmars-d wrote:

 On Fri, Aug 15, 2014 at 12:53:29PM -0700, Walter Bright via Digitalmars-d
wrote:

 Currently, D supports:

 1. C++ function name mangling
 2. C++ namespace name mangling
 3. C++ class field and vtbl[] layout
 4. C++ function calling conventions

 But what is missing is name mangling to match C++ templates. This
 makes it awkward and tedious to construct a D interface to a C++
 template.

 Andrei has proposed that D support, for templates declared within an
 extern(C++) block, C++ name mangling. This should not be difficult, it
 should not break any existing D source code, and will enable limited
 interaction with C++ templates from D.

 One nice side benefit is that no other language offers such support,
 and given the increasing ubiquity of C++ template use, it would give
 us a nice leg up.


 Sounds like an interesting idea.

 What are the limits to this, though? Obviously, anything involving
 D-only features can't possibly work, like alias parameters, typesafe
 variadics, etc..


The tighter integration the better. This is crucial to make D a good 
investment for Facebook going forward. It may as well be D's killer 
feature the same way C integration has been C++'s killer feature back in 
the day.

A few more details about this. Simple free function mangling should be 
the first step. The next step would be to allow full C++ object 
manipulation from D, using D rules and semantics. Consider the 
definition of std::string:

template <
   class charT,
   class traits = char_traits<charT>,
   class Alloc = allocator<charT>


class basic_string {
   bool empty() const { ... }
   ...
private:
   ... state ...
};

That could be replicated in D as follows:

extern(C++)
struct basic_string(charT,
   traits = char_traits!charT,
   Alloc = allocator!charT)
{
     bool empty() const; // no definition!
           // generate mangled call to C++ function
     ...
private:
     ... state ...
}

The state/layout must be duplicated. The D compiler would generate 
proper mangled calls and even (in the future) proper constructor and 
destructor calls, which would take us to the holy grail - seamless C++ 
object manipulation from D. From the C++ side this is needed:

template class basic_string<char>;

That instructs the C++ compiler to make all methods of that 
instantiation available for linking. Then D would simply compile calls 
to them, achieving parity save for inlining. Inlining is important so 
probably in stage 3 we should allow inlining on the D side as well.


Andrei



 Aug 15 2014



 next sibling parent  "Idan Arye" <GenericNPC gmail.com>  writes:


On Saturday, 16 August 2014 at 01:55:44 UTC, Andrei Alexandrescu 
wrote:

 The state/layout must be duplicated. The D compiler would 
 generate proper mangled calls and even (in the future) proper 
 constructor and destructor calls, which would take us to the 
 holy grail - seamless C++ object manipulation from D. From the 
 C++ side this is needed:

 template class basic_string<char>;

 That instructs the C++ compiler to make all methods of that 
 instantiation available for linking. Then D would simply 
 compile calls to them, achieving parity save for inlining. 
 Inlining is important so probably in stage 3 we should allow 
 inlining on the D side as well.


 Andrei


Wouldn't that mean that the you'll need to make a .cpp file with 
all the template instantiations used by the D code and compile it 
with a C++ compiler?

dmd can't do all that automatically for you, since it does not 
contain a C++ compiler, but maybe it can be ran with the `-c` 
flag to disable linking, produce that .cpp file, and the build 
system will be responsible for compiling that file and linking it 
together with the rest of the project?



 Aug 15 2014



prev sibling next sibling parent reply "Kagamin" <spam here.lot>  writes:


On Saturday, 16 August 2014 at 01:55:44 UTC, Andrei Alexandrescu 
wrote:

 template <
   class charT,
   class traits = char_traits<charT>,
   class Alloc = allocator<charT>


 class basic_string {
   bool empty() const { ... }
   ...
 private:
   ... state ...
 };

 That could be replicated in D as follows:

 extern(C++)
 struct basic_string(charT,
   traits = char_traits!charT,
   Alloc = allocator!charT)
 {
     bool empty() const; // no definition!
           // generate mangled call to C++ function


Doesn't function defined in header behave like `static inline`, 
or hidden. There could be nothing to link with.



 Aug 16 2014



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 8/16/14, 11:27 AM, Kagamin wrote:

 Doesn't function defined in header behave like `static inline`, or
 hidden. There could be nothing to link with.


No, inline functions are still extern. -- Andrei



 Aug 16 2014



prev sibling  parent reply "Kagamin" <spam here.lot>  writes:


On Saturday, 16 August 2014 at 01:55:44 UTC, Andrei Alexandrescu 
wrote:

 extern(C++)
 struct basic_string(charT,
   traits = char_traits!charT,
   Alloc = allocator!charT)
 {


Also does D already support value semantics for C++ classes? That 
would be interesting, I was thinking about direct interfacing 
with wxWidgets.



 Aug 16 2014



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 8/16/14, 11:31 AM, Kagamin wrote:

 On Saturday, 16 August 2014 at 01:55:44 UTC, Andrei Alexandrescu wrote:

 extern(C++)
 struct basic_string(charT,
   traits = char_traits!charT,
   Alloc = allocator!charT)
 {


 Also does D already support value semantics for C++ classes? That would
 be interesting, I was thinking about direct interfacing with wxWidgets.


Yes, it does (no multiple inheritance of implementation). We must get to 
the next level - support struct construction, copying, and destruction. 
-- Andrei



 Aug 16 2014



  parent  "deadalnix" <deadalnix gmail.com>  writes:


On Saturday, 16 August 2014 at 22:15:12 UTC, Andrei Alexandrescu
wrote:

 Yes, it does (no multiple inheritance of implementation). We 
 must get to the next level - support struct construction, 
 copying, and destruction. -- Andrei


I spend a fair amount of time on that problem ~2 years ago. 2
problems:
   - struct default constructor.
   - struct with disabled postblit, must be able to use opAssign or
alike.



 Aug 16 2014



prev sibling next sibling parent reply "Tofu Ninja" <emmons0 purdue.edu>  writes:


On Friday, 15 August 2014 at 19:53:28 UTC, Walter Bright wrote:

 Currently, D supports:

 1. C++ function name mangling
 2. C++ namespace name mangling
 3. C++ class field and vtbl[] layout
 4. C++ function calling conventions

 But what is missing is name mangling to match C++ templates. 
 This makes it awkward and tedious to construct a D interface to 
 a C++ template.

 Andrei has proposed that D support, for templates declared 
 within an extern(C++) block, C++ name mangling. This should not 
 be difficult, it should not break any existing D source code, 
 and will enable limited interaction with C++ templates from D.

 One nice side benefit is that no other language offers such 
 support, and given the increasing ubiquity of C++ template use, 
 it would give us a nice leg up.


Would this only be usable be usable for templates already
instantiated in the c++ object file you were linking with?



 Aug 15 2014



  parent  "Peter Alexander" <peter.alexander.au gmail.com>  writes:


On Saturday, 16 August 2014 at 02:23:45 UTC, Tofu Ninja wrote:

 Would this only be usable be usable for templates already
 instantiated in the c++ object file you were linking with?


Yes, it has to be. You would need to use C++'s explicit template 
instantiation to create an object file with all the requisite 
symbols.



 Aug 16 2014



prev sibling next sibling parent  "b" <yes no.com>  writes:


  Ballsy move Walter. But super useful if you guys get it working;0



 Aug 16 2014



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 8/15/2014 12:53 PM, Walter Bright wrote:

 But what is missing is name mangling to match C++ templates.


I was perusing the source, and whaddayaknow, it already does C++ template 
mangling, at least for Linux!

There are no tests in the test suite for this, and it gets it all wrong for 
::std. But it's a good starting point to try things out.

This should help move things forward:

https://github.com/D-Programming-Language/dmd/pull/3873



 Aug 17 2014