www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Proof of Concept: Binding to and extending C++ objects via a metric

reply downs <default_357-line yahoo.de> writes:
I've just committed a module to tools (tools.cpp,
http://dsource.org/projects/scrapple/Trunk/tools/tools/cpp.d ) that should
enable linking and using any C++ library with D programs without creating a C
wrapper for it.

The code has been tested with a simple multiple-inheritance sample:
http://paste.dprogramming.com/dpv8hpjp

The D code to bind to this is http://paste.dprogramming.com/dphlv1ot . It
requires a symlink to the tools folder in its directory.

Caveats:

 - D1
 - Only works on g++4's "newapi" layout and mangling (but should be reasonably
portable).
 - Diamond pattern is untested and unsupported.
 - Class protection is completely unsupported - public only.
 - RTTI must be generated manually (normally the compiler would do this). See
the D half of the sample.
 - Dynamic casting from and to generated classes is not implemented.
   - The C++ code that does this is long and full of special cases, so
   - For the example, I ended up hardcoding a manual translation between A and
B.
   - If you need dynamic casting, I recommend hardcoding the required paths
based on classinfo pointer comparisons.
 - The C++ DSL parser is not very tested. Occasional bugs may occur.
 - When building with LDC, fixup_vtable() needs to be called for every newly
created C++ class object.
   - This is because LDC does not yet support initialization from global
variable addresses.

Building the sample is a two-step process:

g++ test.cpp -c -g -o test.o
gdc -J. cpp_demo.d tools/cpp.d tools/base.d tools/smart_import.d tools/ctfe.d
tools/compat.d tools/text.d tools/tests.d tools/log.d test.o \
	-o cpptest -lstdc++ -L /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/ -g # adapt
library path as appropriate


Writing this also gave us what I believe to be a fair contender to Most
Horrifying Statement of D Ever Written:

  fndefs ~= `mixin("extern(C) `~t~` "~`~mname~`~"`~fnparams~`; ");`~'\n'~
      `mixin("`~fn~`("~refToParamList("Params!(typeof(&"~`~mname~`~"))",
isRef!(typeof(mixin("&"~`~mname~`))))~") {
      return "~`~mname~`~"("~refToValueList(isRef!(typeof(mixin("&"~`~mname~`))))~");
    }
  "); `;

 --downs
Dec 09 2009
next sibling parent reply BCS <none anon.com> writes:
please tell me the subject is a freudian slip (or just a typo).
Dec 09 2009
next sibling parent reply downs <default_357-line yahoo.de> writes:
BCS wrote:
 please tell me the subject is a freudian slip (or just a typo).
 
 
Considering the amount of abuse me and the compiler put each other through in the course of writing this, including such gems as incorrect tuple lengths, incorrect tuple slices, endless errors without line numbers, spurious template instantiation errors with nonsensical parameters, and at least one case where T[0] evaluated to T, necessitating the workaround Tuple!(T[0])[0], it felt appropriate.
Dec 09 2009
parent reply retard <re tard.com.invalid> writes:
Thu, 10 Dec 2009 01:01:34 +0100, downs wrote:

 including such gems as incorrect
 tuple lengths, incorrect tuple slices, endless errors without line
 numbers, spurious template instantiation errors with nonsensical
 parameters, and at least one case where T[0] evaluated to T,
 necessitating the workaround Tuple!(T[0])[0], it felt appropriate.
Why are these needed? Because of compiler bugs?
Dec 09 2009
parent reply downs <default_357-line yahoo.de> writes:
retard wrote:
 Thu, 10 Dec 2009 01:01:34 +0100, downs wrote:
 
 including such gems as incorrect
 tuple lengths, incorrect tuple slices, endless errors without line
 numbers, spurious template instantiation errors with nonsensical
 parameters, and at least one case where T[0] evaluated to T,
 necessitating the workaround Tuple!(T[0])[0], it felt appropriate.
Why are these needed? Because of compiler bugs?
Apparently, in certain very specific corner cases, my version of GDC believes that when I say "T[0]", I really mean "T". In that case, "Tuple!(T[0])[0]" is like saying "really very the first element". :)
Dec 09 2009
parent BCS <none anon.com> writes:
Hello downs,

 retard wrote:
 
 Thu, 10 Dec 2009 01:01:34 +0100, downs wrote:
 
 including such gems as incorrect
 tuple lengths, incorrect tuple slices, endless errors without line
 numbers, spurious template instantiation errors with nonsensical
 parameters, and at least one case where T[0] evaluated to T,
 necessitating the workaround Tuple!(T[0])[0], it felt appropriate.
Why are these needed? Because of compiler bugs?
Apparently, in certain very specific corner cases, my version of GDC believes that when I say "T[0]", I really mean "T". In that case, "Tuple!(T[0])[0]" is like saying "really very the first element". :)
I call it a Bug. Do you have a test case?
Dec 10 2009
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"BCS" <none anon.com> wrote in message 
news:a6268ffdff18cc46f7aa8b7a32 news.digitalmars.com...
 please tell me the subject is a freudian slip (or just a typo).
It's a very precise figure: 1 metric fuckton == 1,000 meteric shitloads
Dec 09 2009
parent Bernard Helyer <b.helyer gmail.com> writes:
On 10/12/09 14:41, Nick Sabalausky wrote:
 "BCS"<none anon.com>  wrote in message
 news:a6268ffdff18cc46f7aa8b7a32 news.digitalmars.com...
 please tell me the subject is a freudian slip (or just a typo).
It's a very precise figure: 1 metric fuckton == 1,000 meteric shitloads
Though I suppose in metric it would be a fucktonne. :D
Dec 10 2009
prev sibling next sibling parent g <g g.g> writes:
omg
Dec 09 2009
prev sibling next sibling parent reply g <g g.g> writes:
downs Wrote:

 I've just committed a module to tools (tools.cpp,
http://dsource.org/projects/scrapple/Trunk/tools/tools/cpp.d ) that should
enable linking and using any C++ library with D programs without creating a C
wrapper for it.
 
 The code has been tested with a simple multiple-inheritance sample:
http://paste.dprogramming.com/dpv8hpjp
 
 The D code to bind to this is http://paste.dprogramming.com/dphlv1ot . It
requires a symlink to the tools folder in its directory.
 
 Caveats:
 
  - D1
  - Only works on g++4's "newapi" layout and mangling (but should be reasonably
portable).
  - Diamond pattern is untested and unsupported.
  - Class protection is completely unsupported - public only.
  - RTTI must be generated manually (normally the compiler would do this). See
the D half of the sample.
  - Dynamic casting from and to generated classes is not implemented.
    - The C++ code that does this is long and full of special cases, so
    - For the example, I ended up hardcoding a manual translation between A and
B.
    - If you need dynamic casting, I recommend hardcoding the required paths
based on classinfo pointer comparisons.
  - The C++ DSL parser is not very tested. Occasional bugs may occur.
  - When building with LDC, fixup_vtable() needs to be called for every newly
created C++ class object.
    - This is because LDC does not yet support initialization from global
variable addresses.
 
 Building the sample is a two-step process:
 
 g++ test.cpp -c -g -o test.o
 gdc -J. cpp_demo.d tools/cpp.d tools/base.d tools/smart_import.d tools/ctfe.d
tools/compat.d tools/text.d tools/tests.d tools/log.d test.o \
 	-o cpptest -lstdc++ -L /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/ -g # adapt
library path as appropriate
 
 
 Writing this also gave us what I believe to be a fair contender to Most
Horrifying Statement of D Ever Written:
 
   fndefs ~= `mixin("extern(C) `~t~` "~`~mname~`~"`~fnparams~`; ");`~'\n'~
       `mixin("`~fn~`("~refToParamList("Params!(typeof(&"~`~mname~`~"))",
isRef!(typeof(mixin("&"~`~mname~`))))~") {
       return "~`~mname~`~"("~refToValueList(isRef!(typeof(mixin("&"~`~mname~`))))~");
     }
   "); `;
 
  --downs
if you have problems seeing the first link: http://dsource.org/projects/scrapple/browser/trunk/tools/tools/cpp.d
Dec 09 2009
parent downs <default_357-line yahoo.de> writes:
g wrote:
 downs Wrote:
 
 I've just committed a module to tools (tools.cpp,
http://dsource.org/projects/scrapple/Trunk/tools/tools/cpp.d ) that should
enable linking and using any C++ library with D programs without creating a C
wrapper for it.

 The code has been tested with a simple multiple-inheritance sample:
http://paste.dprogramming.com/dpv8hpjp

 The D code to bind to this is http://paste.dprogramming.com/dphlv1ot . It
requires a symlink to the tools folder in its directory.

 Caveats:

  - D1
  - Only works on g++4's "newapi" layout and mangling (but should be reasonably
portable).
  - Diamond pattern is untested and unsupported.
  - Class protection is completely unsupported - public only.
  - RTTI must be generated manually (normally the compiler would do this). See
the D half of the sample.
  - Dynamic casting from and to generated classes is not implemented.
    - The C++ code that does this is long and full of special cases, so
    - For the example, I ended up hardcoding a manual translation between A and
B.
    - If you need dynamic casting, I recommend hardcoding the required paths
based on classinfo pointer comparisons.
  - The C++ DSL parser is not very tested. Occasional bugs may occur.
  - When building with LDC, fixup_vtable() needs to be called for every newly
created C++ class object.
    - This is because LDC does not yet support initialization from global
variable addresses.

 Building the sample is a two-step process:

 g++ test.cpp -c -g -o test.o
 gdc -J. cpp_demo.d tools/cpp.d tools/base.d tools/smart_import.d tools/ctfe.d
tools/compat.d tools/text.d tools/tests.d tools/log.d test.o \
 	-o cpptest -lstdc++ -L /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/ -g # adapt
library path as appropriate


 Writing this also gave us what I believe to be a fair contender to Most
Horrifying Statement of D Ever Written:

   fndefs ~= `mixin("extern(C) `~t~` "~`~mname~`~"`~fnparams~`; ");`~'\n'~
       `mixin("`~fn~`("~refToParamList("Params!(typeof(&"~`~mname~`~"))",
isRef!(typeof(mixin("&"~`~mname~`))))~") {
       return "~`~mname~`~"("~refToValueList(isRef!(typeof(mixin("&"~`~mname~`))))~");
     }
   "); `;

  --downs
if you have problems seeing the first link: http://dsource.org/projects/scrapple/browser/trunk/tools/tools/cpp.d
Oops. Sorry for that. (Never a post without an error!)
Dec 09 2009
prev sibling next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from downs (default_357-line yahoo.de)'s article
 I've just committed a module to tools (tools.cpp,
http://dsource.org/projects/scrapple/Trunk/tools/tools/cpp.d ) that should enable linking and using any C++ library with D programs without creating a C wrapper for it.
 The code has been tested with a simple multiple-inheritance sample:
http://paste.dprogramming.com/dpv8hpjp
 The D code to bind to this is http://paste.dprogramming.com/dphlv1ot . It
requires a symlink to the tools folder in its directory.
 Caveats:
  - D1
  - Only works on g++4's "newapi" layout and mangling (but should be reasonably
portable).
  - Diamond pattern is untested and unsupported.
  - Class protection is completely unsupported - public only.
  - RTTI must be generated manually (normally the compiler would do this). See
the D half of the sample.
  - Dynamic casting from and to generated classes is not implemented.
    - The C++ code that does this is long and full of special cases, so
    - For the example, I ended up hardcoding a manual translation between A and
B.
    - If you need dynamic casting, I recommend hardcoding the required paths
based on classinfo pointer comparisons.
  - The C++ DSL parser is not very tested. Occasional bugs may occur.
  - When building with LDC, fixup_vtable() needs to be called for every newly
created C++ class object.
    - This is because LDC does not yet support initialization from global
variable addresses.
 Building the sample is a two-step process:
 g++ test.cpp -c -g -o test.o
 gdc -J. cpp_demo.d tools/cpp.d tools/base.d tools/smart_import.d tools/ctfe.d
tools/compat.d tools/text.d tools/tests.d tools/log.d test.o \
 	-o cpptest -lstdc++ -L /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/ -g # adapt
library
path as appropriate
 Writing this also gave us what I believe to be a fair contender to Most
Horrifying Statement of D Ever Written:
   fndefs ~= `mixin("extern(C) `~t~` "~`~mname~`~"`~fnparams~`; ");`~'\n'~
       `mixin("`~fn~`("~refToParamList("Params!(typeof(&"~`~mname~`~"))",
isRef!(typeof(mixin("&"~`~mname~`))))~") {
       return
"~`~mname~`~"("~refToValueList(isRef!(typeof(mixin("&"~`~mname~`))))~");
     }
   "); `;
  --downs
Awesome! One quick question. This may be a stupid question, as I don't understand at all yet how this works: Can you instantiate C++ templates with this thing, or does it work only w/ non-templated C++ code?
Dec 09 2009
parent downs <default_357-line yahoo.de> writes:
dsimcha wrote:
 == Quote from downs (default_357-line yahoo.de)'s article
 I've just committed a module to tools (tools.cpp,
http://dsource.org/projects/scrapple/Trunk/tools/tools/cpp.d ) that should enable linking and using any C++ library with D programs without creating a C wrapper for it.
 The code has been tested with a simple multiple-inheritance sample:
http://paste.dprogramming.com/dpv8hpjp
 The D code to bind to this is http://paste.dprogramming.com/dphlv1ot . It
requires a symlink to the tools folder in its directory.
 Caveats:
  - D1
  - Only works on g++4's "newapi" layout and mangling (but should be reasonably
portable).
  - Diamond pattern is untested and unsupported.
  - Class protection is completely unsupported - public only.
  - RTTI must be generated manually (normally the compiler would do this). See
the D half of the sample.
  - Dynamic casting from and to generated classes is not implemented.
    - The C++ code that does this is long and full of special cases, so
    - For the example, I ended up hardcoding a manual translation between A and
B.
    - If you need dynamic casting, I recommend hardcoding the required paths
based on classinfo pointer comparisons.
  - The C++ DSL parser is not very tested. Occasional bugs may occur.
  - When building with LDC, fixup_vtable() needs to be called for every newly
created C++ class object.
    - This is because LDC does not yet support initialization from global
variable addresses.
 Building the sample is a two-step process:
 g++ test.cpp -c -g -o test.o
 gdc -J. cpp_demo.d tools/cpp.d tools/base.d tools/smart_import.d tools/ctfe.d
tools/compat.d tools/text.d tools/tests.d tools/log.d test.o \
 	-o cpptest -lstdc++ -L /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/ -g # adapt
library
path as appropriate
 Writing this also gave us what I believe to be a fair contender to Most
Horrifying Statement of D Ever Written:
   fndefs ~= `mixin("extern(C) `~t~` "~`~mname~`~"`~fnparams~`; ");`~'\n'~
       `mixin("`~fn~`("~refToParamList("Params!(typeof(&"~`~mname~`~"))",
isRef!(typeof(mixin("&"~`~mname~`))))~") {
       return
"~`~mname~`~"("~refToValueList(isRef!(typeof(mixin("&"~`~mname~`))))~");
     }
   "); `;
  --downs
Awesome! One quick question. This may be a stupid question, as I don't understand at all yet how this works: Can you instantiate C++ templates with this thing, or does it work only w/ non-templated C++ code?
Nontemplates only. C++ templates require a C++ compiler. If you instantiate them in C++ code, you can then use them in D code, but I haven't looked into how the mangling would look like for this.
Dec 09 2009
prev sibling next sibling parent Don <nospam nospam.com> writes:
downs wrote:
 I've just committed a module to tools (tools.cpp,
http://dsource.org/projects/scrapple/Trunk/tools/tools/cpp.d ) that should
enable linking and using any C++ library with D programs without creating a C
wrapper for it.
 
 The code has been tested with a simple multiple-inheritance sample:
http://paste.dprogramming.com/dpv8hpjp
 
 The D code to bind to this is http://paste.dprogramming.com/dphlv1ot . It
requires a symlink to the tools folder in its directory.
 
 Caveats:
 
  - D1
  - Only works on g++4's "newapi" layout and mangling (but should be reasonably
portable).
At http://www.agner.org/optimize/, you can find a document (calling_conventions.pdf) which gives full details of the C++ name mangling schemes for various compilers.
 Writing this also gave us what I believe to be a fair contender to Most
Horrifying Statement of D Ever Written:
 
   fndefs ~= `mixin("extern(C) `~t~` "~`~mname~`~"`~fnparams~`; ");`~'\n'~
       `mixin("`~fn~`("~refToParamList("Params!(typeof(&"~`~mname~`~"))",
isRef!(typeof(mixin("&"~`~mname~`))))~") {
       return "~`~mname~`~"("~refToValueList(isRef!(typeof(mixin("&"~`~mname~`))))~");
     }
   "); `;
Ugh, that's unreadable. I found that with a little function called string enquote(string s), which changes s into "s", this kind of stuff looks much less terrible. In particular you get rid of the nasty mix of ` and ". fndefs ~= "mixin(" ~ enquote("extern(C)" ~ t ~ " ") ~ "~" ~ mname ~ "~" ~ enquote(fnparams ~ ";") ~ ");" ~ "mixin(" ~ .... // etc
Dec 10 2009
prev sibling parent reply Tim Matthews <tim.matthews7 gmail.com> writes:
downs wrote:
 I've just committed a module to tools that should enable linking and using any
C++ library with D programs without creating a C wrapper for it.
 
Does 'any' c++ library even include the ones in c++ namespaces? If so good job thanks.
Dec 10 2009
parent reply downs <default_357-line yahoo.de> writes:
Tim Matthews wrote:
 downs wrote:
 I've just committed a module to tools that should enable linking and
 using any C++ library with D programs without creating a C wrapper for
 it.
Does 'any' c++ library even include the ones in c++ namespaces? If so good job thanks.
I don't know about namespaces, but that's a problem with mangling and thus, fixable. I'm currently wrestling with bullet physics, which doesn't use namespaces, so this isn't priority right now :p
Dec 12 2009
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 12 Dec 2009 15:25:53 +0300, downs <default_357-line yahoo.de>  
wrote:

 Tim Matthews wrote:
 downs wrote:
 I've just committed a module to tools that should enable linking and
 using any C++ library with D programs without creating a C wrapper for
 it.
Does 'any' c++ library even include the ones in c++ namespaces? If so good job thanks.
I don't know about namespaces, but that's a problem with mangling and thus, fixable. I'm currently wrestling with bullet physics, which doesn't use namespaces, so this isn't priority right now :p
What's the wrapper performance penalty (if any)? And can the code be a bit sanitized? As it is, I believe it is impossible to maintain by anyone except the original author, which is a bad thing.
Dec 12 2009
parent BCS <none anon.com> writes:
Hello Denis,

 On Sat, 12 Dec 2009 15:25:53 +0300, downs <default_357-line yahoo.de>
 wrote:
 
 Tim Matthews wrote:
 
 downs wrote:
 
 I've just committed a module to tools that should enable linking
 and using any C++ library with D programs without creating a C
 wrapper for it.
 
Does 'any' c++ library even include the ones in c++ namespaces? If so good job thanks.
I don't know about namespaces, but that's a problem with mangling and thus, fixable. I'm currently wrestling with bullet physics, which doesn't use namespaces, so this isn't priority right now :p
And can the code be a bit sanitized? As it is, I believe it is impossible to maintain by anyone except the original author, which is a bad thing.
Given he works with bullet physics, I'd say being the only one who understands it qualifies as a "Bad Thing".
Dec 12 2009