www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - C++ -> D converter mentioned in AMA

reply "Szymon Gatner" <noemail gmail.com> writes:
Andrei's AMA has interesting answer:

"One of the main D(md) contributors, Daniel Murphy is working on 
automatic conversion tool that eventually will convert DMD's C++ 
codebase to D."

Is this tool already available? Are there any guidelines about 
how to code in C++ to ease the conversion preocess (or even make 
it possible). I would be VERY interested in such a tool when the 
time comes so in the mean time I could slowly "fix" existing C++ 
codebase for future conversion.
Oct 03 2013
next sibling parent reply "simendsjo" <simendsjo gmail.com> writes:
On Thursday, 3 October 2013 at 10:42:32 UTC, Szymon Gatner wrote:
 Andrei's AMA has interesting answer:

 "One of the main D(md) contributors, Daniel Murphy is working 
 on automatic conversion tool that eventually will convert DMD's 
 C++ codebase to D."

 Is this tool already available? Are there any guidelines about 
 how to code in C++ to ease the conversion preocess (or even 
 make it possible). I would be VERY interested in such a tool 
 when the time comes so in the mean time I could slowly "fix" 
 existing C++ codebase for future conversion.

I think Daniel Murphys tool is aimed directly at the DMD codebase. It probably has plenty of hacks to solve the problem. There are plenty of pull-requests to ease the conversion, so I don't think this is a tool that works for other projects out of the box. The code can be found here: https://github.com/yebblies/dmd VisualD includes a generic tool to automatically convert C++ to D. I haven't tried it myself, but you can find some information here: http://www.dsource.org/projects/visuald/wiki/Tour/CppConversion
Oct 03 2013
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-10-03 12:54, simendsjo wrote:

 * https://github.com/jacob-carlborg/dstep?source=cc

OT: what does ?source=cc do? -- /Jacob Carlborg
Oct 03 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-10-03 13:01, simendsjo wrote:

 Ah, you're probably right.
 Another link then :)
 http://dlang.org/cpp_interface.html

And it's only for creating bindings for headers. -- /Jacob Carlborg
Oct 03 2013
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On 10/03/2013 12:52 PM, simendsjo wrote:
 I think Daniel Murphys tool is aimed directly at the DMD codebase.

That's it's key feature, any attempt to first come up with a generic tool would be doomed to fail. Also DMD's code base uses only a limited subset of C++ which makes it more amenable to automatic translation.
Oct 03 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-10-04 01:24, Martin Nowak wrote:

 That's it's key feature, any attempt to first come up with a generic
 tool would be doomed to fail. Also DMD's code base uses only a limited
 subset of C++ which makes it more amenable to automatic translation.

Having #ifdef inside declarations must be a nightmare for a tool like this. Making that properly work on generic code bases sound impossible to me. -- /Jacob Carlborg
Oct 03 2013
parent reply Jacob Carlborg <doob me.com> writes:
On 2013-10-04 08:37, Szymon Gatner wrote:

 Well, that is nothing Clang can't handle. The subset is what I was
 asking for -  there has to be something that tool handles correctly, right?

Of course Clang will be able to lex and parse it. But how should it be translated? void foo (int a #if BAR , int b #endif ) { ... } You cannot do the exact same thing in D: void foo (int a version (BAR) { , int b } ) { ... } Doing these crazy things are only possible with a preprocessor. Then you need to duplicate the function, use a string mixin or something else that's ugly. We can take a simpler example: #if _WIN32 void foo (int); #elif __APPLE__ void foo (long long); #elif linux void foo (long long); #endif Translating this manually it would look like this: version (Windows) void foo (int); else version (OSX) void foo (long); else version (linux) void foo (long); But how could this be translated automatically? In this case you would want to have all the above preprocessor macros enabled, at the same time. Or somehow run it multiple times with different macros enabled and merge them. I don't know how the preprocessor API looks like in Clang. If you could search for hard coded identifiers or something similar. -- /Jacob Carlborg
Oct 04 2013
parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jacob Carlborg" <doob me.com> wrote in message 
news:l2lqik$1vdt$1 digitalmars.com...
 On 2013-10-04 08:37, Szymon Gatner wrote:

 Well, that is nothing Clang can't handle. The subset is what I was
 asking for -  there has to be something that tool handles correctly, 
 right?

Of course Clang will be able to lex and parse it. But how should it be translated? void foo (int a #if BAR , int b #endif ) { ... } You cannot do the exact same thing in D: void foo (int a version (BAR) { , int b } ) { ... } Doing these crazy things are only possible with a preprocessor. Then you need to duplicate the function, use a string mixin or something else that's ugly. We can take a simpler example: #if _WIN32 void foo (int); #elif __APPLE__ void foo (long long); #elif linux void foo (long long); #endif Translating this manually it would look like this: version (Windows) void foo (int); else version (OSX) void foo (long); else version (linux) void foo (long); But how could this be translated automatically? In this case you would want to have all the above preprocessor macros enabled, at the same time. Or somehow run it multiple times with different macros enabled and merge them. I don't know how the preprocessor API looks like in Clang. If you could search for hard coded identifiers or something similar. -- /Jacob Carlborg

I deal with this by not running a preprocessor. The #if directives are parsed as if they're real C++ constructs, and this means everything inside (and around) them must be valid C++ code. With this constraint, translating them to static if/version and doing all versions simultaneously becomes possible.
Oct 04 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-10-04 12:21, Daniel Murphy wrote:

 I deal with this by not running a preprocessor.  The #if directives are
 parsed as if they're real C++ constructs, and this means everything inside
 (and around) them must be valid C++ code.

 With this constraint, translating them to static if/version and doing all
 versions simultaneously becomes possible.

Then you need to A) build your own preprocessor or B) limiting yourself to non-generic code, as you have done in this case. This was my original point, having it work on generic code. -- /Jacob Carlborg
Oct 04 2013
prev sibling next sibling parent "simendsjo" <simendsjo gmail.com> writes:
On Thursday, 3 October 2013 at 10:52:42 UTC, simendsjo wrote:
 On Thursday, 3 October 2013 at 10:42:32 UTC, Szymon Gatner 
 wrote:
 Andrei's AMA has interesting answer:

 "One of the main D(md) contributors, Daniel Murphy is working 
 on automatic conversion tool that eventually will convert 
 DMD's C++ codebase to D."

 Is this tool already available? Are there any guidelines about 
 how to code in C++ to ease the conversion preocess (or even 
 make it possible). I would be VERY interested in such a tool 
 when the time comes so in the mean time I could slowly "fix" 
 existing C++ codebase for future conversion.

I think Daniel Murphys tool is aimed directly at the DMD codebase. It probably has plenty of hacks to solve the problem. There are plenty of pull-requests to ease the conversion, so I don't think this is a tool that works for other projects out of the box. The code can be found here: https://github.com/yebblies/dmd VisualD includes a generic tool to automatically convert C++ to D. I haven't tried it myself, but you can find some information here: http://www.dsource.org/projects/visuald/wiki/Tour/CppConversion

There are also a couple of tools for header files: * https://github.com/jacob-carlborg/dstep?source=cc * http://dlang.org/htod.html
Oct 03 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 3 October 2013 at 10:54:47 UTC, simendsjo wrote:
 There are also a couple of tools for header files:
 * https://github.com/jacob-carlborg/dstep?source=cc
 * http://dlang.org/htod.html

AFAIK, those are for C/ObjC, not C++
Oct 03 2013
prev sibling next sibling parent "simendsjo" <simendsjo gmail.com> writes:
On Thursday, 3 October 2013 at 10:57:24 UTC, Dicebot wrote:
 On Thursday, 3 October 2013 at 10:54:47 UTC, simendsjo wrote:
 There are also a couple of tools for header files:
 * https://github.com/jacob-carlborg/dstep?source=cc
 * http://dlang.org/htod.html

AFAIK, those are for C/ObjC, not C++

Ah, you're probably right. Another link then :) http://dlang.org/cpp_interface.html
Oct 03 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Thursday, 3 October 2013 at 10:52:42 UTC, simendsjo wrote:
 On Thursday, 3 October 2013 at 10:42:32 UTC, Szymon Gatner 
 wrote:
 Andrei's AMA has interesting answer:

 "One of the main D(md) contributors, Daniel Murphy is working 
 on automatic conversion tool that eventually will convert 
 DMD's C++ codebase to D."

 Is this tool already available? Are there any guidelines about 
 how to code in C++ to ease the conversion preocess (or even 
 make it possible). I would be VERY interested in such a tool 
 when the time comes so in the mean time I could slowly "fix" 
 existing C++ codebase for future conversion.

I think Daniel Murphys tool is aimed directly at the DMD codebase. It probably has plenty of hacks to solve the problem. There are plenty of pull-requests to ease the conversion, so I don't think this is a tool that works for other projects out of the box. The code can be found here: https://github.com/yebblies/dmd VisualD includes a generic tool to automatically convert C++ to D. I haven't tried it myself, but you can find some information here: http://www.dsource.org/projects/visuald/wiki/Tour/CppConversion

Thanks, I actually have VisualD installed but didn't know about that functionality tho looking at the docs it is pretty limited especially when it comes to templates :( Sorry to hear about Daniel's tool. Maybe some day...
Oct 03 2013
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 3 October 2013 at 11:08:29 UTC, Szymon Gatner wrote:
 Thanks, I actually have VisualD installed but didn't know about 
 that functionality tho looking at the docs it is pretty limited 
 especially when it comes to templates :(

 Sorry to hear about Daniel's tool. Maybe some day...

Expecting such tool to "just work" is very unrealistic. C++ is very complex language and some behavior just can't be mapped to D directly. For example, one issue with current DMD source translation is that in C++ you can split class implementation among different translation units. Not in D. Of course, libclang helps a lot with such tools (you don't need to write full C++ front-end on your own anymore!) but the complexity of translation is very high and you need to manually recheck it anyway so this does not seem like a feasible _generic_ tool, only something that can be adopted for specific projects.
Oct 03 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Thursday, 3 October 2013 at 11:23:25 UTC, Dicebot wrote:
 On Thursday, 3 October 2013 at 11:08:29 UTC, Szymon Gatner 
 wrote:
 Thanks, I actually have VisualD installed but didn't know 
 about that functionality tho looking at the docs it is pretty 
 limited especially when it comes to templates :(

 Sorry to hear about Daniel's tool. Maybe some day...

Expecting such tool to "just work" is very unrealistic. C++ is very complex language and some behavior just can't be mapped to D directly. For example, one issue with current DMD source translation is that in C++ you can split class implementation among different translation units. Not in D. Of course, libclang helps a lot with such tools (you don't need to write full C++ front-end on your own anymore!) but the complexity of translation is very high and you need to manually recheck it anyway so this does not seem like a feasible _generic_ tool, only something that can be adopted for specific projects.

I fully realize that and actually I was really surprised that such a tool is in production considering amount of code in DMD. OTOH if it can work for DMD then in theory it should be possible to (to some degree) reuse it on other codebase *if* it conforms to certain coding rules/standard right?
Oct 03 2013
prev sibling next sibling parent "Tourist" <gravatar gravatar.com> writes:
On Thursday, 3 October 2013 at 11:29:38 UTC, Szymon Gatner wrote:
 On Thursday, 3 October 2013 at 11:23:25 UTC, Dicebot wrote:
 On Thursday, 3 October 2013 at 11:08:29 UTC, Szymon Gatner 
 wrote:
 Thanks, I actually have VisualD installed but didn't know 
 about that functionality tho looking at the docs it is pretty 
 limited especially when it comes to templates :(

 Sorry to hear about Daniel's tool. Maybe some day...

Expecting such tool to "just work" is very unrealistic. C++ is very complex language and some behavior just can't be mapped to D directly. For example, one issue with current DMD source translation is that in C++ you can split class implementation among different translation units. Not in D. Of course, libclang helps a lot with such tools (you don't need to write full C++ front-end on your own anymore!) but the complexity of translation is very high and you need to manually recheck it anyway so this does not seem like a feasible _generic_ tool, only something that can be adopted for specific projects.

I fully realize that and actually I was really surprised that such a tool is in production considering amount of code in DMD. OTOH if it can work for DMD then in theory it should be possible to (to some degree) reuse it on other codebase *if* it conforms to certain coding rules/standard right?

An official reply from Daniel ;) https://github.com/D-Programming-Language/dmd/pull/1980#issuecomment-19539800
Oct 03 2013
prev sibling next sibling parent "simendsjo" <simendsjo gmail.com> writes:
On Thursday, 3 October 2013 at 11:31:51 UTC, Tourist wrote:
 On Thursday, 3 October 2013 at 11:29:38 UTC, Szymon Gatner 
 wrote:
 On Thursday, 3 October 2013 at 11:23:25 UTC, Dicebot wrote:
 On Thursday, 3 October 2013 at 11:08:29 UTC, Szymon Gatner 
 wrote:
 Thanks, I actually have VisualD installed but didn't know 
 about that functionality tho looking at the docs it is 
 pretty limited especially when it comes to templates :(

 Sorry to hear about Daniel's tool. Maybe some day...

Expecting such tool to "just work" is very unrealistic. C++ is very complex language and some behavior just can't be mapped to D directly. For example, one issue with current DMD source translation is that in C++ you can split class implementation among different translation units. Not in D. Of course, libclang helps a lot with such tools (you don't need to write full C++ front-end on your own anymore!) but the complexity of translation is very high and you need to manually recheck it anyway so this does not seem like a feasible _generic_ tool, only something that can be adopted for specific projects.

I fully realize that and actually I was really surprised that such a tool is in production considering amount of code in DMD. OTOH if it can work for DMD then in theory it should be possible to (to some degree) reuse it on other codebase *if* it conforms to certain coding rules/standard right?

An official reply from Daniel ;) https://github.com/D-Programming-Language/dmd/pull/1980#issuecomment-19539800

It's been self-hosting for two months..? Perhaps DMD 2.065 will be written in D! :)
Oct 03 2013
prev sibling next sibling parent "simendsjo" <simendsjo gmail.com> writes:
On Thursday, 3 October 2013 at 11:49:42 UTC, Jacob Carlborg wrote:
 On 2013-10-03 12:54, simendsjo wrote:

 * https://github.com/jacob-carlborg/dstep?source=cc

OT: what does ?source=cc do?

Hmm. No idea. I just searched for the github repo and copied the link.
Oct 03 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Thursday, 3 October 2013 at 11:31:51 UTC, Tourist wrote:

 An official reply from Daniel ;)
 https://github.com/D-Programming-Language/dmd/pull/1980#issuecomment-19539800

Still, there is some hope :P
Oct 03 2013
prev sibling next sibling parent "nazriel" <spam dzfl.pl> writes:
On Thursday, 3 October 2013 at 10:42:32 UTC, Szymon Gatner wrote:
 Andrei's AMA has interesting answer:

 "One of the main D(md) contributors, Daniel Murphy is working 
 on automatic conversion tool that eventually will convert DMD's 
 C++ codebase to D."

 Is this tool already available? Are there any guidelines about 
 how to code in C++ to ease the conversion preocess (or even 
 make it possible). I would be VERY interested in such a tool 
 when the time comes so in the mean time I could slowly "fix" 
 existing C++ codebase for future conversion.

1st of all, Daniel Murphy vel yebblies rox. 2nd, I was the one who responded at Andrei's AMA thread :p 3rd, Andrei's response is the killer one, http://www.reddit.com/r/IAmA/comments/1nl9at/i_am_a_member_of_facebooks_hhvm_team_a_c_and_d/ccjusx7
Oct 03 2013
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Friday, 4 October 2013 at 06:33:10 UTC, Jacob Carlborg wrote:
 On 2013-10-04 01:24, Martin Nowak wrote:

 That's it's key feature, any attempt to first come up with a 
 generic
 tool would be doomed to fail. Also DMD's code base uses only a 
 limited
 subset of C++ which makes it more amenable to automatic 
 translation.

Having #ifdef inside declarations must be a nightmare for a tool like this. Making that properly work on generic code bases sound impossible to me.

Well, that is nothing Clang can't handle. The subset is what I was asking for - there has to be something that tool handles correctly, right?
Oct 03 2013
prev sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Szymon Gatner" <noemail gmail.com> wrote in message 
news:jqvduhyvfufpzovpyqbj forum.dlang.org...
 Andrei's AMA has interesting answer:

 "One of the main D(md) contributors, Daniel Murphy is working on automatic 
 conversion tool that eventually will convert DMD's C++ codebase to D."

 Is this tool already available? Are there any guidelines about how to code 
 in C++ to ease the conversion preocess (or even make it possible). I would 
 be VERY interested in such a tool when the time comes so in the mean time 
 I could slowly "fix" existing C++ codebase for future conversion.

The tool is available here: https://github.com/yebblies/magicport2 But as others have said, it is not meant to be a general purpose tool. The same approach could easily be applied to another large and consistent project, but not without adapting the tool to your needs. DMD uses a very nice subset of C++ (very few templates, no stl, no MI, etc) so most of the things I've needed to clean up were actually C-isms. You can get a comprehensive list of changes by looking at past commits with [DDMD] in the title, along with the un-merged ones here: https://github.com/D-Programming-Language/dmd/pull/1980
Oct 04 2013