www.digitalmars.com         C & C++   DMDScript  

digitalmars.dip.ideas - Create a full C++ parser

reply rempas <rempas tutanota.com> writes:
**Idea/Problem**

Ok, I know this will probably not happen, but I like D, and I'm 
going to give it a chance in hopes that I'll be able to convince 
you why you must do that. Now, let me start by saying that I do 
understand how hard it is to write a parser for a programming 
language, let alone C++. However, I will try to explain the 
reason that the trouble is more than worth it as, it will solve 
D's 2 biggest problems and complains that people have and not use 
it.

So let me first start by saying that I think that the D language 
is the best language that we currently have. While not without 
its flaws and things that could be massively improved, it's still 
better than the second best, which is C++. However, D has a very 
big disadvantage, which is library support. A lot of big and 
powerful libraries are written in C++ and while there is the 
ability to manually create the bindings, it can be a real pain to 
manually make it for big projects and having to keep up with 
every update (especially major version ones).

Now, you might tell me that all that aren't something new, and 
you already knew them, but what I'm going to present is that, the 
only thing that stops D from not only been more popular than it 
is, but also, been the most used language that there is out 
there, is a full compatibility with C++! Yeah, I know that it 
sounds just a bold theory, and you may deem that the risk isn't 
worth the trouble but, let me ask you one thing. What's the 
biggest reason D hasn't caught up with big languages? What is the 
number one complaint that people make about D (the garbage 
collector is number 2)? Yes, libraries!

Now, D has "importC" and, lots of C++ libraries have C bindings, 
but the problem is, C doesn't have classes and, you will need to 
do manual work to create a "D way" of using the library. Also, 
not every C++ library has C bindings. For example, 
[Louvre](https://github.com/CuarzoSoftware/Louvre) does not! If I 
want to use it from D, I have to:

* Wait for it to create C bindings (and hope they are maintained 
in the future)
* Manually port the library myself. Which also includes porting 
other C++ libraries because I will get start with the 
[weston-example](https://github.com/CuarzoSoftware/Louvre/tree/main/src/examples/
ouvre-weston-clone) that they showcase in their repos.

**Improvements**

Implementing a C++ parser will have the following advantages:
* C++ libraries will be able to natively been used in D. This 
includes "macros" and templates that have not been initialized in 
the actual library and would need an *additional* initialization 
from the project that would use them (making the process even 
more tedious, slow and overall annoying).

* C++ and D code will be able to be combined, giving the ability 
for any C++ project to more easily and smoothly get fully transit 
to D. That will bring even more popularity and trust to the 
language.

* C++ has smart pointers, which means that we will be able to use 
C++'s standard library for performance sensitive projects, 
solving D's number 2 complain (which is the garbage collector) 
and giving even more trust to people to see D as a real 
competitor that can get the place of C++.

**Implementation**

First, such project will require a great knowledge of the D 
compiler and great skills of writing an efficient parser. That's 
why, if anyone is to do it, it better be the DMD contributors. 
Second, this is something that will take time, so I believe that 
the best approach is to try parsing some real libraries, keep 
improving the parser little by little. Implement, test, fix bugs, 
test, and repeat! Let's start by the STL library (so we can use 
the smart pointers as fast as possible) and then, move to more 
common and big libraries. With steady work, in 2–3 years, we will 
hopefully have a fully working C++ parser without any bugs. Or at 
least one that can parse the biggest and most important C++ 
libraries.
Jun 14
next sibling parent rempas <rempas tutanota.com> writes:
On Saturday, 14 June 2025 at 11:39:01 UTC, rempas wrote:

 [**Idea/Problem** .. most important C++ libraries]


Oh and something I forgot to say. A small parser would of course 
mean a preprocessor. Now, this gives one more advantages of not 
requiring an external C/C++ compiler (both for C and C++). This 
will also give us the ability to be able to not require extra 
files and been able to add the headers in the D files. Something 
like the following:

```d
importC <gtk/gtk.h>; // Header, as a C file (including C11 
features)
importCXX <iostream>; // Header, as a C++ file
importC cpp_module; // C++ module (no reason for the two "XX" in 
the end as, C has no modules!)

void main() {
    std.cout << "Hello from C++'s print function!\n";
}
```
Jun 14
prev sibling next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
If you look back in the archives you can see posts from Walter during 
the mid 2000's saying that he did not want C++ binding in D.

Too complex, too much effort for very little gain.

Right now our AST supports a subset of C++, and does not have the 
capability to handle C++ templates. It works for C + COM and not much 
more than that.

An example of a C++ feature which we do not support is multiple 
inheritance. Our AST, semantic analysis and codegen do not support it. 
We cannot bind to it. And this is before we get into compiler specific 
stuff like type information. Dmd is even missing the Windows 64bit 
exception handling of MSVC.

Now consider C, we support most of what C does. What we are missing is 
some semantics of macros, some types and of course typedef, but overall 
the compiler is fully capable of handling it.

ImportC is just a parser with automatic calling out to the macro 
preprocessor sure, but that is because all the AST and semantic analysis 
is in place and mature.

Adding C++ support isn't a 2-3 year project, even if it was just a 
parser (its wayyyy more complex than you are thinking it is). Its all 
this other stuff.

If somebody wants to take this on, here are a list of projects that you 
can prove yourself on:

1. Supports Win64 exceptions to dmd (approved by Walter)
2. Add typedef to dmd (requires a DIP)
3. Implement a macro processor and figure out how to define the 
predefined macros for each target and then make system headers work out 
of the box.
4. Implement 16bit float type (requires a DIP)

These four things are not controversial, at least not compared to stuff 
like multiple inheritance. If you can implement them, then you might 
have a chance to succeed with a ImportC++ feature.

I suspect everyone would love to have ImportC++, the question is how 
much work it would take, and right now its well beyond the benefits.
Jun 14
parent reply monkyyy <crazymonkyyy gmail.com> writes:
On Saturday, 14 June 2025 at 12:19:47 UTC, Richard (Rikki) Andrew 
Cattermole wrote:

 
 If somebody wants to take this on, here are a list of projects 
 that you can prove yourself on:

 1. Supports Win64 exceptions to dmd (approved by Walter)
 2. Add typedef to dmd (requires a DIP)
 3. Implement a macro processor and figure out how to define the 
 predefined macros for each target and then make system headers 
 work out of the box.
 4. Implement 16bit float type (requires a DIP)


What are you doing? This isnt happening, why are you giving out a 
todo list.

Toxic optimism
Jun 14
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 15/06/2025 12:44 AM, monkyyy wrote:

 On Saturday, 14 June 2025 at 12:19:47 UTC, Richard (Rikki) Andrew 
 Cattermole wrote:

 If somebody wants to take this on, here are a list of projects that 
 you can prove yourself on:

 1. Supports Win64 exceptions to dmd (approved by Walter)
 2. Add typedef to dmd (requires a DIP)
 3. Implement a macro processor and figure out how to define the 
 predefined macros for each target and then make system headers work 
 out of the box.
 4. Implement 16bit float type (requires a DIP)


 
 What are you doing? This isnt happening, why are you giving out a todo 
 list.
 
 Toxic optimism


If someone was willing to shepherd this, they should understand:

1. This isn't a small amount of work.
2. Have a path forward if they were willing.

I would much rather explain why the knee-jerk reaction to the concept of 
ImportC++ and have it understood why it isn't planned for D, than for us 
to tell someone 'no' without them understanding nor able to make 
progress on. Who knows? They might be able to.

I will give an example where this policy of mine is a positive.
Yesterday I got approval to have my DFA engine merged as long as its 
self contained and remains behind a preview switch in terms of scope so 
that it can be experimented with. The reason I started work on it? 
Because I understood that Walter can't put more time into this aspect of 
the compiler and its a requirement for us to ever have RC in the 
language. It would've shortened many months leading up to the start of 
implementation if I had known that I had to take charge of the 
implementation, not just the design.

As a community we are very bad at helping to onboard people into 
shepherding of new features, and I want to see that fixed.
Jun 14
prev sibling parent Lance Bachmeier <no spam.net> writes:
On Saturday, 14 June 2025 at 11:39:01 UTC, rempas wrote:


 Now, D has "importC" and, lots of C++ libraries have C 
 bindings, but the problem is, C doesn't have classes and, you 
 will need to do manual work to create a "D way" of using the 
 library. Also, not every C++ library has C bindings. For 
 example, [Louvre](https://github.com/CuarzoSoftware/Louvre) 
 does not! If I want to use it from D, I have to:

 * Wait for it to create C bindings (and hope they are 
 maintained in the future)
 * Manually port the library myself. Which also includes porting 
 other C++ libraries because I will get start with the 
 [weston-example](https://github.com/CuarzoSoftware/Louvre/tree/main/src/examples/
ouvre-weston-clone) that they showcase in their repos.


I think the best way forward would be taking advantage of the 
recent work on SWIG. Their last major release added experimental 
support for C: https://swig.org/Doc4.3/C.html#C
Jun 14