Digital Mars - D.gnu - Study of GCC frontend and Walter's DMD compiler sources

↑ ↓ ← → andy <acoliver apache.org> writes:

Hi,

I've been studying GCC and Walter's sources.  The GCC sources are very 
much of the C world.  (specifically of the "my entire program is a 
series of #DEFINEs" persuasion).  The dmd sources are of course in C++, 
but make judicious use of C++ features.

I think it would be increadibly difficult to write a GCC front end in 
C++.  Therefore I think we'll have to port the D front end to straight C 
first, while attemping to parallel Walter's overall object structure. 
Its unfortunate because it would be nice to have a non-divergent 
versions, but I don't think there is an easier way to do it.  Any 
thoughts?  (practical thoughts only please)

The GCC front-end sources are spartanly documented; however, I posted a 
series of links, one of which is the documentation the GCC COBOL front 
end folks produced.

I've put a couple tasks on the sourceforge task manager.  I think our 
steps are (though not necessarily in this order):

1. Port DMD front end sources to C - explained above.

2. Create an overall framework/coupling for working with the GCC front 
end.  This serves two purposes.  First of all, to isolate us from the 
changing GCC front-end interface.  Second of all, the GCC front end 
stuff is ugly, we should be able to encapsulate it and produce something 
that allows us to manage it at a higher level so that we only have to 
get intimate with it once.  The rest of the time we'll work above it.

3. attach the front end to the coupling.

As I've said, I'm pretty rusty with both C and C++, so I'm hoping some 
other folks will jump in early on.  (Which is unfortunately just the 
opposite of how things usually go ;-) ).

Over the next while (unless someone shows me a practical way to attach 
the two), I'll start committing a port of the D front end to C to the 
project CVS at sourceforge.net/projects/brightd

Thoughts, comments?

-Andy

Jun 02 2002

↑ ↓ ← → Andy Walker <Andy_member pathlink.com> writes:

In article <3CFA4813.6030306 apache.org>, andy says...
Hi,

I've been studying GCC and Walter's sources.  The GCC sources are very 
much of the C world.  (specifically of the "my entire program is a 
series of #DEFINEs" persuasion).  The dmd sources are of course in C++, 
but make judicious use of C++ features.


Back from Site.  Finally.  Lots of very long days.  I did have a day off and a
few evenings, so I have been working on Dgnu.

DMD sources in C++:  I have spent a couple of days trying to find ways to 
deal with this.  My heart sank when I started looking at the DMD code and 
finally realized that this was all C++, not C.  (I actually started coding C++
before I used C, so obvious incompatibilities do not jump out at me until 
I compile the thing and thousands of error messages appear.)

One option is a sort of awk scriptie C++ to C compiler.  It does not have to be 
a full-function compiler, because Walter does not use every possible C++
syntax in his code.  Still, it will have to properly handle all the non-C 
characteristics.  I have started this,  but it is coming along very slowly.  
On the nice side, it will only have to work on the released portion of DMD, 
which is, IIRC, about 27,000 lines of code.  Miniscule, really, considering
what it does.  

Another option is to find a C++ to C compiler and use it.  I have done some
searching on the web, but I have not yet found anything.  (It is out there, 
because I recall having seen it, I just do not recall where.)  The real 
concern I have about this is that it will probably turn Walter's elegant code 
into garbage.  This offends me.

Another option is to just grit my teeth and rewrite it all by hand into C, then 
hook it up to the GCC system.  That way, I can maintain much of Walter's 
elegance, and still get something working in a reasonable time.  

Comments?

Andy Walker

I think it would be increadibly difficult to write a GCC front end in 
C++.  Therefore I think we'll have to port the D front end to straight C 
first, while attemping to parallel Walter's overall object structure. 
Its unfortunate because it would be nice to have a non-divergent 
versions, but I don't think there is an easier way to do it.  Any 
thoughts?  (practical thoughts only please)

The GCC front-end sources are spartanly documented; however, I posted a 
series of links, one of which is the documentation the GCC COBOL front 
end folks produced.

I've put a couple tasks on the sourceforge task manager.  I think our 
steps are (though not necessarily in this order):

1. Port DMD front end sources to C - explained above.

2. Create an overall framework/coupling for working with the GCC front 
end.  This serves two purposes.  First of all, to isolate us from the 
changing GCC front-end interface.  Second of all, the GCC front end 
stuff is ugly, we should be able to encapsulate it and produce something 
that allows us to manage it at a higher level so that we only have to 
get intimate with it once.  The rest of the time we'll work above it.

3. attach the front end to the coupling.

As I've said, I'm pretty rusty with both C and C++, so I'm hoping some 
other folks will jump in early on.  (Which is unfortunately just the 
opposite of how things usually go ;-) ).

Over the next while (unless someone shows me a practical way to attach 
the two), I'll start committing a port of the D front end to C to the 
project CVS at sourceforge.net/projects/brightd

Thoughts, comments?

-Andy


Andy Walker

Jun 09 2002

↑ ↓ ← → "Walter" <walter digitalmars.com> writes:

"Andy Walker" <Andy_member pathlink.com> wrote in message
news:ae1gb7$1oih$1 digitaldaemon.com...
 In article <3CFA4813.6030306 apache.org>, andy says...
Hi,

I've been studying GCC and Walter's sources.  The GCC sources are very
much of the C world.  (specifically of the "my entire program is a
series of #DEFINEs" persuasion).  The dmd sources are of course in C++,
but make judicious use of C++ features.


 Back from Site.  Finally.  Lots of very long days.  I did have a day off


 few evenings, so I have been working on Dgnu.

 DMD sources in C++:  I have spent a couple of days trying to find ways to
 deal with this.  My heart sank when I started looking at the DMD code and
 finally realized that this was all C++, not C.  (I actually started coding


 before I used C, so obvious incompatibilities do not jump out at me until
 I compile the thing and thousands of error messages appear.)

 One option is a sort of awk scriptie C++ to C compiler.  It does not have


 a full-function compiler, because Walter does not use every possible C++
 syntax in his code.  Still, it will have to properly handle all the non-C
 characteristics.  I have started this,  but it is coming along very


 On the nice side, it will only have to work on the released portion of


 which is, IIRC, about 27,000 lines of code.  Miniscule, really,


 what it does.

 Another option is to find a C++ to C compiler and use it.  I have done


 searching on the web, but I have not yet found anything.  (It is out


 because I recall having seen it, I just do not recall where.)  The real
 concern I have about this is that it will probably turn Walter's elegant


 into garbage.  This offends me.

 Another option is to just grit my teeth and rewrite it all by hand into C,


 hook it up to the GCC system.  That way, I can maintain much of Walter's
 elegance, and still get something working in a reasonable time.

 Comments?

 Andy Walker



D is all written in C++. But it doesn't use much of C++ features - the only
grief will likely be the heavy use of virtual functions. Most member
functions can be rewritten from:
    f->foo(x);
to:
    Foo_foo(f, x);

Single inheritance:
    class A { ... };
    class B : A { ... };
can be:
    typedef struct A { ... } A;
    typedef struct B { A a; ... } B;

Virtual functions can be handled by having a:
    void **vptr;
as the first member of each struct, and then initialize it in the
constructor to the vtbl[] for that struct. Calling it would require a macro
for each function:
    class A
    {    virtual int foo(args...);
    };
   A *a;
   a->foo(args);
would be:
    typedef int (*A_foo_fp)(args...);
    #define A_foo(this)    (*(A_foo_fp)(this->_vptr[0]))
    A_foo(a)(args...);

Constructors:
    class A
    {    A(args...);
    };
would be:
    A *A_ctor(A *this, args...);

Following these conventions will make the C code have a good correspondence
with the C code.

Jun 10 2002

↑ ↓ ← → Andy Walker <Andy_member pathlink.com> writes:

In article <ae1obm$22ci$1 digitaldaemon.com>, Walter says...
 One option is a sort of awk scriptie C++ to C compiler.  It does not have


 a full-function compiler, because Walter does not use every possible C++
 syntax in his code.  


D is all written in C++. But it doesn't use much of C++ features - the only
grief will likely be the heavy use of virtual functions. 


I had that feeling when I looked at it.  Didn't do any statistics, though.

Most member
functions can be rewritten from:
    f->foo(x);
to:
    Foo_foo(f, x);

Single inheritance:
    class A { ... };
    class B : A { ... };
can be:
    typedef struct A { ... } A;
    typedef struct B { A a; ... } B;

Virtual functions can be handled by having a:
    void **vptr;
as the first member of each struct, and then initialize it in the
constructor to the vtbl[] for that struct. Calling it would require a macro
for each function:
    class A
    {    virtual int foo(args...);
    };
   A *a;
   a->foo(args);
would be:
    typedef int (*A_foo_fp)(args...);
    #define A_foo(this)    (*(A_foo_fp)(this->_vptr[0]))
    A_foo(a)(args...);

Constructors:
    class A
    {    A(args...);
    };
would be:
    A *A_ctor(A *this, args...);

Following these conventions will make the C code have a good correspondence
with the C code.


That was not really how I was thinking about approaching it.  I tend to be too
brute force. 
Yours is so much better that I will do it your way.  This will be relatively
easy to implement
with the awk script.

Andy Walker

Jun 11 2002

↑ ↓ ← → andy <acoliver apache.org> writes:

 
 Back from Site.  Finally.  Lots of very long days.  I did have a day off and a
 few evenings, so I have been working on Dgnu.
 
 DMD sources in C++:  I have spent a couple of days trying to find ways to 
 deal with this.  My heart sank when I started looking at the DMD code and 
 finally realized that this was all C++, not C.  (I actually started coding C++
 before I used C, so obvious incompatibilities do not jump out at me until 
 I compile the thing and thousands of error messages appear.)
 


Yeah, at first glance I saw structs, some c and h files, no cpps and I 
thought "great its in C"... Then I took a more in depth glance and gasped.

 One option is a sort of awk scriptie C++ to C compiler.  It does not have to
be 
 a full-function compiler, because Walter does not use every possible C++
 syntax in his code.  Still, it will have to properly handle all the non-C 
 characteristics.  I have started this,  but it is coming along very slowly.  
 On the nice side, it will only have to work on the released portion of DMD, 
 which is, IIRC, about 27,000 lines of code.  Miniscule, really, considering
 what it does.  
 


I kinda like this idea because we can sync up more easily later.

 Another option is to find a C++ to C compiler and use it.  I have done some
 searching on the web, but I have not yet found anything.  (It is out there, 
 because I recall having seen it, I just do not recall where.)  The real 
 concern I have about this is that it will probably turn Walter's elegant code 
 into garbage.  This offends me.
 


yes, and we'd have to hook code into it, which would require us to read 
it... yuck.

 Another option is to just grit my teeth and rewrite it all by hand into C,
then 
 hook it up to the GCC system.  That way, I can maintain much of Walter's 
 elegance, and still get something working in a reasonable time.  
 


If you have a good handle on how to do the code generator, then I favor 
that idea.  I don't really know awk but I'll do my best to catch up. 
And its probably about time I learned.

It might help, though walter has very few hooks to the backend, if 
walter and we could agree on interfaces to the back end so that this can 
be easily plugged in and out in parallel.  Meaning we could get to where 
we just run the C-ize program on walters code and do our own work on our 
"backend" (mid layer between D frontend and GNU).  This would also 
potentially allow others to do the same perhaps for other compilers. 
Granted this is a utopian view, and I'm sure we'd spend time modifying 
the thing for subsequent releases, but I think the pain could be 
minimized into a small non-bleeding ulcer.

Writing our own divergent C version and maintaining it would be more 
painful I think.  If we can get to where the generated C version just 
hooks into the backend (or rather our abstraction of it), that would be 
sweet.  The only way it would sound like a real good idea to write it by 
hand is if Walter were willing to switch from C++ to C.  (that would 
produce the least gap and achieve higher portability, but he may have 
reasons for not wishing to do such).

-andy

 Comments?
 
 Andy Walker
 
 
I think it would be increadibly difficult to write a GCC front end in 
C++.  Therefore I think we'll have to port the D front end to straight C 
first, while attemping to parallel Walter's overall object structure. 
Its unfortunate because it would be nice to have a non-divergent 
versions, but I don't think there is an easier way to do it.  Any 
thoughts?  (practical thoughts only please)

The GCC front-end sources are spartanly documented; however, I posted a 
series of links, one of which is the documentation the GCC COBOL front 
end folks produced.

I've put a couple tasks on the sourceforge task manager.  I think our 
steps are (though not necessarily in this order):

1. Port DMD front end sources to C - explained above.

2. Create an overall framework/coupling for working with the GCC front 
end.  This serves two purposes.  First of all, to isolate us from the 
changing GCC front-end interface.  Second of all, the GCC front end 
stuff is ugly, we should be able to encapsulate it and produce something 
that allows us to manage it at a higher level so that we only have to 
get intimate with it once.  The rest of the time we'll work above it.

3. attach the front end to the coupling.

As I've said, I'm pretty rusty with both C and C++, so I'm hoping some 
other folks will jump in early on.  (Which is unfortunately just the 
opposite of how things usually go ;-) ).

Over the next while (unless someone shows me a practical way to attach 
the two), I'll start committing a port of the D front end to C to the 
project CVS at sourceforge.net/projects/brightd

Thoughts, comments?

-Andy


 
 Andy Walker

Jun 10 2002

↑ ↓ ← → "Martin M. Pedersen" <mmp www.moeller-pedersen.dk> writes:

"andy" <acoliver apache.org> wrote in message
news:3D04D8C5.50909 apache.org...

 Another option is to just grit my teeth and rewrite it all by hand into




 hook it up to the GCC system.  That way, I can maintain much of Walter's
 elegance, and still get something working in a reasonable time.




This was my first idea, because I would like a bison/C solution. However, I
came to the conclusion that D cannot be specified as a LALR(1) grammar as
required by bison. The problem I ran into was the deeply recursive
Parser::is... methods. Other LALR(N) parser generators exists, but I have no
experience with them.

 One option is a sort of awk scriptie C++ to C compiler.  It does not




 a full-function compiler, because Walter does not use every possible C++
 syntax in his code.  Still, it will have to properly handle all the




 characteristics.  I have started this,  but it is coming along very




 On the nice side, it will only have to work on the released portion of




 which is, IIRC, about 27,000 lines of code.  Miniscule, really,




 what it does.




Before trying to make it C code, we need it to compile as the C++ sources
they are. That is not trivial either. Tonight I have had luck making
"parse.c" and "lexer.c" compile MSVS6 using only automated changes using an
AWK script (attached). One of the problems is that the source is incomplete.
Until now I have been missing:

    Id
    StringTable
    StringValue
    lstring

I expect I will run into more later. Only Walter knows why they are not part
of the archive. Maybe it is just a mishap, or maybe he has his reasons not
to publish them. But it is, BTW, also part of the reason I started out
reimplementing the thing from scratch.

 I kinda like this idea because we can sync up more easily later.


That would be great. I guess that the parser is still open for major
changes, so a mechanism that will allow us to easily keep the implementation
in sync will be of great value. I don't know how difficult it would be, or
if there are reasons for Walter not to do so, but it would be very helpful
if Walter provided us with a package that was complete enough to compile and
link - at least with DMC.

Regards,
Martin M. Pedersen

Jun 10 2002

↑ ↓ ← → Andy Walker <Andy_member pathlink.com> writes:

In article <ae3a7m$qt0$1 digitaldaemon.com>, Martin M. Pedersen says...
"andy" <acoliver apache.org> wrote in message
news:3D04D8C5.50909 apache.org...

 Another option is to just grit my teeth and rewrite it all by hand into




 hook it up to the GCC system.  That way, I can maintain much of Walter's
 elegance, and still get something working in a reasonable time.




This was my first idea, because I would like a bison/C solution. However, I
came to the conclusion that D cannot be specified as a LALR(1) grammar as
required by bison. The problem I ran into was the deeply recursive
Parser::is... methods. Other LALR(N) parser generators exists, but I have no
experience with them.

 One option is a sort of awk scriptie C++ to C compiler.  It does not




 a full-function compiler, because Walter does not use every possible C++
 syntax in his code.  Still, it will have to properly handle all the




 characteristics.  I have started this,  but it is coming along very




 On the nice side, it will only have to work on the released portion of




 which is, IIRC, about 27,000 lines of code.  Miniscule, really,




 what it does.




Before trying to make it C code, we need it to compile as the C++ sources
they are. That is not trivial either. Tonight I have had luck making
"parse.c" and "lexer.c" compile MSVS6 using only automated changes using an
AWK script (attached). One of the problems is that the source is incomplete.
Until now I have been missing:

    Id
    StringTable
    StringValue
    lstring

I expect I will run into more later. Only Walter knows why they are not part
of the archive. Maybe it is just a mishap, or maybe he has his reasons not
to publish them. But it is, BTW, also part of the reason I started out
reimplementing the thing from scratch.

 I kinda like this idea because we can sync up more easily later.


That would be great. I guess that the parser is still open for major
changes, so a mechanism that will allow us to easily keep the implementation
in sync will be of great value. I don't know how difficult it would be, or
if there are reasons for Walter not to do so, but it would be very helpful
if Walter provided us with a package that was complete enough to compile and
link - at least with DMC.

Regards,
Martin M. Pedersen


Yeeahooo!  More Toys!

Andy Walker

Jun 11 2002

↑ ↓ ← → Andy Walker <Andy_member pathlink.com> writes:

In article <3D04D8C5.50909 apache.org>, andy says...


 


If you have a good handle on how to do the code generator, then I favor 
that idea.  I don't really know awk but I'll do my best to catch up. 
And its probably about time I learned.


I have A handle.  Whether it is good or not remains to be seen.  Walter's 
posts earlier today will be a big help.

Awk is ort of fun, really. I will be happy to give pointers and advice, if you
are interested..  

It might help, though walter has very few hooks to the backend, if 
walter and we could agree on interfaces to the back end so that this can 
be easily plugged in and out in parallel.  Meaning we could get to where 
we just run the C-ize program on walters code and do our own work on our 
"backend" (mid layer between D frontend and GNU).  This would also 
potentially allow others to do the same perhaps for other compilers. 
Granted this is a utopian view, and I'm sure we'd spend time modifying 
the thing for subsequent releases, but I think the pain could be 
minimized into a small non-bleeding ulcer.


I am happy to share the awk scripts.  Once I get them working better, I will
post them as "freely given to the public".  I am really not too concerned 
about adapting to release changes.  There will be some work, but not bad, 
and no matter where you go, there will be work related to release changes.

Writing our own divergent C version and maintaining it would be more 
painful I think.  If we can get to where the generated C version just 
hooks into the backend (or rather our abstraction of it), that would be 
sweet.  The only way it would sound like a real good idea to write it by 
hand is if Walter were willing to switch from C++ to C.  (that would 
produce the least gap and achieve higher portability, but he may have 
reasons for not wishing to do such).


It is just selfishness, but I do not want Walter to ever think about how he 
would change his coding practices to conform to my needs.  I love the help, 
but that Bright D front-end is just really pretty to me.  It does what it is
supposed
to do.  I would hate for Walter to muck it up, trying to adapt to some arbitrary

format.  And for the record, GNU CC is one very arbitrary format.

-andy



Andy Walker

Jun 11 2002

↑ ↓ ← → andy <acoliver apache.org> writes:

</snip>

 I have A handle.  Whether it is good or not remains to be seen.  Walter's 
 posts earlier today will be a big help.
 
 Awk is ort of fun, really. I will be happy to give pointers and advice, if you
 are interested..  
 


cool.  Yes, sounds good.

<snip/>

 
 I am happy to share the awk scripts.  Once I get them working better, I will
 post them as "freely given to the public".  I am really not too concerned 
 about adapting to release changes.  There will be some work, but not bad, 
 and no matter where you go, there will be work related to release changes.
 


Cool.  Don't worry about getting them finished, just conceptually ready.
I'll work on it.

 
 It is just selfishness, but I do not want Walter to ever think about how he 
 would change his coding practices to conform to my needs.  I love the help, 
 but that Bright D front-end is just really pretty to me.  It does what it is
 supposed
 to do.  I would hate for Walter to muck it up, trying to adapt to some
arbitrary
 format.  And for the record, GNU CC is one very arbitrary format.


Think I'm a GNU appologist or something?  Not really.  I don't like the 
premise of the GPL.

-Andy

 
-andy


 <snip>
 
 Andy Walker

Jun 11 2002