Digital Mars - D.gnu - Compiling DMD frontend in Linux

I'm working on getting DMD to compile in Linux using GCC.  I've had to 
change quite a lot of stuff - differences in compilers, missing code, 
things like that.  I'll be giving a more detailed report later.

It's currently compiling all the objects, but I haven't tried linking 
them.  There's a lot of stub functions to write.

Jul 22 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Cool!
It would have been great when you would have reported this before as I
also already have done some of this.
Jan



Burton Radons wrote:

 I'm working on getting DMD to compile in Linux using GCC.  I've had to
 change quite a lot of stuff - differences in compilers, missing code,
 things like that.  I'll be giving a more detailed report later.

 It's currently compiling all the objects, but I haven't tried linking
 them.  There's a lot of stub functions to write.

Jul 22 2002

↑ ↓ ← → Burton Radons <loth users.sourceforge.net> writes:

Jan Knepper wrote:

 It would have been great when you would have reported this before as I
 also already have done some of this.



Oh, it just took this night anyway, unless if you have other code.  It's 
(DMD, that is) now compiling a test properly and is throwing an 
unimplemented on Module::genobjfile, which is supposed to do everything 
in the backend.  So everything past that point is backend.

I'm not interested in working on BrightD, so I'm now going ahead with 
writing my own backend.  I'll put down lots of documentation as I figure 
out things and make a zip sometime.


 Burton Radons wrote:
 
I'm working on getting DMD to compile in Linux using GCC.  I've had to
change quite a lot of stuff - differences in compilers, missing code,
things like that.  I'll be giving a more detailed report later.

It's currently compiling all the objects, but I haven't tried linking
them.  There's a lot of stub functions to write.

Jul 22 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Burton Radons wrote:

 It would have been great when you would have reported this before as I
 also already have done some of this.


 (DMD, that is) now compiling a test properly and is throwing an
 unimplemented on Module::genobjfile, which is supposed to do everything
 in the backend.  So everything past that point is backend.


Well, it only took me an hour or two... So it's not that bad...
One of the cute things to know is that Walter seems to use a compiler trick
in DMC++ where he includes total.h as first header for every source file.

 I'm not interested in working on BrightD, so I'm now going ahead with
 writing my own backend.  I'll put down lots of documentation as I figure
 out things and make a zip sometime.


You mean, you are not interested in the GLUE layer for the D-front-end and
the GCC-back-end?
What are you writing a back-end for?
If you want to make if 'public' we might be able to setup a newsgroup or add
it to http://www.opend.org/

Jan

Jul 22 2002

↑ ↓ ← → Burton Radons <loth users.sourceforge.net> writes:

Jan Knepper wrote:

 Burton Radons wrote:
 
 
It would have been great when you would have reported this before as I
also already have done some of this.


(DMD, that is) now compiling a test properly and is throwing an
unimplemented on Module::genobjfile, which is supposed to do everything
in the backend.  So everything past that point is backend.


 Well, it only took me an hour or two... So it's not that bad...
 One of the cute things to know is that Walter seems to use a compiler trick
 in DMC++ where he includes total.h as first header for every source file.



Ah, I was wondering why I needed to add so many includes.  The only 
frontend code that's missing was stringbuffer.


I'm not interested in working on BrightD, so I'm now going ahead with
writing my own backend.  I'll put down lots of documentation as I figure
out things and make a zip sometime.


 You mean, you are not interested in the GLUE layer for the D-front-end and
 the GCC-back-end?
 What are you writing a back-end for?



Entertainment, education.  Also, using GNU as a backend doesn't fit in 
my long-term plan of a compiler-in-a-library.  I want to eventually 
convert DMD, with the backend, into D, and allow getting the parse tree 
using a single method call.  I'd convert it to D now but I'm sick of 
Windows.


 If you want to make if 'public' we might be able to setup a newsgroup or add
 it to http://www.opend.org/


A bit premature but thanks for the offer.

The code:

     int main ()
     {
         return 45;
     }

Now creates a correct object file.  I'm 0.01% done.  :-)

Jul 22 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Burton Radons wrote:

 Jan Knepper wrote:

 Burton Radons wrote:


It would have been great when you would have reported this before as I
also already have done some of this.


(DMD, that is) now compiling a test properly and is throwing an
unimplemented on Module::genobjfile, which is supposed to do everything
in the backend.  So everything past that point is backend.


 Well, it only took me an hour or two... So it's not that bad...
 One of the cute things to know is that Walter seems to use a compiler trick
 in DMC++ where he includes total.h as first header for every source file.


 Ah, I was wondering why I needed to add so many includes.  The only
 frontend code that's missing was stringbuffer.


Are you sure?
I am pretty sure the following header files were missing...
cc.h, cgcv.h, code.h, context.h, dt.h, el.h, global.h, id.h, oper.h, outbuf.h,
port.h, stringtable.h, type.h.

Most significant is id.h as I think it contains "class Id" with quite a few
static
members used throughout the code.

I'm not interested in working on BrightD, so I'm now going ahead with
writing my own backend.  I'll put down lots of documentation as I figure
out things and make a zip sometime.


 You mean, you are not interested in the GLUE layer for the D-front-end and
 the GCC-back-end?
 What are you writing a back-end for?


 my long-term plan of a compiler-in-a-library.  I want to eventually
 convert DMD, with the backend, into D, and allow getting the parse tree
 using a single method call.  I'd convert it to D now but I'm sick of
 Windows.


<g>

 If you want to make if 'public' we might be able to setup a newsgroup or add
 it to http://www.opend.org/


 A bit premature but thanks for the offer.

 The code:

      int main ()
      {
          return 45;
      }

 Now creates a correct object file.  I'm 0.01% done.  :-)


Are you just having a lot of time on your hands?

Jan

Jul 22 2002

↑ ↓ ← → "Walter" <walter digitalmars.com> writes:

"Jan Knepper" <jan smartsoft.cc> wrote in message
news:3D3C3E84.D8F932DC smartsoft.cc...
 Most significant is id.h as I think it contains "class Id" with quite a


 members used throughout the code.


id.h and id.c are generated by the file idgen.c, which should be included
with the source code.

Jul 22 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Walter wrote:

 "Jan Knepper" <jan smartsoft.cc> wrote in message
 news:3D3C3E84.D8F932DC smartsoft.cc...
 Most significant is id.h as I think it contains "class Id" with quite a


 members used throughout the code.


 id.h and id.c are generated by the file idgen.c, which should be included
 with the source code.


Thanks!

Jan

Jul 22 2002

↑ ↓ ← → Burton Radons <loth users.sourceforge.net> writes:

Jan Knepper wrote:

 Burton Radons wrote:
 
 
Jan Knepper wrote:


Burton Radons wrote:



It would have been great when you would have reported this before as I
also already have done some of this.


(DMD, that is) now compiling a test properly and is throwing an
unimplemented on Module::genobjfile, which is supposed to do everything
in the backend.  So everything past that point is backend.


One of the cute things to know is that Walter seems to use a compiler trick
in DMC++ where he includes total.h as first header for every source file.


frontend code that's missing was stringbuffer.


 Are you sure?
 I am pretty sure the following header files were missing...
 cc.h, cgcv.h, code.h, context.h, dt.h, el.h, global.h, id.h, oper.h, outbuf.h,
 port.h, stringtable.h, type.h.



I meant stringtable there.  The rest are for the backend, with a couple 
pieces that appear to have been moved to other files (global and outbuf).

dchar is another one (I don't know what header file it's from), and a 
source of a small problem - the code assumes wchar_t is two bytes at 
several points.  Not a large thing, but if your produced strings are 
nonsensical, you now know why.


 Most significant is id.h as I think it contains "class Id" with quite a few
static
 members used throughout the code.



Walter noted that it's generated.  impcnvtab.c is also generated, from 
impcnvgen.c.


I'm not interested in working on BrightD, so I'm now going ahead with
writing my own backend.  I'll put down lots of documentation as I figure
out things and make a zip sometime.


the GCC-back-end?
What are you writing a back-end for?


my long-term plan of a compiler-in-a-library.  I want to eventually
convert DMD, with the backend, into D, and allow getting the parse tree
using a single method call.  I'd convert it to D now but I'm sick of
Windows.





I would be willing to have an internal API that can be matched up with 
GCC, but I could only compromise with generating RTI, since producing 
this machine-specific code (the syntax is inspecific, but you need to 
know the machine and calling structure to use it) is not a great 
inconvenience and I'll have to move towards that eventually anyway.  For 
BrightD, OpenD, GLUE, or whatever you want to call it, converting D's 
node tree to GCC's is more your ideal, but for me it adds too much 
complexity.


If you want to make if 'public' we might be able to setup a newsgroup or add
it to http://www.opend.org/



The code:

     int main ()
     {
         return 45;
     }

Now creates a correct object file.  I'm 0.01% done.  :-)


 Are you just having a lot of time on your hands?


At times.  Hello, world! is now working; that makes it about 0.1%. 
Walter, I found that the code:

     printf ("Hello, world!", "Foo foo foo!");

Was producing the most unexpected:

     printf ("Hello, world!", (wchar [] [12]) (wchar [12]) "Foo foo foo!");

I guess the code generator is assuming the parser knows what it's doing 
and is producing nonsense as a result.  I've previously clocked it as 
being the same as "(wchar *)".

Jul 23 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Burton Radons wrote:

 Are you sure?
 I am pretty sure the following header files were missing...
 cc.h, cgcv.h, code.h, context.h, dt.h, el.h, global.h, id.h, oper.h, outbuf.h,
 port.h, stringtable.h, type.h.


 I meant stringtable there.  The rest are for the backend, with a couple
 pieces that appear to have been moved to other files (global and outbuf).

 dchar is another one (I don't know what header file it's from), and a
 source of a small problem - the code assumes wchar_t is two bytes at
 several points.  Not a large thing, but if your produced strings are
 nonsensical, you now know why.


I think Walter will give you dchar.h...

 Most significant is id.h as I think it contains "class Id" with quite a few
static
 members used throughout the code.


 impcnvgen.c.


OK, let's check into that too!

Entertainment, education.  Also, using GNU as a backend doesn't fit in
my long-term plan of a compiler-in-a-library.  I want to eventually
convert DMD, with the backend, into D, and allow getting the parse tree
using a single method call.  I'd convert it to D now but I'm sick of
Windows.




 GCC, but I could only compromise with generating RTI, since producing
 this machine-specific code (the syntax is inspecific, but you need to
 know the machine and calling structure to use it) is not a great
 inconvenience and I'll have to move towards that eventually anyway.  For
 BrightD, OpenD, GLUE, or whatever you want to call it, converting D's
 node tree to GCC's is more your ideal, but for me it adds too much
 complexity.


OK, you're using the D-front-end to target a different architecture. Understood.

 Are you just having a lot of time on your hands?


 Walter, I found that the code:
      printf ("Hello, world!", "Foo foo foo!");
 Was producing the most unexpected:
      printf ("Hello, world!", (wchar [] [12]) (wchar [12]) "Foo foo foo!");
 I guess the code generator is assuming the parser knows what it's doing
 and is producing nonsense as a result.  I've previously clocked it as
 being the same as "(wchar *)".


<g>
I guess that is part of the deal with a compiler that is in progress.
I don't think there is anything else the code generator can do. It is being
called by
the front-end and lives on data provided by the front-end. The parse should
deliver the
correct 'tree' structure to the code generator to generate the code from.

Jan

Jul 23 2002

↑ ↓ ← → Jonathan Andrew <jon ece.arizona.edu> writes:

 <g>
 I guess that is part of the deal with a compiler that is in progress.
 I don't think there is anything else the code generator can do. It is being
called by
 the front-end and lives on data provided by the front-end. The parse should
deliver the
 correct 'tree' structure to the code generator to generate the code from.
 
 Jan
 


Is there any information about the output that the D front end creates?
I know the gcc tree structure is woefully undocumented. Also, if
anybody has gotten the D front end to compile under Linux,
any chance of posting a zip or tarball of the ported code on
opend.org so others (like me) can play around with it without
reinventing the wheel? (albeit a complicated wheel.)

Thanks,
-Jon

Jul 23 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Jonathan Andrew wrote:

 <g>
 I guess that is part of the deal with a compiler that is in progress.
 I don't think there is anything else the code generator can do. It is being
called by
 the front-end and lives on data provided by the front-end. The parse should
deliver the
 correct 'tree' structure to the code generator to generate the code from.


 Is there any information about the output that the D front end creates?


It's called Walter Bright <g>

 I know the gcc tree structure is woefully undocumented. Also, if
 anybody has gotten the D front end to compile under Linux,
 any chance of posting a zip or tarball of the ported code on
 opend.org so others (like me) can play around with it without
 reinventing the wheel? (albeit a complicated wheel.)


I am working on that and as soon as I have it all done (just compile I mean) it
indeed will
appear on opend.org. Actually it will be in CVS and a HEAD tarball will be
created daily.

Jan

Jul 23 2002

↑ ↓ ← → Jonathan Andrew <jon ece.arizona.edu> writes:

Jan Knepper wrote:
 
 I am working on that and as soon as I have it all done (just compile I mean)
it indeed will
 appear on opend.org. Actually it will be in CVS and a HEAD tarball will be
created daily.
 
 Jan
 


Fantastic! I'm really looking forward to it.

-Jon

Jul 23 2002

↑ ↓ ← → Burton Radons <loth users.sourceforge.net> writes:

Jonathan Andrew wrote:

 
 <g>
 I guess that is part of the deal with a compiler that is in progress.
 I don't think there is anything else the code generator can do. It is 
 being called by
 the front-end and lives on data provided by the front-end. The parse 
 should deliver the
 correct 'tree' structure to the code generator to generate the code from.


 Is there any information about the output that the D front end creates?



It's not too complex - it's a bunch of fully formed structs, unlike 
GCC's generic LISPish.  But I'm adding header documentation as I go.

 I know the gcc tree structure is woefully undocumented. Also, if
 anybody has gotten the D front end to compile under Linux,
 any chance of posting a zip or tarball of the ported code on
 opend.org so others (like me) can play around with it without
 reinventing the wheel? (albeit a complicated wheel.)


Oh, sure:

http://amateur-scrolls.sourceforge.net/old/dli-0.0.1.tar.gz

My additions, outside of fixing it to compile and behave properly in 
Linux, is mostly restrained to "machine.h" and "machine-i386.cc".  If 
you strip off the backend entirely, the main thing you'll need to fix up 
is the stringtable implementation, which I just stubbed as a linear list.

Jul 24 2002

↑ ↓ ← → Jonathan Andrew <jon ece.arizona.edu> writes:

Burton Radons wrote:

 It's not too complex - it's a bunch of fully formed structs, unlike 
 GCC's generic LISPish.  But I'm adding header documentation as I go.
 


This is one thing that I was unsure of, from what I've read, the RTL
is LISPish, but before that comes a syntax tree, which should just
be a bunch of tree structs, right? Maybe I am confused on this.

 
 Oh, sure:
 
 http://amateur-scrolls.sourceforge.net/old/dli-0.0.1.tar.gz
 
 My additions, outside of fixing it to compile and behave properly in 
 Linux, is mostly restrained to "machine.h" and "machine-i386.cc".  If 
 you strip off the backend entirely, the main thing you'll need to fix up 
 is the stringtable implementation, which I just stubbed as a linear list.


Thanks!

-Jon

Jul 24 2002

↑ ↓ ← → "Walter" <walter digitalmars.com> writes:

"Burton Radons" <loth users.sourceforge.net> wrote in message
news:3D3D3B83.60203 users.sourceforge.net...
 dchar is another one (I don't know what header file it's from), and a
 source of a small problem - the code assumes wchar_t is two bytes at
 several points.


Those dependencies are bugs, could you send me the file/line of them?

dchar is more or less my failed attempt at a unified char/wchar/mbcs header.

Jul 23 2002

↑ ↓ ← → Burton Radons <loth users.sourceforge.net> writes:

Walter wrote:

 "Burton Radons" <loth users.sourceforge.net> wrote in message
 news:3D3D3B83.60203 users.sourceforge.net...
 
dchar is another one (I don't know what header file it's from), and a
source of a small problem - the code assumes wchar_t is two bytes at
several points.


 Those dependencies are bugs, could you send me the file/line of them?



In lexer.c search for "case '\\':" and the StringConstant functions. 
Also, there are a couple uses of writeword where writedchar is intended, 
and a couple "/ 2" snippets should be "/ sizeof (wchar_t)", or perhaps 
"dchar".  Also charConstant, but that's dead code anyway.  Sorry that I 
can't be more specific, but I passed the file through indent.


 dchar is more or less my failed attempt at a unified char/wchar/mbcs header.


I'd assumed it was a failed attempt at UTF-8, actually.  :-)

Jul 24 2002

↑ ↓ ← → "Steven Shaw" <steven_shaw iprimus.com.au> writes:

"Burton Radons" <loth users.sourceforge.net> wrote in message
news:3D3C378A.3050508 users.sourceforge.net...
 Jan Knepper wrote:

 Burton Radons wrote:


It would have been great when you would have reported this before as I
also already have done some of this.


(DMD, that is) now compiling a test properly and is throwing an
unimplemented on Module::genobjfile, which is supposed to do everything
in the backend.  So everything past that point is backend.


 Well, it only took me an hour or two... So it's not that bad...
 One of the cute things to know is that Walter seems to use a compiler




 in DMC++ where he includes total.h as first header for every source




 Ah, I was wondering why I needed to add so many includes.  The only
 frontend code that's missing was stringbuffer.


I'm not interested in working on BrightD, so I'm now going ahead with
writing my own backend.  I'll put down lots of documentation as I figure
out things and make a zip sometime.


 You mean, you are not interested in the GLUE layer for the D-front-end




 the GCC-back-end?
 What are you writing a back-end for?



 Entertainment, education.  Also, using GNU as a backend doesn't fit in
 my long-term plan of a compiler-in-a-library.  I want to eventually
 convert DMD, with the backend, into D, and allow getting the parse tree
 using a single method call.  I'd convert it to D now but I'm sick of
 Windows.


 If you want to make if 'public' we might be able to setup a newsgroup or




 it to http://www.opend.org/


 A bit premature but thanks for the offer.

 The code:

      int main ()
      {
          return 45;
      }

 Now creates a correct object file.  I'm 0.01% done.  :-)



You might want to consider Gnu lightning or cgen
(http://sources.redhat.com/cgen/) as starting points.
Althought that will be less educational!

Jul 24 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Burton Radons wrote:

 Well, it only took me an hour or two... So it's not that bad...
 One of the cute things to know is that Walter seems to use a compiler trick
 in DMC++ where he includes total.h as first header for every source file.


 frontend code that's missing was stringbuffer.


You can do the same with GCC.
I do not know which compiler you are using. Just use as compiler switch -include
total.h and safe yourself a lot of changes!

Jan

Jul 24 2002

↑ ↓ ← → Richard Levitte <levitte stacken.kth.se> writes:

Jan Knepper wrote:
 You mean, you are not interested in the GLUE layer for the D-front-end and
 the GCC-back-end?
 What are you writing a back-end for?


Actually, I'm rather glad he does, it'll show me how to do it on other 
platforms, hopefully.

Hi, BTW.  I stumbled into this yesterday, and decided D is definitely 
worth looking at, and I'm already thinking of porting to a different 
operating system (VMS, and that's why an already written backend might 
be a good thing to have around).

Jan 12 2003

↑ ↓ ← → "Walter" <walter digitalmars.com> writes:

"Richard Levitte" <levitte stacken.kth.se> wrote in message
news:avt0vk$u7q$1 digitaldaemon.com...
 Hi, BTW.  I stumbled into this yesterday, and decided D is definitely
 worth looking at, and I'm already thinking of porting to a different
 operating system (VMS, and that's why an already written backend might
 be a good thing to have around).


Glad to have you join us! -Walter

Jan 13 2003

↑ ↓ ← → Burton Radons <loth users.sourceforge.net> writes:

Status update.  The last few days I've been looking into various 
intermediate representation systems.  I'll be going to some variety in 
the future, but for now I decided to stick with the crappy stack-based 
code it's currently producing.  I was actually interested in LCC's 
system, but when I looked at the produced code I saw that it was really 
horrible.  This is good, as it's comparable to GCC in the computer 
language shootout, so producing cruddy code is not a death knell by any 
means, although it has a bad worst-case.

All integer arithmetic is done, and some of floating-point.  Locals are 
in and I'm just now working on arrays and my next hurdle is to start to 
port, compile and use Phobos.

Jul 27 2002

↑ ↓ ← → Jan Knepper <jan smartsoft.cc> writes:

Which compiler are you using to compile the d-font-end?



Burton Radons wrote:

 Status update.  The last few days I've been looking into various
 intermediate representation systems.  I'll be going to some variety in
 the future, but for now I decided to stick with the crappy stack-based
 code it's currently producing.  I was actually interested in LCC's
 system, but when I looked at the produced code I saw that it was really
 horrible.  This is good, as it's comparable to GCC in the computer
 language shootout, so producing cruddy code is not a death knell by any
 means, although it has a bad worst-case.

 All integer arithmetic is done, and some of floating-point.  Locals are
 in and I'm just now working on arrays and my next hurdle is to start to
 port, compile and use Phobos.

Jul 27 2002

↑ ↓ ← → Burton Radons <loth users.sourceforge.net> writes:

It's that time again.  Classes are working, as is TypeInfo.  TypeInfo 
took quite a bit of code and looking-into to get operational; you have 
to look for a VarExp referring to a TypeInfoDeclaration and output it in 
the module, generating at least an instance.  I go all out and generate 
both the instance and a subclass of TypeInfo to make everything 
on-the-level; partial subtyping works (creating a specialised vtable but 
using TypeInfo as the class), but is just asking for problems to show up 
in low-level user code.

The milestone I'm aiming for here is to finish up the arrays without 
doing any stubbing on the way.  Once that is done and I put up version 
0.0.2, the only missing language features will be exceptions and 
threading concerns, long, complex, and associative arrays, switch, and 
other things.  This would make me about 30% finished, so the body is 
already done, with the easier gaps ahead.

Aug 03 2002