www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - [OT?] C compiler written form scratch in D

reply "Stefan Koch" <uplink.coder googlemail.com> writes:
Hello,

I'd like to announce that I am going to be writing a C-compiler 
in D.
Without flex or bison or anything like that.
Just pure handwritten D.
I will shoot videos of my progress, and I will explain how a 
compiler really works.

If you think that is like HandmadeHero you are right!
Caseys videos inspired my to do this.

I hope that some of you will be watching this.

Please tell me what you think if that announcement, and feel free 
to ask anything you like.

Regards Stefan (Uplink_Coder).
Dec 07 2014
next sibling parent reply "MattCoder" <stopthespam mail.com> writes:
On Sunday, 7 December 2014 at 19:13:42 UTC, Stefan Koch wrote:

 Please tell me what you think if that announcement, and feel 
 free to ask anything you like.


Nice and some useful links:

https://www.reddit.com/r/programming/comments/2latu2/c4_c_in_4_functions/
http://homepage.ntlworld.com/edmund.grimley-evans/cc500/
https://github.com/alexfru/SmallerC

good luck,

Matheus.
Dec 07 2014
parent "Stefan Koch" <uplink.coder googlemail.com> writes:
On Sunday, 7 December 2014 at 20:17:30 UTC, MattCoder wrote:

 On Sunday, 7 December 2014 at 19:13:42 UTC, Stefan Koch wrote:

 Please tell me what you think if that announcement, and feel 
 free to ask anything you like.


 Nice and some useful links:

 https://www.reddit.com/r/programming/comments/2latu2/c4_c_in_4_functions/
 http://homepage.ntlworld.com/edmund.grimley-evans/cc500/
 https://github.com/alexfru/SmallerC

 good luck,

 Matheus.


Thanks for thoose links!
Intresting stuff.
Dec 07 2014
prev sibling next sibling parent "Dan Cirnat" <dan alt.md> writes:
On Sunday, 7 December 2014 at 19:13:42 UTC, Stefan Koch wrote:

 I'd like to announce that I am going to be writing a C-compiler 
 in D.

 I will shoot videos of my progress, and I will explain how a 
 compiler really works.


I would be very interested in the videos, can't wait. :)
Dec 08 2014
prev sibling next sibling parent reply Shriramana Sharma via Digitalmars-d-announce writes:
On Monday, December 8, 2014, Stefan Koch via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:


 Please tell me what you think if that announcement, and feel free to ask
 anything you


Please do post updates here. I think it would be a great show-casing of D's
capabilities.

Just a suggestion and I may be totally off mark here, but will you have it
directly output machine code? Or just the front end? I keep hearing praises
of LLVM back-end so if your compiler outputs the LLVM IR code then maybe
you can have the rest of the benefit from LLVM.

But of course, if this is a PoC that D Caen be used to write a full
compiler to output machine code then LLVM's off...


--=20
Shriramana Sharma =E0=AE=B6=E0=AF=8D=E0=AE=B0=E0=AF=80=E0=AE=B0=E0=AE=AE=E0=
=AE=A3=E0=AE=B6=E0=AE=B0=E0=AF=8D=E0=AE=AE=E0=AE=BE =E0=A4=B6=E0=A5=8D=E0=
=A4=B0=E0=A5=80=E0=A4=B0=E0=A4=AE=E0=A4=A3=E0=A4=B6=E0=A4=B0=E0=A5=8D=E0=A4=
=AE=E0=A4=BE
Dec 07 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
On Monday, 8 December 2014 at 11:08:36 UTC, Shriramana Sharma via 
Digitalmars-d-announce wrote:

 Please do post updates here. I think it would be a great 
 show-casing of D's
 capabilities.

 Just a suggestion and I may be totally off mark here, but will 
 you have it
 directly output machine code? Or just the front end? I keep 
 hearing praises
 of LLVM back-end so if your compiler outputs the LLVM IR code 
 then maybe
 you can have the rest of the benefit from LLVM.

 But of course, if this is a PoC that D Caen be used to write a 
 full
 compiler to output machine code then LLVM's off...


I want to do a C backend first.
Building an LLVM Backand out of that is a small step.
Dec 08 2014
next sibling parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
First two videos are up

think of them as beta quality!

https://www.youtube.com/watch?v=aeBSsuCCRFo
and
https://www.youtube.com/watch?v=crQk929crCE
Dec 08 2014
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 8 December 2014 at 16:28:24 UTC, Stefan Koch wrote:

 First two videos are up

 think of them as beta quality!

 https://www.youtube.com/watch?v=aeBSsuCCRFo
 and
 https://www.youtube.com/watch?v=crQk929crCE


The quality is completely shot for both audio and video. The text 
is very fuzzy even in 720p and the audio has been subject to some 
extreme lossy compression.
Dec 08 2014
next sibling parent "MattCoder" <stopthespam mail.com> writes:
On Monday, 8 December 2014 at 17:26:59 UTC, John Colvin wrote:

 ...The text is very fuzzy even in 720p and the audio has been 
 subject to some extreme lossy compression.


I don't know... in 720p I can't complain about the text but on 
the otherwise in 480 is really a bit fuzzy.

Well as a suggestion I think you could expand the window where 
you are writing the code. Or even focus entire on it.

I'm waiting for more videos. :)

Matheus.
Dec 08 2014
prev sibling parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
On Monday, 8 December 2014 at 17:26:59 UTC, John Colvin wrote:

 On Monday, 8 December 2014 at 16:28:24 UTC, Stefan Koch wrote:

 First two videos are up

 think of them as beta quality!

 https://www.youtube.com/watch?v=aeBSsuCCRFo
 and
 https://www.youtube.com/watch?v=crQk929crCE


 The quality is completely shot for both audio and video. The 
 text is very fuzzy even in 720p and the audio has been subject 
 to some extreme lossy compression.


The audio-input is very bad!
what you are hearing are not compression artifacts it's the noise 
filter hard at work.
Dec 08 2014
next sibling parent reply "John" <john.joyus gmail.com> writes:
On Monday, 8 December 2014 at 19:35:54 UTC, Stefan Koch wrote:

 On Monday, 8 December 2014 at 17:26:59 UTC, John Colvin wrote:

 On Monday, 8 December 2014 at 16:28:24 UTC, Stefan Koch wrote:

 First two videos are up

 think of them as beta quality!

 https://www.youtube.com/watch?v=aeBSsuCCRFo
 and
 https://www.youtube.com/watch?v=crQk929crCE


 The quality is completely shot for both audio and video. The 
 text is very fuzzy even in 720p and the audio has been subject 
 to some extreme lossy compression.


 The audio-input is very bad!
 what you are hearing are not compression artifacts it's the 
 noise filter hard at work.


You may have to either pause when you need to cough and sneeze or 
just edit that out. I am interested in this topic but the 
horrible quality of audio video made me puke.
Dec 08 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
On Monday, 8 December 2014 at 21:04:17 UTC, John wrote:

 On Monday, 8 December 2014 at 19:35:54 UTC, Stefan Koch wrote:



 think of them as beta quality!





 You may have to either pause when you need to cough and sneeze 
 or just edit that out. I am interested in this topic but the 
 horrible quality of audio video made me puke.



Sorry. I will shoot the next videos when I am healthy again.

I thought about writing the code beforehand, and then just go 
through and explain it. What do you think of that ?

 ibuclaw You did not misshear, I really said that :( . I was 
referring I to the fact, that I do not parse uppercase letters 
atm. Probably should edit out such non-sense.
Dec 08 2014
parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 9 Dec 2014 07:00, "Stefan Koch via Digitalmars-d-announce" <
digitalmars-d-announce puremagic.com> wrote:

 On Monday, 8 December 2014 at 21:04:17 UTC, John wrote:

 On Monday, 8 December 2014 at 19:35:54 UTC, Stefan Koch wrote:




 think of them as beta quality!






 You may have to either pause when you need to cough and sneeze or just




edit that out. I am interested in this topic but the horrible quality of
audio video made me puke.

 Sorry. I will shoot the next videos when I am healthy again.

 I thought about writing the code beforehand, and then just go through and


explain it. What do you think of that ?

  ibuclaw You did not misshear, I really said that :( . I was referring I


to the fact, that I do not parse uppercase letters atm. Probably should
edit out such non-sense.

Yah, my next question would have been: How do you intend to have a self
hosted C compiler handwritten in pure D?  :)
Dec 08 2014
parent "Stefan Koch" <uplink.coder googlemail.com> writes:
On Tuesday, 9 December 2014 at 07:58:51 UTC, Iain Buclaw via 
Digitalmars-d-announce wrote:

 On 9 Dec 2014 07:00, "Stefan Koch via Digitalmars-d-announce" <
 digitalmars-d-announce puremagic.com> wrote:

 On Monday, 8 December 2014 at 21:04:17 UTC, John wrote:

 On Monday, 8 December 2014 at 19:35:54 UTC, Stefan Koch wrote:




 think of them as beta quality!






 You may have to either pause when you need to cough and 
 sneeze or just




 edit that out. I am interested in this topic but the horrible 
 quality of
 audio video made me puke.

 Sorry. I will shoot the next videos when I am healthy again.

 I thought about writing the code beforehand, and then just go 
 through and


 explain it. What do you think of that ?

  ibuclaw You did not misshear, I really said that :( . I was 
 referring I


 to the fact, that I do not parse uppercase letters atm. 
 Probably should
 edit out such non-sense.

 Yah, my next question would have been: How do you intend to 
 have a self
 hosted C compiler handwritten in pure D?  :)


Well, I intended to extend the compiler later on.
So I can compile C+x where the X is whatever is needed to get it 
self-hosting.
Dec 09 2014
prev sibling parent Iain Buclaw via Digitalmars-d-announce writes:
On 8 December 2014 at 19:35, Stefan Koch via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Monday, 8 December 2014 at 17:26:59 UTC, John Colvin wrote:

 On Monday, 8 December 2014 at 16:28:24 UTC, Stefan Koch wrote:

 First two videos are up

 think of them as beta quality!

 https://www.youtube.com/watch?v=aeBSsuCCRFo
 and
 https://www.youtube.com/watch?v=crQk929crCE



 The quality is completely shot for both audio and video. The text is very
 fuzzy even in 720p and the audio has been subject to some extreme lossy
 compression.



 The audio-input is very bad!
 what you are hearing are not compression artifacts it's the noise filter
 hard at work.


Yah, talking ad-lib is not fun, nor an easy thing to do.

1) Script your presentation, making notes of the time it will take for
you to talk through each section.
2) Screencast without audio, following the script to your timings.
3) Get yourself a Zoom H1, and record the audio over the video.

It *is* the season to be sick and bunged in.  But that's not a good
excuse for spending 15 minutes sneezing and coughing your way through
a pre-recorded talk. :-)


By the way, on a more technical note, I was confused by some of your
comments, such as not using camel casing and D's foreach because it
makes self hosting difficult?

Maybe I misheard, but it seems absurd to my ears, but then what do I know. :-)

Iain.
Dec 08 2014
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:

 I want to do a C backend first.
 Building an LLVM Backand out of that is a small step.


There is already a very popular C to C compiler out there. It is
called cat, and come out of the box with any UNIX like system.
Dec 08 2014
next sibling parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 9 Dec 2014 00:50, "deadalnix via Digitalmars-d-announce" <
digitalmars-d-announce puremagic.com> wrote:

 On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:

 I want to do a C backend first.
 Building an LLVM Backand out of that is a small step.



 There is already a very popular C to C compiler out there. It is
 called cat, and come out of the box with any UNIX like system.


Just so happens to be compatible for D to D compilation too.  This tool is
awesome!=C2=BF?=C2=A1
Dec 09 2014
parent "Stefan Koch" <uplink.coder googlemail.com> writes:
On Tuesday, 9 December 2014 at 08:10:14 UTC, Iain Buclaw via 
Digitalmars-d-announce wrote:

 On 9 Dec 2014 00:50, "deadalnix via Digitalmars-d-announce" <
 digitalmars-d-announce puremagic.com> wrote:

 On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:

 I want to do a C backend first.
 Building an LLVM Backand out of that is a small step.



 There is already a very popular C to C compiler out there. It 
 is
 called cat, and come out of the box with any UNIX like system.


 Just so happens to be compatible for D to D compilation too.  
 This tool is
 awesome!¿?¡


cat is the fastest transcompiler I have ever seen!
Dec 09 2014
prev sibling next sibling parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2014-12-09 00:45:41 +0000, deadalnix said:


 On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:

 I want to do a C backend first.
 Building an LLVM Backand out of that is a small step.


 
 There is already a very popular C to C compiler out there. It is
 called cat, and come out of the box with any UNIX like system.


Any link? I tried to google it but it's such a generic word etc. no luck.

--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Dec 09 2014
next sibling parent reply "eles" <eles eles.com> writes:
On Tuesday, 9 December 2014 at 10:54:22 UTC, Robert M. Münch 
wrote:

 On 2014-12-09 00:45:41 +0000, deadalnix said:


 On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:





 Any link? I tried to google it but it's such a generic word 
 etc. no luck.


http://linux.die.net/man/1/cat

It was a joke. Could also say "notepad" on Windows.
Dec 09 2014
parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2014-12-09 11:15:19 +0000, eles said:


 On Tuesday, 9 December 2014 at 10:54:22 UTC, Robert M. Münch wrote:

 On 2014-12-09 00:45:41 +0000, deadalnix said:
 

 On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:




 

 Any link? I tried to google it but it's such a generic word etc. no luck.


 
 http://linux.die.net/man/1/cat
 
 It was a joke. Could also say "notepad" on Windows.


:-) At least one you catched...

I was wondering... but you know, there are so many strange names etc. 
used... who knows. Better to check.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Dec 11 2014
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 9 December 2014 at 10:54:22 UTC, Robert M. Münch
wrote:

 On 2014-12-09 00:45:41 +0000, deadalnix said:


 On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:

 I want to do a C backend first.
 Building an LLVM Backand out of that is a small step.


 
 There is already a very popular C to C compiler out there. It 
 is
 called cat, and come out of the box with any UNIX like system.


 Any link? I tried to google it but it's such a generic word 
 etc. no luck.

 --
 Robert M. Münch
 http://www.saphirion.com
 smarter | better | faster


That was a joke.

http://unixhelp.ed.ac.uk/CGI/man-cgi?cat
Dec 09 2014
prev sibling parent Shachar Shemesh <shachar weka.io> writes:
On 09/12/14 02:45, deadalnix wrote:

 On Monday, 8 December 2014 at 15:44:55 UTC, Stefan Koch wrote:

 I want to do a C backend first.
 Building an LLVM Backand out of that is a small step.


 There is already a very popular C to C compiler out there. It is
 called cat, and come out of the box with any UNIX like system.


I know it was meant as a joke, but I couldn't resist nitpicking.

When Stroustrup started out with an OO successor to C, he wrote a 
preprocessor that would convert his new syntax to C, to be passed to a C 
compiler. When the language started to evolve, that proved insufficient, 
and he wrote CFront, a C++ to C compiler.

He defines the difference between a preprocessor and a compiler along 
these lines: With a compiler, any and all compilation errors need to be 
caught by the front end. If the front end accepted your program, and the 
back end fails to compile it, it is, by definition, a compiler bug.

As such, cat (nor its close neighbor, "tac | tac"), are not a compiler, 
since it does not catch compilation errors.

Sorry for the nitpick.
Shachar
Dec 11 2014
prev sibling next sibling parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2014-12-07 19:13:41 +0000, Stefan Koch said:


 I'd like to announce that I am going to be writing a C-compiler in D.
 Without flex or bison or anything like that.
 Just pure handwritten D.


Hi, how about using PEG for parsing etc.? IMO that would be a very good 
showcase for the power of PEG And D.

--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Dec 09 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
On Tuesday, 9 December 2014 at 10:55:24 UTC, Robert M. Münch 
wrote:

 On 2014-12-07 19:13:41 +0000, Stefan Koch said:


 I'd like to announce that I am going to be writing a 
 C-compiler in D.
 Without flex or bison or anything like that.
 Just pure handwritten D.


 Hi, how about using PEG for parsing etc.? IMO that would be a 
 very good showcase for the power of PEG And D.

 --
 Robert M. Münch
 http://www.saphirion.com
 smarter | better | faster


I will use a handwritten recursive decent parser.
Since that is what I deem the easiest thing to do.
Dec 09 2014
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
 I will use a handwritten recursive decent parser.
 Since that is what I deem the easiest thing to do.


Take a look at the std.lexer proposal first.
Dec 10 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
On Wednesday, 10 December 2014 at 13:02:43 UTC, Tobias Pankrath 
wrote:

 I will use a handwritten recursive decent parser.
 Since that is what I deem the easiest thing to do.


 Take a look at the std.lexer proposal first.


Thanks for your suggestion.
In fact, I know and like std.lexer.

But this is an educational project to showcase compiler 
construction. So everything will be written from scratch.
Dec 10 2014
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Wednesday, 10 December 2014 at 17:19:28 UTC, Stefan Koch wrote:

 On Wednesday, 10 December 2014 at 13:02:43 UTC, Tobias Pankrath 
 wrote:

 I will use a handwritten recursive decent parser.
 Since that is what I deem the easiest thing to do.


 Take a look at the std.lexer proposal first.


 Thanks for your suggestion.
 In fact, I know and like std.lexer.

 But this is an educational project to showcase compiler 
 construction. So everything will be written from scratch.


I didn't want to suggest to use std.lexer, but to showcase the 
nice compile time tricks it employs, e.g. the generation of a 
Trie at compile time.
Dec 11 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
On Thursday, 11 December 2014 at 15:02:02 UTC, Tobias Pankrath 
wrote:

 I didn't want to suggest to use std.lexer, but to showcase the 
 nice compile time tricks it employs, e.g. the generation of a 
 Trie at compile time.


std.lexer is awesome, no doubt about it!
But it is not in the scope of building a dead simple compiler.

I am still about a way to simplify parsing.
It is still not dead simple :(
Dec 11 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
I will start uploading official videos at Monday.

If you are interested in qcc please join me in irc.
server : irc.freenode.net
channel : #qcc
Dec 11 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
New videos are online :)
part 1:
https://www.youtube.com/watch?v=_YAUfd41URA
part 2:
https://www.youtube.com/watch?v=RGLgiPhwskM

please give me feedback in this thread
Dec 15 2014
next sibling parent reply "TiberiuGal" <galtiberiu gmail.com> writes:
On Monday, 15 December 2014 at 12:37:18 UTC, Stefan Koch wrote:

 New videos are online :)
 part 1:
 https://www.youtube.com/watch?v=_YAUfd41URA
 part 2:
 https://www.youtube.com/watch?v=RGLgiPhwskM

 please give me feedback in this thread


Hi.

I'm an intermediate level programmer and I'm interested in
learning about compilers, so I was very exited when I red your
post, and I hope you'll continue.
A few suggestions I have:
1. maybe you should pick your target audience, and only talk to
that audience. I don't think you need to explain while loops to
people who want to learn compilers and you can't teach compilers
to people who don't get while loops.

2.  talk more about concepts and decision making instead of
source - we can follow the source, but there is no obvious answer
to "why". So, for example, you go through the source for the
lexer, but maybe you should talk instead about the lexer concept-
What is it, How will it work, What's the input and output?!

3. it was already mentioned, but it's worth stressing: you should
script your recordings.

4. There are a few easy tricks to achieve good quality audio with
standard microphones. Maybe if you script your video, and provide
the script , the audio will not matter that much.

5. remove the long pauses. we can pause the video if we need more
time to process.

So,  I'm waiting for the next videos, thank you
Dec 16 2014
parent reply "Stefan Koch" <uplink.coder googlemail.com> writes:
please tell me what concepts you would like to have explained.

perhaps I should do a live-stream for Q&A maybe ?
Dec 16 2014
parent reply "TiberiuGal" <galtiberiu gmail.com> writes:
On Tuesday, 16 December 2014 at 10:01:58 UTC, Stefan Koch wrote:

 please tell me what concepts you would like to have explained.

 perhaps I should do a live-stream for Q&A maybe ?


I don't know. Maybe Q&A could be helpful - not sure if a live 
stream is necessary ...
Dec 17 2014
parent "Stefan Koch" <uplink.coder googlemail.com> writes:
New videos are comeing soon :)
Topics : Intro to compilers
          Parseing and the AsT
          Identifier-resolution and scope
Dec 21 2014
prev sibling parent "John" <john.joyus gmail.com> writes:
On Monday, 15 December 2014 at 12:37:18 UTC, Stefan Koch wrote:

 New videos are online :)
 part 1:
 https://www.youtube.com/watch?v=_YAUfd41URA
 part 2:
 https://www.youtube.com/watch?v=RGLgiPhwskM

 please give me feedback in this thread



I have watched the videos in speed 2 (double speed) and they look 
pretty good, thanks.
Dec 29 2014
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/7/14 11:13 AM, Stefan Koch wrote:

 Hello,

 I'd like to announce that I am going to be writing a C-compiler in D.
 Without flex or bison or anything like that.
 Just pure handwritten D.
 I will shoot videos of my progress, and I will explain how a compiler
 really works.

 If you think that is like HandmadeHero you are right!
 Caseys videos inspired my to do this.

 I hope that some of you will be watching this.

 Please tell me what you think if that announcement, and feel free to ask
 anything you like.

 Regards Stefan (Uplink_Coder).


Sounds like a really interesting idea. -- Andrei
Dec 27 2014