www.digitalmars.com         C & C++   DMDScript  

D - Complete D grammar spec available?

reply Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
In going through the spec, I've noticed a small number of omissions or 
undefined productions in the D grammar. For instance, on the "Declarations" 
page, under "Decl" you find:

    	BasicType BasicType2 Declarators ;
    	BasicType BasicType2 FunctionDeclarators

But 'BasicType2' is not defined anywhere (unless I simply haven't found 
it). Just after that, "Declarator" is used in the definition of 
"Declarators", but is not defined.

I realize it's not a "formal" document, and most of it we can figure out 
readily by reading the examples. I just want to make sure I understand all 
of it correctly.

-- 
dave
Apr 05 2004
parent reply "Walter" <walter digitalmars.com> writes:
I'm afraid that's all there is at the moment.
Apr 05 2004
parent reply Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
"Walter" <walter digitalmars.com> wrote:

 I'm afraid that's all there is at the moment.

Thanks, Walter. It's not a huge problem. I have begun a project to support D in Visual Studio .NET with syntax highlighting, Intellisense and all that (using the Babel SDK), so there were some points about the grammar that I wanted to make sure I had right. I can probably figure out most of it, and can examine your front-end code as well (or use it itself). If I get stuck on anything, I will post a query for you here. -- dave
Apr 05 2004
parent reply "Walter" <walter digitalmars.com> writes:
"Dave Sieber" <dsieber spamnot.sbcglobal.net> wrote in message
news:Xns94C29C6BC72BFdsiebersbc 63.105.9.61...
 "Walter" <walter digitalmars.com> wrote:

 I'm afraid that's all there is at the moment.

Thanks, Walter. It's not a huge problem. I have begun a project to support D in Visual Studio .NET with syntax highlighting, Intellisense and all

 (using the Babel SDK), so there were some points about the grammar that I
 wanted to make sure I had right.

 I can probably figure out most of it, and can examine your front-end code
 as well (or use it itself). If I get stuck on anything, I will post a

 for you here.

If you want to do an updated grammar, and put it all on one page, that would be cool.
Apr 09 2004
parent reply Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
"Walter" <walter digitalmars.com> wrote:

 If you want to do an updated grammar, and put it all on one page, that
 would be cool.

I very well may do that! It'd be a good thing to have in any case. The first reason this came up, BTW, is that the Babel SDK for Microsoft's Visual Studio .NET, which provides support for syntax highlighting, Intellisense, completion, matching, etc., can be utilized in several fashions. The simplest and quickest is via a framework they have supplied which allows you to (almost) plug in a lex-based lexer and yacc-based parser, give it a few special instructions, and they handle the rest. They assumed (not incorrectly) that most languages have a yacc-based grammar available, but you can go to the next level down and program it all yourself if you want (which I think would be the best way to do it in the long run, with your front-end code). So I was primarily interested in seeing if and how it could work, and a yacc-based grammar for D would have let me check it out quickly. I got the syntax highlighting working using Flex (easy, just make a lex spec and keyword list), but for delimiter matching, tool tips, completion, etc., they want a Yacc grammar. I started one for Bison from the online spec for D, but quickly found the missing definitions I initially enquired about here. Unfortunately, I have a lot of other work I need to finish right now, so it may be a few weeks before I can return to it. I am definitely interested in providing full support for D within Visual Studio .NET, including Intellisense and full project management. -- dave
Apr 09 2004
parent reply Brad Anderson <brad dsource.dot.org> writes:
This grammar would also be a great start for a code formatting utility 
like Jalopy for Java.  Burton had a lexer in his DIG code, too, and I've 
been wanting to combine all of this into some neat utilities for 
source-code formatting.

BA

Dave Sieber wrote:
 "Walter" <walter digitalmars.com> wrote:
 
 
If you want to do an updated grammar, and put it all on one page, that
would be cool.

I very well may do that! It'd be a good thing to have in any case. The first reason this came up, BTW, is that the Babel SDK for Microsoft's Visual Studio .NET, which provides support for syntax highlighting, Intellisense, completion, matching, etc., can be utilized in several fashions. The simplest and quickest is via a framework they have supplied which allows you to (almost) plug in a lex-based lexer and yacc-based parser, give it a few special instructions, and they handle the rest. They assumed (not incorrectly) that most languages have a yacc-based grammar available, but you can go to the next level down and program it all yourself if you want (which I think would be the best way to do it in the long run, with your front-end code). So I was primarily interested in seeing if and how it could work, and a yacc-based grammar for D would have let me check it out quickly. I got the syntax highlighting working using Flex (easy, just make a lex spec and keyword list), but for delimiter matching, tool tips, completion, etc., they want a Yacc grammar. I started one for Bison from the online spec for D, but quickly found the missing definitions I initially enquired about here. Unfortunately, I have a lot of other work I need to finish right now, so it may be a few weeks before I can return to it. I am definitely interested in providing full support for D within Visual Studio .NET, including Intellisense and full project management.

Apr 09 2004
parent reply Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
Brad Anderson <brad dsource.dot.org> wrote:

 This grammar would also be a great start for a code formatting utility
 like Jalopy for Java.  Burton had a lexer in his DIG code, too, and
 I've been wanting to combine all of this into some neat utilities for 
 source-code formatting.

I think it'd be worth doing all of these things -- and give D the proper tool support it deserves. -- dave
Apr 09 2004
parent reply Ilya Minkov <minkov cs.tum.edu> writes:
I have been looking at different parser generators and the one i hapen 
to like especially is COCO/R. It's old but nice, it's code is only a few 
kilolines, and it has been ported to many languages. Porting is a 2-step 
process, because it requieres itself to build: first make a COCO/R 
written in Java or C++ output D parsers, than port that over to D - the 
second step is optional. Nontheless i think that new tools (also ones 
created with it) should be written in D.

The parsers it generates are non-reentrant, i think this is something 
what one might think of changing when making it output D - however i'm 
not really sure since it would compromise the performance.

It is only LL(1) as it is, but on the spots where this is requiered 
arbitrary lookahead can be taken as well. I think parsing D should be 
possible with it. An example CSharp grammar has been lying around somewhere.

It outputs both parser and lexer together, and lexer has some special 
features like optimized comment support etc.

-eye

Dave Sieber schrieb:
 Brad Anderson <brad dsource.dot.org> wrote:
 
 
This grammar would also be a great start for a code formatting utility
like Jalopy for Java.  Burton had a lexer in his DIG code, too, and
I've been wanting to combine all of this into some neat utilities for 
source-code formatting.

I think it'd be worth doing all of these things -- and give D the proper tool support it deserves.

Apr 12 2004
parent reply Dave Sieber <dsieber spamnot.sbcglobal.net> writes:
Ilya Minkov <minkov cs.tum.edu> wrote:

 I have been looking at different parser generators and the one i hapen
 to like especially is COCO/R. It's old but nice, it's code is only a
 few kilolines, and it has been ported to many languages. Porting is a
 2-step process, because it requieres itself to build: first make a
 COCO/R written in Java or C++ output D parsers, than port that over to
 D - the second step is optional. Nontheless i think that new tools
 (also ones created with it) should be written in D.

I will have to take a look at COCO, based on your praise for it here! I'd been looking around at various parser generators over the years, and ANTLR looks like a potential good choice, although I haven't worked with it yet. After using yacc for several projects (most recently a C# parser) I am ready for almost anything else. Using lex/yacc is like programming in C -- a move backwards :-)
 It is only LL(1) as it is, but on the spots where this is requiered 
 arbitrary lookahead can be taken as well. I think parsing D should be 
 possible with it. An example CSharp grammar has been lying around
 somewhere. 

Yeah, D shouldn't be a problem, it's nowhere near as bad as C++, and probably about the same complexity as C# (when you add in all the attributes and things it has that Java doesn't).
 It outputs both parser and lexer together, and lexer has some special 
 features like optimized comment support etc.

ANTLR also outputs parser and lexer, and apparently has a number of other cool features, such as graph rewriting, which is useful for language translations, reformatting, etc. I think one of the Java refactoring tools out there uses it (I wonder what Eclipse uses, or if they rolled their own?) -- dave
Apr 12 2004
next sibling parent Brad Anderson <brad sankaty.dot.com> writes:
 ANTLR also outputs parser and lexer, and apparently has a number of other 
 cool features, such as graph rewriting, which is useful for language 
 translations, reformatting, etc. I think one of the Java refactoring 
 tools out there uses it (I wonder what Eclipse uses, or if they rolled 
 their own?)

Jalopy uses ANTLR. Not sure about Eclipse.
Apr 12 2004
prev sibling parent reply "=?iso-8859-1?Q?Robert_M._M=FCnch?=" <robert.muench robertmuench.de> writes:
On Mon, 12 Apr 2004 13:49:17 +0000 (UTC), Dave Sieber  
<dsieber spamnot.sbcglobal.net> wrote:

 I will have to take a look at COCO, based on your praise for it here! I'd
 been looking around at various parser generators over the years, and  
 ANTLR looks like a potential good choice, although I haven't worked with  
 it yet.

Hi, I have used ANTLR a couple of years back and it was the best parser generator I worked with. If I would have to choose a tool, that's my choice. -- Robert M. Münch Management & IT Freelancer http://www.robertmuench.de
Apr 13 2004
parent Ilya Minkov <minkov cs.tum.edu> writes:
Robert M. M=FCnch schrieb:
 On Mon, 12 Apr 2004 13:49:17 +0000 (UTC), Dave Sieber =20
 <dsieber spamnot.sbcglobal.net> wrote:
=20
 I will have to take a look at COCO, based on your praise for it here! =


 been looking around at various parser generators over the years, and  =


 ANTLR looks like a potential good choice, although I haven't worked=20
 with  it yet.

=20 Hi, I have used ANTLR a couple of years back and it was the best parser=

 generator I worked with. If I would have to choose a tool, that's my =20
 choice.

I would think that ANTLR is a spiritual follower of COCO/R, which should = be about 20 years old by now. I see that ANTLR is an exremely powerful=20 and flexible system... I must look closer at it. However, parser generators are both LL, and converting grammars between=20 them must only be a matter of syntax. -eye P.S. What this leads to me... one could do a sort of D system which=20 could compile console Java programs, such as ANTLR. Not sure how it's=20 worth it though.
Apr 13 2004