www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ANTLR grammar

reply Andy Friesen <andy ikagames.com> writes:
ANTLR is ANother Tool for Language Recognition.  It's the same idea as 
Lex and YACC, but has (I think) better syntax, and directly supports 
generating re-entrant parsers in C++, C# or Java. (Python support is on 
the way, as I understand it.  Anybody want to take a crack at D 
support?)  You can get it at <http://antlr.org>

I haven't run this through the complete gauntlet just yet, but 
preliminary tests suggest that it handles a respectable subset of D, if 
not the whole thing. :)

<http://andy.tadan.us/d/d.g>

There isn't much else going on in this parser just yet, it's little more 
than a syntax checker, but that also means that it's easier to gut it if 
you want to use the grammar for something else. :)

  -- andy
Sep 01 2004
next sibling parent resistor AT mac DOT com <resistor_member pathlink.com> writes:
In article <ch3st2$1scq$1 digitaldaemon.com>, Andy Friesen says...
ANTLR is ANother Tool for Language Recognition.  It's the same idea as 
Lex and YACC, but has (I think) better syntax, and directly supports 
generating re-entrant parsers in C++, C# or Java. (Python support is on 
the way, as I understand it.  Anybody want to take a crack at D 
support?)  You can get it at <http://antlr.org>

I haven't run this through the complete gauntlet just yet, but 
preliminary tests suggest that it handles a respectable subset of D, if 
not the whole thing. :)

<http://andy.tadan.us/d/d.g>

There isn't much else going on in this parser just yet, it's little more 
than a syntax checker, but that also means that it's easier to gut it if 
you want to use the grammar for something else. :)

  -- andy

Sep 01 2004
prev sibling parent reply resistor AT mac DOT com <resistor_member pathlink.com> writes:
Awesome...

Except, I can't get it to compile.  Attempting to compile the generated
DParser.cpp and DLexer.cpp files 
gives me:

In file included from DParser.cpp:2:
DParser.hpp:84: warning: declaration does not declare anything
DParser.hpp:84: error: parse error before `typeof'
DParser.cpp:1927: error: parse error before `typeof'
DParser.cpp:1931: error: `LPAREN' was not declared in this scope
DParser.cpp:1931: warning: ISO C++ forbids declaration of `match' with no type
DParser.cpp:1932: warning: ISO C++ forbids declaration of `expression' with no 
type
DParser.cpp:1933: error: `RPAREN' was not declared in this scope
DParser.cpp:1933: warning: ISO C++ forbids declaration of `match' with no type
DParser.cpp:1933: error: redefinition of `int match'
DParser.cpp:1931: error: `int match' previously defined here
DParser.cpp:1934: error: parse error before `}' token
DParser.cpp:1938: warning: ISO C++ forbids declaration of `consume' with no 
type
DParser.cpp:1939: error: `_tokenSet_35' was not declared in this scope
DParser.cpp:1939: warning: ISO C++ forbids declaration of `consumeUntil' with 
no type
DParser.cpp:1940: error: parse error before `}' token
DParser.cpp: In member function `void DParser::basicType()':
DParser.cpp:2133: error: parse error before `)' token

I'm using ANTLR 2.7.4, so I don't know what I could be doing wrong.  Any ideas?

-Owen

In article <ch3st2$1scq$1 digitaldaemon.com>, Andy Friesen says...
ANTLR is ANother Tool for Language Recognition.  It's the same idea as 
Lex and YACC, but has (I think) better syntax, and directly supports 
generating re-entrant parsers in C++, C# or Java. (Python support is on 
the way, as I understand it.  Anybody want to take a crack at D 
support?)  You can get it at <http://antlr.org>

I haven't run this through the complete gauntlet just yet, but 
preliminary tests suggest that it handles a respectable subset of D, if 
not the whole thing. :)

<http://andy.tadan.us/d/d.g>

There isn't much else going on in this parser just yet, it's little more 
than a syntax checker, but that also means that it's easier to gut it if 
you want to use the grammar for something else. :)

  -- andy

Sep 01 2004
parent reply Andy Friesen <andy ikagames.com> writes:
resistor AT mac DOT com wrote:
 Awesome...
 
 Except, I can't get it to compile.  Attempting to compile the generated
 DParser.cpp and DLexer.cpp files 
 gives me:
 
 In file included from DParser.cpp:2:
 DParser.hpp:84: warning: declaration does not declare anything
 DParser.hpp:84: error: parse error before `typeof'
 DParser.cpp:1927: error: parse error before `typeof'
 DParser.cpp:1931: error: `LPAREN' was not declared in this scope
 DParser.cpp:1931: warning: ISO C++ forbids declaration of `match' with no type
 DParser.cpp:1932: warning: ISO C++ forbids declaration of `expression' with no 
 type
 DParser.cpp:1933: error: `RPAREN' was not declared in this scope
 DParser.cpp:1933: warning: ISO C++ forbids declaration of `match' with no type
 DParser.cpp:1933: error: redefinition of `int match'
 DParser.cpp:1931: error: `int match' previously defined here
 DParser.cpp:1934: error: parse error before `}' token
 DParser.cpp:1938: warning: ISO C++ forbids declaration of `consume' with no 
 type
 DParser.cpp:1939: error: `_tokenSet_35' was not declared in this scope
 DParser.cpp:1939: warning: ISO C++ forbids declaration of `consumeUntil' with 
 no type
 DParser.cpp:1940: error: parse error before `}' token
 DParser.cpp: In member function `void DParser::basicType()':
 DParser.cpp:2133: error: parse error before `)' token
 
 I'm using ANTLR 2.7.4, so I don't know what I could be doing wrong.  Any ideas?

Curious. It looks as though it doesn't like the use of 'typeof' as an identifier. What compiler are you using? -- andy
Sep 01 2004
parent reply resistor AT mac DOT com <resistor_member pathlink.com> writes:
resistor$ gcc -v
Reading specs from /usr/libexec/gcc/darwin/ppc/3.3/specs
Thread model: posix
gcc version 3.3 20030304 (Apple Computer, Inc. build 1666)

-Owen

P.S.  I've never had trouble building ANTLR grammars before.

In article <ch6b2g$bo$1 digitaldaemon.com>, Andy Friesen says...
resistor AT mac DOT com wrote:
 Awesome...
 
 Except, I can't get it to compile.  Attempting to compile the generated
 DParser.cpp and DLexer.cpp files 
 gives me:
 
 In file included from DParser.cpp:2:
 DParser.hpp:84: warning: declaration does not declare anything
 DParser.hpp:84: error: parse error before `typeof'
 DParser.cpp:1927: error: parse error before `typeof'
 DParser.cpp:1931: error: `LPAREN' was not declared in this scope
 DParser.cpp:1931: warning: ISO C++ forbids declaration of `match' with no type
 DParser.cpp:1932: warning: ISO C++ forbids declaration of `expression' with no 
 type
 DParser.cpp:1933: error: `RPAREN' was not declared in this scope
 DParser.cpp:1933: warning: ISO C++ forbids declaration of `match' with no type
 DParser.cpp:1933: error: redefinition of `int match'
 DParser.cpp:1931: error: `int match' previously defined here
 DParser.cpp:1934: error: parse error before `}' token
 DParser.cpp:1938: warning: ISO C++ forbids declaration of `consume' with no 
 type
 DParser.cpp:1939: error: `_tokenSet_35' was not declared in this scope
 DParser.cpp:1939: warning: ISO C++ forbids declaration of `consumeUntil' with 
 no type
 DParser.cpp:1940: error: parse error before `}' token
 DParser.cpp: In member function `void DParser::basicType()':
 DParser.cpp:2133: error: parse error before `)' token
 
 I'm using ANTLR 2.7.4, so I don't know what I could be doing wrong.  Any ideas?

Curious. It looks as though it doesn't like the use of 'typeof' as an identifier. What compiler are you using? -- andy

Sep 02 2004
parent reply Andy Friesen <andy ikagames.com> writes:
resistor AT mac DOT com wrote:
 resistor$ gcc -v
 Reading specs from /usr/libexec/gcc/darwin/ppc/3.3/specs
 Thread model: posix
 gcc version 3.3 20030304 (Apple Computer, Inc. build 1666)
 
 -Owen
 
 P.S.  I've never had trouble building ANTLR grammars before.

It looks like g++ defines 'typeof' as a keyword. You can fix it by changing the name of the typeof rule to typeofType or some such. There may also be a compile flag to disable that particular extension. -- andy
Sep 02 2004
parent resistor AT mac DOT com <resistor_member pathlink.com> writes:
OK.  Fixed that by running G++ with the -ansi flag.  For future reference,
however, it might help to 
change that identifier, as G++ is a pretty common compiler.

Anyways, I got it compiled and hooked up with a simple driver main.cpp.  I ran
this against the wc.d 
test file, with these results:

wc.d:9:13: unexpected token: "   lines   words   bytes file\n"
wc.d:9:46: expecting SEMI, found ')'
wc.d:16:37: unexpected token: .
wc.d:16:46: expecting SEMI, found ')'
wc.d:24:21: unexpected token: !
wc.d:34:17: unexpected token: "%8lu%8lu%8lu %.*s\n"
wc.d:34:38: expecting SEMI, found ','
wc.d:34:52: expecting SEMI, found ','
wc.d:34:64: expecting SEMI, found ')'
wc.d:41:17: unexpected token:
"--------------------------------------\n%8lu%8lu%8lu total"
wc.d:41:77: expecting SEMI, found ','
wc.d:42:29: expecting SEMI, found ','
wc.d:42:31: unexpected token: c_total

Looks like it parsed most of the program quite well, with only some problems
with string literals as 
function parameters.  Overall, though, great work!!

-Owen

In article <ch7t3t$n04$1 digitaldaemon.com>, Andy Friesen says...
resistor AT mac DOT com wrote:
 resistor$ gcc -v
 Reading specs from /usr/libexec/gcc/darwin/ppc/3.3/specs
 Thread model: posix
 gcc version 3.3 20030304 (Apple Computer, Inc. build 1666)
 
 -Owen
 
 P.S.  I've never had trouble building ANTLR grammars before.

It looks like g++ defines 'typeof' as a keyword. You can fix it by changing the name of the typeof rule to typeofType or some such. There may also be a compile flag to disable that particular extension. -- andy

Sep 02 2004