www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Any usable parsers for D2 around?

reply Andrej Mitrovic <none none.com> writes:
Hey,

I've been looking for a D2 parser.. there seems to be a few D1 lexers, parsers,
and even some minimal semantic analysis tools but I can't find much of anything
for D2. Perhaps Goldie will be D2 compatible some day soon. :)

There's a "CodeAnalyzer" lexer/parser with some minimal semantics, but it's D1
only and kind of dead for some years. The Poseidon author added some D2 support
for it in his editor, so I have kept an eye on that. But interfacing with D1
code from D2 is kind of complicated, and I need to use it from D2. (I guess I
could get away with writing a DLL with a simple interface for D2 or
something..).

I had a visit on the prowiki site and most of the parser projects there are
either D1-only or dead. So is there anything usable for D2? Specifically I need
these tools for use with an editor/IDE, which is something I'm working on for
fun atm. But working on a parser is probably a ton of work, + the whole thing
screams NIH to me. Luckily enough I don't have to reinvent a GUI and an editing
control, there's DFL and Scintilla which are pretty awesome for my needs. 

Maybe I should take a look at how Descent/DDT and VisualD do their parsing.. :)
Oct 14 2010
next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Fri, 15 Oct 2010 02:47:28 +0400, Andrej Mitrovic <none none.com> wrote:


 Hey,

 I've been looking for a D2 parser.. there seems to be a few D1 lexers,  
 parsers, and even some minimal semantic analysis tools but I can't find  
 much of anything for D2. Perhaps Goldie will be D2 compatible some day  
 soon. :)

 There's a "CodeAnalyzer" lexer/parser with some minimal semantics, but  
 it's D1 only and kind of dead for some years. The Poseidon author added  
 some D2 support for it in his editor, so I have kept an eye on that. But  
 interfacing with D1 code from D2 is kind of complicated, and I need to  
 use it from D2. (I guess I could get away with writing a DLL with a  
 simple interface for D2 or something..).

 I had a visit on the prowiki site and most of the parser projects there  
 are either D1-only or dead. So is there anything usable for D2?  
 Specifically I need these tools for use with an editor/IDE, which is  
 something I'm working on for fun atm. But working on a parser is  
 probably a ton of work, + the whole thing screams NIH to me. Luckily  
 enough I don't have to reinvent a GUI and an editing control, there's  
 DFL and Scintilla which are pretty awesome for my needs.

 Maybe I should take a look at how Descent/DDT and VisualD do their  
 parsing.. :)


You can try using ddmd (dsource.org/projects/ddmd).
Oct 14 2010
prev sibling next sibling parent reply div0 <div0 sourceforge.net> writes:
On 14/10/2010 23:47, Andrej Mitrovic wrote:

 Hey,

 I've been looking for a D2 parser.. there seems to be a few D1 lexers,
parsers, and even some minimal semantic analysis tools but I can't find much of
anything for D2. Perhaps Goldie will be D2 compatible some day soon. :)

 There's a "CodeAnalyzer" lexer/parser with some minimal semantics, but it's D1
only and kind of dead for some years. The Poseidon author added some D2 support
for it in his editor, so I have kept an eye on that. But interfacing with D1
code from D2 is kind of complicated, and I need to use it from D2. (I guess I
could get away with writing a DLL with a simple interface for D2 or
something..).

 I had a visit on the prowiki site and most of the parser projects there are
either D1-only or dead. So is there anything usable for D2? Specifically I need
these tools for use with an editor/IDE, which is something I'm working on for
fun atm. But working on a parser is probably a ton of work, + the whole thing
screams NIH to me. Luckily enough I don't have to reinvent a GUI and an editing
control, there's DFL and Scintilla which are pretty awesome for my needs.

 Maybe I should take a look at how Descent/DDT and VisualD do their parsing.. :)


If you just need a d2 parser library, then my port of boost::spirit 
might fill you need:

http://www.sstk.co.uk/spiritd.php

(I pretty sure the bug that stopped it working got fixed in the last 
release, but I've not tested it yet.)

Though spirit generates LL parsers which aren't really suited for 
parsing human generated input; as getting accurate error location 
reporting is tricky, but it's handy for small inline parsers and other 
simple grammars.

If you are after a d2 parser written in d2, then I dunno.
You could write that using spirit in theory but it would be a job and a 
half.

-- 
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk
Oct 15 2010
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Thanks guys. I'll have a look at those later.

On 10/15/10, div0 <div0 sourceforge.net> wrote:

 On 14/10/2010 23:47, Andrej Mitrovic wrote:

 Hey,

 I've been looking for a D2 parser.. there seems to be a few D1 lexers,
 parsers, and even some minimal semantic analysis tools but I can't find
 much of anything for D2. Perhaps Goldie will be D2 compatible some day
 soon. :)

 There's a "CodeAnalyzer" lexer/parser with some minimal semantics, but
 it's D1 only and kind of dead for some years. The Poseidon author added
 some D2 support for it in his editor, so I have kept an eye on that. But
 interfacing with D1 code from D2 is kind of complicated, and I need to use
 it from D2. (I guess I could get away with writing a DLL with a simple
 interface for D2 or something..).

 I had a visit on the prowiki site and most of the parser projects there
 are either D1-only or dead. So is there anything usable for D2?
 Specifically I need these tools for use with an editor/IDE, which is
 something I'm working on for fun atm. But working on a parser is probably
 a ton of work, + the whole thing screams NIH to me. Luckily enough I don't
 have to reinvent a GUI and an editing control, there's DFL and Scintilla
 which are pretty awesome for my needs.

 Maybe I should take a look at how Descent/DDT and VisualD do their
 parsing.. :)


 If you just need a d2 parser library, then my port of boost::spirit
 might fill you need:

 http://www.sstk.co.uk/spiritd.php

 (I pretty sure the bug that stopped it working got fixed in the last
 release, but I've not tested it yet.)

 Though spirit generates LL parsers which aren't really suited for
 parsing human generated input; as getting accurate error location
 reporting is tricky, but it's handy for small inline parsers and other
 simple grammars.

 If you are after a d2 parser written in d2, then I dunno.
 You could write that using spirit in theory but it would be a job and a
 half.

 --
 My enormous talent is exceeded only by my outrageous laziness.
 http://www.ssTk.co.uk
Oct 15 2010
prev sibling next sibling parent Lutger <lutger.blijdestijn gmail.com> writes:
Andrej Mitrovic wrote:


 Hey,
 
 I've been looking for a D2 parser.. there seems to be a few D1 lexers,
 parsers, and even some minimal semantic analysis tools but I can't find
 much of anything for D2. Perhaps Goldie will be D2 compatible some day
 soon. :)
 
 There's a "CodeAnalyzer" lexer/parser with some minimal semantics, but
 it's D1 only and kind of dead for some years. The Poseidon author added
 some D2 support for it in his editor, so I have kept an eye on that. But
 interfacing with D1 code from D2 is kind of complicated, and I need to
 use it from D2. (I guess I could get away with writing a DLL with a
 simple interface for D2 or something..).
 
 I had a visit on the prowiki site and most of the parser projects there
 are either D1-only or dead. So is there anything usable for D2?
 Specifically I need these tools for use with an editor/IDE, which is
 something I'm working on for fun atm. But working on a parser is probably
 a ton of work, + the whole thing screams NIH to me. Luckily enough I
 don't have to reinvent a GUI and an editing control, there's DFL and
 Scintilla which are pretty awesome for my needs.
 
 Maybe I should take a look at how Descent/DDT and VisualD do their
 parsing.. :)


You can try working with ddmd on dsource. It is a port from the dmd 
frontend, currently at 2.039.
Oct 15 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, October 14, 2010 15:47:28 Andrej Mitrovic wrote:

 Hey,
 
 I've been looking for a D2 parser.. there seems to be a few D1 lexers,
 parsers, and even some minimal semantic analysis tools but I can't find
 much of anything for D2. Perhaps Goldie will be D2 compatible some day
 soon. :)
 
 There's a "CodeAnalyzer" lexer/parser with some minimal semantics, but it's
 D1 only and kind of dead for some years. The Poseidon author added some D2
 support for it in his editor, so I have kept an eye on that. But
 interfacing with D1 code from D2 is kind of complicated, and I need to use
 it from D2. (I guess I could get away with writing a DLL with a simple
 interface for D2 or something..).
 
 I had a visit on the prowiki site and most of the parser projects there are
 either D1-only or dead. So is there anything usable for D2? Specifically I
 need these tools for use with an editor/IDE, which is something I'm
 working on for fun atm. But working on a parser is probably a ton of work,
 + the whole thing screams NIH to me. Luckily enough I don't have to
 reinvent a GUI and an editing control, there's DFL and Scintilla which are
 pretty awesome for my needs.
 
 Maybe I should take a look at how Descent/DDT and VisualD do their
 parsing.. :)


It has periodically been suggested that a parser for D be put in Phobos, but 
that obviously hasn't been done yet. There is obviously some level of need for 
it though. And if an appropriate parsere _were_ put into Phobos, then at least 
it would be standard.

- Jonathan M Davis
Oct 15 2010
prev sibling next sibling parent Trass3r <un known.com> writes:
 I've been looking for a D2 parser..


http://code.google.com/p/dil/
Oct 16 2010
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Andrej Mitrovic" <none none.com> wrote in message 
news:i981a0$i6a$1 digitalmars.com...

 Hey,

 I've been looking for a D2 parser.. there seems to be a few D1 lexers, 
 parsers, and even some minimal semantic analysis tools but I can't find 
 much of anything for D2. Perhaps Goldie will be D2 compatible some day 
 soon. :)


Any post about parsers always seems to lead to confusion: Do you want to 
parse D2 code or do you want to parse some other language *using* D2?

If the former, then, yea, like others said, I'd just use DDMD (In fact, I'm 
already doing so on a pre-alpha project: www.dsource.org/projects/dax ). 
DDMD already has all (or at least most) of the semantic processing and 
everything ready to go, and that's the hard part; parsing is comparatively 
easy.

If the latter (parsing some other language and just using D2 to parse it), 
then if you don't mind dealing with trunk, Goldie has already been switched 
over to D2/Phobos. It just isn't quite cleaned up enough for an official 
release yet (the documentation hasn't been updated, the 
documentation-generating tool is still in the middle of being ported because 
it's waiting on a fix that will be in 2.050, and there's another small thing 
or two I wanted to get in). I can help you out with any troubles using the 
Goldie trunk if you want.

Sorry I didn't see this post earlier.
Oct 26 2010
parent "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:ia8d66$1f7g$1 digitalmars.com...

 "Andrej Mitrovic" <none none.com> wrote in message 
 news:i981a0$i6a$1 digitalmars.com...

 Hey,

 I've been looking for a D2 parser.. there seems to be a few D1 lexers, 
 parsers, and even some minimal semantic analysis tools but I can't find 
 much of anything for D2. Perhaps Goldie will be D2 compatible some day 
 soon. :)


 If the latter (parsing some other language and just using D2 to parse it), 
 then if you don't mind dealing with trunk, Goldie has already been 
 switched over to D2/Phobos. It just isn't quite cleaned up enough for an 
 official release yet (the documentation hasn't been updated, the 
 documentation-generating tool is still in the middle of being ported 
 because it's waiting on a fix that will be in 2.050, and there's another 
 small thing or two I wanted to get in). I can help you out with any 
 troubles using the Goldie trunk if you want.


Although, since you said want to use it for an IDE, Goldie might not be up 
to it yet: AIUI, code editor controls usually do a "partial re-parsing" of 
one-line-at-a-time. Goldie's strictly "parse the whole thing" at the moment.
Oct 26 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 14/10/2010 23:47, Andrej Mitrovic wrote:

 Maybe I should take a look at how Descent/DDT and VisualD do their parsing.. :)


With Descent/DDT, you won't find much love there. Both use the Descent 
parser, a Java port of DMD, which is out of date and no longer updated. 
Even if it was, you'd still be better with DMD itself, or DDMD. (unless 
you wanted a Java based parser)

I so wished there was a ANTLR based parser for D, or at least an ANTLR 
grammar.


-- 
Bruno Medeiros - Software Engineer
Oct 28 2010
parent Justin C Calvarese <jccalvarese gmail.com> writes:
== Quote from Bruno Medeiros (brunodomedeiros+spam com.gmail)'s article

 I so wished there was a ANTLR based parser for D, or at least an ANTLR
 grammar.


I remember seeing some talk in the D newsgroups over the years about the desire
for an ANTLR file
for D. After a quick search of Wiki4D, I found this:
http://www.prowiki.org/wiki4d/wiki.cgi?GrammarParsers#ANTLR-D

That lead me to the ANLTR-D project at dsource.org.

This was the most useful-looking file that I found in SVN:
http://www.dsource.org/projects/antlrd/browser/toys/v3d/parsed.g:

/** A parser for D v1.
  * It's a little loose in parsing declarations, as correctly discerning
  * a function header is complicated, and it's a bit smarter than DMD in a few
  * other places. It can pick out some expression/declaration ambiguities and
  * mark them as such, and it allows mixins to begin expressions.
  * Ambiguities between types and expressions are not caught, and always default
  * to types.
  */

(I don't know have any idea whether it's too out-of-date to be useful or
whether something else
might be wrong with it.)

jcc7
Oct 28 2010