www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Parser generator?

reply Wouter Verhelst <wouter grep.be> writes:
Hi folks,

Does someone know of a parser generator for D?

If it doesn't exist, I can write a parser by hand, but having it
generated (at least for my initial permutation) seems like a better
idea...

Thanks,

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
Jul 04 2012
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, July 04, 2012 14:53:02 Wouter Verhelst wrote:

 Hi folks,
 
 Does someone know of a parser generator for D?
 
 If it doesn't exist, I can write a parser by hand, but having it
 generated (at least for my initial permutation) seems like a better
 idea...
 
 Thanks,


https://github.com/PhilippeSigaud/Pegged
https://github.com/PhilippeSigaud/Pegged/wiki

- Jonathan M Davis
Jul 04 2012
parent reply Wouter Verhelst <wouter grep.be> writes:
Jonathan M Davis <jmdavisProg gmx.com> writes:


 On Wednesday, July 04, 2012 14:53:02 Wouter Verhelst wrote:

 Hi folks,
 
 Does someone know of a parser generator for D?
 
 If it doesn't exist, I can write a parser by hand, but having it
 generated (at least for my initial permutation) seems like a better
 idea...
 
 Thanks,


 https://github.com/PhilippeSigaud/Pegged
 https://github.com/PhilippeSigaud/Pegged/wiki


Whoa. That's so perfect, it makes me drool.

Thanks.

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
Jul 04 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, July 04, 2012 15:32:16 Wouter Verhelst wrote:

 Jonathan M Davis <jmdavisProg gmx.com> writes:

 On Wednesday, July 04, 2012 14:53:02 Wouter Verhelst wrote:

 Hi folks,
 
 Does someone know of a parser generator for D?
 
 If it doesn't exist, I can write a parser by hand, but having it
 generated (at least for my initial permutation) seems like a better
 idea...
 
 Thanks,


 
 https://github.com/PhilippeSigaud/Pegged
 https://github.com/PhilippeSigaud/Pegged/wiki


 
 Whoa. That's so perfect, it makes me drool.


Yeah. It's pretty cool. It really shows up D's metaprogramming capabilities. 
The one thing to watch out for though is that such metaprogramming tends to 
eat up a lot of memory when compiling at this point (primarily because the 
compiler doesn't manage memory very well at this point - it's approach is very 
simplistic). So, that may or may not cause you problems. It should be fixed 
eventually, but it does sometimes cause problems with this sort of thing. 
std.regex has similar issues.

- Jonathan M Davis
Jul 04 2012
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07/04/2012 11:41 PM, Jonathan M Davis wrote:

 On Wednesday, July 04, 2012 15:32:16 Wouter Verhelst wrote:

 Jonathan M Davis<jmdavisProg gmx.com>  writes:

 On Wednesday, July 04, 2012 14:53:02 Wouter Verhelst wrote:

 Hi folks,

 Does someone know of a parser generator for D?

 If it doesn't exist, I can write a parser by hand, but having it
 generated (at least for my initial permutation) seems like a better
 idea...

 Thanks,


 https://github.com/PhilippeSigaud/Pegged
 https://github.com/PhilippeSigaud/Pegged/wiki


 Whoa. That's so perfect, it makes me drool.


 Yeah. It's pretty cool. It really shows up D's metaprogramming capabilities.
 The one thing to watch out for though is that such metaprogramming tends to
 eat up a lot of memory when compiling at this point (primarily because the
 compiler doesn't manage memory very well at this point - it's approach is very
 simplistic). So, that may or may not cause you problems. It should be fixed
 eventually, but it does sometimes cause problems with this sort of thing.
 std.regex has similar issues.

 - Jonathan M Davis


https://github.com/PhilippeSigaud/Pegged/wiki/Grammars-as-D-Modules
Jul 04 2012
prev sibling parent reply Wouter Verhelst <wouter grep.be> writes:
Jonathan M Davis <jmdavisProg gmx.com> writes:


 On Wednesday, July 04, 2012 15:32:16 Wouter Verhelst wrote:

 Jonathan M Davis <jmdavisProg gmx.com> writes:

 On Wednesday, July 04, 2012 14:53:02 Wouter Verhelst wrote:

 Hi folks,
 
 Does someone know of a parser generator for D?
 
 If it doesn't exist, I can write a parser by hand, but having it
 generated (at least for my initial permutation) seems like a better
 idea...
 
 Thanks,


 
 https://github.com/PhilippeSigaud/Pegged
 https://github.com/PhilippeSigaud/Pegged/wiki


 
 Whoa. That's so perfect, it makes me drool.


 Yeah. It's pretty cool. It really shows up D's metaprogramming
 capabilities. 


It's a bit hell to debug it, though. But I finally managed to get a
working parser out of it.

Except now I can't use it, for some reason. Code:

void buildGraph(Output o) {
        void parseToGraph(ParseTree p) {
                writeln(p.ruleName);
        }

        parseToGraph(o.parseTree);
}

void parseconfigs() {
        Output o = ENI.parse(readText("/tmp/ifaces_data"));
        buildGraph(o);
}

produces the following error message:

dmd  -gc -w -unittest -I../../Pegged -c -ofmain.o main.d
ipcfg/parser.d(45): Error: struct pegged.peg.Output(TParseTree) if
(isParseTree!(TParseTree)) is used as a type
make: *** [main.o] Error 1

Help?


 The one thing to watch out for though is that such metaprogramming tends to 
 eat up a lot of memory when compiling at this point (primarily because the 
 compiler doesn't manage memory very well at this point - it's approach is very 
 simplistic).


I'll say. My parser does more than just identifiers and comments, but
it's still not that complex. And yet I manage 1GiB+. Whoa.

But that's still acceptable.


 So, that may or may not cause you problems.


Not really.

What does cause problems, though, is that it won't compile with gdc:

gdmd -release  -I../../Pegged -c -ofmain.o main.d
/home/wouter/code/d/Pegged/pegged/grammar.d:128: Error: template
pegged.grammar.PEGGED!(ParseTree).PEGGED.parse(ParseLevel pl =
ParseLevel.parsing) parse(ParseLevel pl = ParseLevel.parsing) matches more than
one template declaration,
/home/wouter/code/d/Pegged/pegged/grammar.d(111):parse(ParseLevel pl =
ParseLevel.parsing) and
/home/wouter/code/d/Pegged/pegged/grammar.d(126):parse(ParseLevel pl =
ParseLevel.parsing)
make: *** [main.o] Error 1

GDC 4.6, though, so I'll wait until Iain uploads 4.7 to Debian; if that
doesn't fix it, I'll file bugs where appropriate.


 It should be fixed 
 eventually, but it does sometimes cause problems with this sort of thing. 
 std.regex has similar issues.


Gotcha.

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
Jul 05 2012
parent reply Philippe Sigaud <philippe.sigaud gmail.com> writes:
 Yeah. It's pretty cool. It really shows up D's metaprogramming
 capabilities.


 It's a bit hell to debug it, though. But I finally managed to get a
 working parser out of it.


I received your suggestion of a full debug mode explaining what rules where
activated and wich did not.


 Except now I can't use it, for some reason. Code:

 void buildGraph(Output o) {
         void parseToGraph(ParseTree p) {
                 writeln(p.ruleName);
         }

         parseToGraph(o.parseTree);
 }

 void parseconfigs() {
         Output o = ENI.parse(readText("/tmp/ifaces_data"));
         buildGraph(o);
 }

 produces the following error message:

 dmd  -gc -w -unittest -I../../Pegged -c -ofmain.o main.d
 ipcfg/parser.d(45): Error: struct pegged.peg.Output(TParseTree) if


(isParseTree!(TParseTree)) is used as a type

 make: *** [main.o] Error 1

 Help?


Roman recently templated the parse tree to allow multiple outputs and we
didn't update the docs, sorry. That makes Pegged output something other
than `Output`. One consequence is hat semantic actions should be templates
to function on different kinds of parse trees.

Try:


 void buildGraph(O)(O o) {
         void parseToGraph(ParseTree p) {
                 writeln(p.ruleName);
         }

         parseToGraph(o.parseTree);
 }

 void parseconfigs() {
         auto o = ENI.parse(readText("/tmp/ifaces_data"));
         buildGraph(o);
 }


What do parseToGraph and buildGraph do?


 What does cause problems, though, is that it won't compile with gdc:

 gdmd -release  -I../../Pegged -c -ofmain.o main.d
 /home/wouter/code/d/Pegged/pegged/grammar.d:128: Error: template


pegged.grammar.PEGGED!(ParseTree).PEGGED.parse(ParseLevel pl =
ParseLevel.parsing) parse(ParseLevel pl = ParseLevel.parsing) matches more
than one template declaration,
/home/wouter/code/d/Pegged/pegged/grammar.d(111):parse(ParseLevel pl =
ParseLevel.parsing) and
/home/wouter/code/d/Pegged/pegged/grammar.d(126):parse(ParseLevel pl =
ParseLevel.parsing)

 make: *** [main.o] Error 1

 GDC 4.6, though, so I'll wait until Iain uploads 4.7 to Debian; if that
 doesn't fix it, I'll file bugs where appropriate.


I find it strange that I don't get that error with DMD.

Philippe
Jul 05 2012
next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Friday, 6 July 2012 at 06:10:28 UTC, Philippe Sigaud wrote:

 Roman recently templated the parse tree to allow multiple 
 outputs and we didn't update the docs, sorry.


That was https://github.com/chadjoan, not me :) But I was amazed 
that my needs have been met so well.
Jul 06 2012
prev sibling next sibling parent "Roman D. Boiko" <rb d-coding.com> writes:
On Friday, 6 July 2012 at 06:10:28 UTC, Philippe Sigaud wrote:

 void buildGraph(Output o) {
         void parseToGraph(ParseTree p) {
                 writeln(p.ruleName);
         }

         parseToGraph(o.parseTree);
 }


 What do parseToGraph and buildGraph do?


buildGraph unwraps parseTree from output, and parseToGraph writes 
a line with rule name of a parse tree. ;)))
Jul 06 2012
prev sibling parent Wouter Verhelst <wouter grep.be> writes:
Philippe Sigaud <philippe.sigaud gmail.com> writes:


 Yeah. It's pretty cool. It really shows up D's metaprogramming
 capabilities.


 It's a bit hell to debug it, though. But I finally managed to get a
 working parser out of it.


 I received your suggestion of a full debug mode explaining what rules where
 activated and wich did not.


Good :-)


 Except now I can't use it, for some reason. Code:

 void buildGraph(Output o) {
         void parseToGraph(ParseTree p) {
                 writeln(p.ruleName);
         }

         parseToGraph(o.parseTree);
 }

 void parseconfigs() {
         Output o = ENI.parse(readText("/tmp/ifaces_data"));
         buildGraph(o);
 }

 produces the following error message:

 dmd  -gc -w -unittest -I../../Pegged -c -ofmain.o main.d
 ipcfg/parser.d(45): Error: struct pegged.peg.Output(TParseTree) if


 (isParseTree!(TParseTree)) is used as a type

 make: *** [main.o] Error 1

 Help?


 Roman recently templated the parse tree to allow multiple outputs and we didn't
 update the docs, sorry. That makes Pegged output something other than `Output`.
 One consequence is hat semantic actions should be templates to function on
 different kinds of parse trees.

 Try:


 void buildGraph(O)(O o) {




Ah yes, that explains. Thanks.


         void parseToGraph(ParseTree p) {
                 writeln(p.ruleName);
         }

         parseToGraph(o.parseTree);
 }

 void parseconfigs() {
         auto o = ENI.parse(readText("/tmp/ifaces_data"));
         buildGraph(o);
 }


 What do parseToGraph and buildGraph do?


Nothing more than that, _currently_. The plan is for them to eventually
build data structures in memory, but I'm not there yet.


 What does cause problems, though, is that it won't compile with gdc:

 gdmd -release  -I../../Pegged -c -ofmain.o main.d
 /home/wouter/code/d/Pegged/pegged/grammar.d:128: Error: template


 pegged.grammar.PEGGED!(ParseTree).PEGGED.parse(ParseLevel pl =
 ParseLevel.parsing) parse(ParseLevel pl = ParseLevel.parsing) matches more than
 one template declaration, /home/wouter/code/d/Pegged/pegged/grammar.d
 (111):parse(ParseLevel pl = ParseLevel.parsing) and /home/wouter/code/d/Pegged/
 pegged/grammar.d(126):parse(ParseLevel pl = ParseLevel.parsing)

 make: *** [main.o] Error 1

 GDC 4.6, though, so I'll wait until Iain uploads 4.7 to Debian; if that
 doesn't fix it, I'll file bugs where appropriate.


 I find it strange that I don't get that error with DMD.


Agreed.

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
Jul 06 2012
prev sibling next sibling parent Mirko Pilger <pilger cymotec.de> writes:
 Does someone know of a parser generator for D?


http://www.complang.org/ragel/
http://www.semitwist.com/goldie/
Jul 04 2012
prev sibling parent Jerome BENOIT <g6299304p rezozer.net> writes:
Apparently Flex/Bison can be used: see last answer in

http://www.digitalmars.com/d/archives/digitalmars/D/31679.html

hth,
Jerome

On 04/07/12 22:53, Wouter Verhelst wrote:

 Hi folks,

 Does someone know of a parser generator for D?

 If it doesn't exist, I can write a parser by hand, but having it
 generated (at least for my initial permutation) seems like a better
 idea...

 Thanks,
Jul 04 2012