digitalmars.D.learn - Help with porting grammar from PEGjs to D for dustjs project!
- Uranuz (20/20) Aug 03 2014 I want to try to implement web template engine dustjs:
- Philippe Sigaud via Digitalmars-d-learn (20/27) Aug 03 2014 Is this the grammar?
- Uranuz (17/17) Aug 03 2014 I am real noob about grammar description languages so I need some
- Philippe Sigaud via Digitalmars-d-learn (31/46) Aug 03 2014 Yes, that's it. Or at least that's the way Pegged does it. I'm not
- Uranuz (2/8) Aug 05 2014 Yes. That's a point. Thanks for all the explanations. I'll try to
- Uranuz (3/12) Aug 05 2014 Is there multiline comments available inside PEGGED template?
- Philippe Sigaud via Digitalmars-d-learn (2/4) Aug 05 2014 No, no multiline comment. That's based on the original PEG grammar,
- Uranuz (21/21) Aug 06 2014 What I was thinking about is possibility to change ParseTree
- Philippe Sigaud via Digitalmars-d-learn (24/31) Aug 06 2014 That's already the case, look at
I want to try to implement web template engine dustjs: http://akdubya.github.io/dustjs/ At the first step implementing parser for it's grammar is needed. As far as code for parsing grammar was generated via PEGjs grammar generator the resulting code is enough long (about 4200 lines of code). I though that it is productive way to make porting manually. It's long and something could change in the source JavaScript code. And it will be not so easy to maintain. So I remembered about PEGGED project for D. As I see formats for writing grammar differs for these two systems and I have no experience with using grammar generators. So I need some help with rewriting grammar from PEGjs into PEGGED. Also I don't understand in PEGGED (I have not tried to use it yet) how to generate some logic from AST. Where should I describe it or should I walk around all nodes for somehow and generate code for them. Goal of this is to use dust template system as template engine at server side. I also considered handlebars.js, but I can't evaluate what is more suitable for my purposes. The choice is just very subjective.
Aug 03 2014
Uranuz:http://akdubya.github.io/dustjs/So I need some help with rewriting grammar from PEGjs into PEGGED.Is this the grammar? https://github.com/akdubya/dustjs/blob/master/src/dust.pegjs If so, then I think I can provide some help. But I don't get what output you want (see below).Also I don't understand in PEGGED (I have not tried to use it yet) how to generate some logic from AST. Where should I describe it or should I walk around all nodes for somehow and generate code for them.You can put semantic actions in the grammar (code between curly brackets). dust.pegjs seems to have that in their grammar definition also (all these { return something } blocks) Or you can walk the parse tree afterwards. See the Pegged tutorial here: https://github.com/PhilippeSigaud/Pegged/wiki/Pegged-Tutorial More particularly: https://github.com/PhilippeSigaud/Pegged/wiki/Using-the-Parse-Tree The example explains (I hope) how to use a wiki grammar to parse wiki text and output LaTeX code.Goal of this is to use dust template system as template engine at server side.More concretely, what's the goal? A template as input and... what should be output? If I understand correctly, dustjs produces Javascript code. Is that what you want? Or do you want D code? Also, did you have a look at vide.d and its Diet templates?
Aug 03 2014
I am real noob about grammar description languages so I need some explanation about it. As far as I understand expressions in curly bracers are used to modify syntax tree just in process of parsing instead of modifying it after? How I could use PEGGED to map some code to these parsed expressions to generate code that will perform operations defined by this grammar? Should I walk around all the syntax tree and just append code to some string and mix it in then or are there some features for code generation? Something that I was thinking about is comparision of resulting syntax tree to check if it was correctly implemented. It would be great if different gramar parsers will output result in common format (JSON or XML for example) and it will be possiple to compare them for equality. But different parsers have different internal format of tree so maybe creating some transformation is possible. With this feature it could be possible to say if parser is working correctly.
Aug 03 2014
On Mon, Aug 4, 2014 at 7:13 AM, Uranuz via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:I am real noob about grammar description languages so I need some explanation about it. As far as I understand expressions in curly bracers are used to modify syntax tree just in process of parsing instead of modifying it after?Yes, that's it. Or at least that's the way Pegged does it. I'm not such a specialist myself, I just dabbled in it to write Pegged. As I understand it, many parsers do not produce a parse tree, they only 'return' what their embedded action tell them to. Personally, I have a slight preference to using the parse tree once it's complete. a) Because that way I have access to all the information (parent nodes, sibling nodes, even far way) whereas when doing it with an action means you only have the local, current node context and b) because it decouples steps that are in my mind separated anyway: parsing, then producing a value out of the parse tree. A bit like ranges in D, and Walter and his components programming speech.How I could use PEGGED to map some code to these parsed expressions to generate code that will perform operations defined by this grammar? Should I walk around all the syntax tree and just append code to some string and mix it in then or are there some features for code generation?You can insert semantic actions inside the grammar definition, as is done by Dustjs, or you can have a function walking the tree afterwards. For D, since Pegged works at compile time, you can have your parse tree at compile time. Use the walking function to generate D code (a string) and mix it in. enum parseTree = Grammar(input); // CT parsing string codeMaker(ParseTree pt) { ... } // recursive walker mixin(codeMaker(parseTree)); See https://github.com/PhilippeSigaud/Pegged/wiki/Using-the-Parse-Tree If what you need is generating Javascript code, no need to do that at compile-time: you can assemble the JS code as a string at runtime and then write it to a file somewhere, I guess.Something that I was thinking about is comparision of resulting syntax tree to check if it was correctly implemented. It would be great if different gramar parsers will output result in common format (JSON or XML for example) and it will be possiple to compare them for equality. But different parsers have different internal format of tree so maybe creating some transformation is possible. With this feature it could be possible to say if parser is working correctly.Different formats and also different languages. I don't see how you can compare a parse tree that's a D object and another tree made by dustjs: you never see the AST produced by dust, you only see the resulting JS code.
Aug 03 2014
Different formats and also different languages. I don't see how you can compare a parse tree that's a D object and another tree made by dustjs: you never see the AST produced by dust, you only see the resulting JS code.Yes. That's a point. Thanks for all the explanations. I'll try to make something useful of it.
Aug 05 2014
On Tuesday, 5 August 2014 at 08:13:25 UTC, Uranuz wrote:Is there multiline comments available inside PEGGED template?Different formats and also different languages. I don't see how you can compare a parse tree that's a D object and another tree made by dustjs: you never see the AST produced by dust, you only see the resulting JS code.Yes. That's a point. Thanks for all the explanations. I'll try to make something useful of it.
Aug 05 2014
Is there multiline comments available inside PEGGED template?No, no multiline comment. That's based on the original PEG grammar,
Aug 05 2014
What I was thinking about is possibility to change ParseTree struct with user-defined version of it. And I was thinking about setting tree type as template parameter to grammar: grammar!(MyParseTree)(" Arithmetic: ... "); Or somethink like this. I think changing source code of library in order to change tree type is not good and should be set as parameter. If it's already implemented please let me knoe because I couldn't find it. And also some minimal interface is needed to be described in documentation for ParseTree (may be ability to set it as class is good in order to have polymorthic nodes with different methods and properties). Of course I can transform PEGGED syntactic tree into another form of tree specified by usage domain. But if it doesn't significantly differs from PEGGED tree (for example node have a pair of additional properties) it only causes into additional memory consumption and CPU to transform tree. But if domain specific tree differs a lot of course we need some useful way to transform trees.
Aug 06 2014
On Wed, Aug 6, 2014 at 9:09 AM, Uranuz via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:What I was thinking about is possibility to change ParseTree struct with user-defined version of it. And I was thinking about setting tree type as template parameter to grammar: grammar!(MyParseTree)(" Arithmetic: ... ");That's already the case, look at https://github.com/PhilippeSigaud/Pegged/blob/master/pegged/parser.d on line 134, for example. struct GenericPegged(TParseTree) { ... } And then, the library alias a standard version (line 1531): alias GenericPegged!(ParseTree).Pegged Pegged; Which means the customer-facing Pegged parser is in fact a generic parser specialized on ParseTree. But you can substitute your own parse tree. That's of course the same for all parsers: GenericXXX(TParseTree) {...} and then alias XXX = GenericXXX!(ParseTree); But you're right, it's not really documented and I don't have a template checking whether TParseTree respect some static inferface. This parameterization was asked by someone on github, but I don't think they used it finally. ParseTree is described here: https://github.com/PhilippeSigaud/Pegged/wiki/Parse-Trees Maybe we can continue this thread by private mail? I'm not sure people on the D list are that interested by the internals of a library.
Aug 06 2014