www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is the declaration grammar definition of 'Parameter' correct?

reply Bernard Helyer <b.helyer gmail.com> writes:
http://www.digitalmars.com/d/2.0/declaration.html

So, cut down:

Decl
     BasicType Declarators ;

BasicType
     int
     ...

BasicType2
     *
     [] and co
     function Parameters

Parameter
     Declarator
     ...

Declarator
     BasicType2 Identifier DeclaratorSuffixes (the suffixes are [] 
[assignexpr] [type] and a template parameter list)


So given all that, I can't see how this:

int function(int, int) a;

can be parsed with that grammar.

Additionally, Declarator requires identifier, so wouldn't that make this:

int function(* a, [] b) c;

a valid Decl according to that grammar. I think this is seriously 
incorrect, but I would be open to correction! :D
Jun 07 2010
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Yeah, it's wrong. (close reads of parse.c are much more useful than 
reading the spec. heh.) A peek in my grammar and...

Parameter:
    ...
    BasicType Declarator
    BasicType Declarator = AssignExpression
    BasicType Declarator ...
    Type
    Type ...

I probably should have filed bug reports back when I was going through 
the grammar. Oh well.
Jun 07 2010
parent reply Bernard Helyer <b.helyer gmail.com> writes:
On 08/06/10 16:00, Ellery Newcomer wrote:

 Yeah, it's wrong. (close reads of parse.c are much more useful than
 reading the spec. heh.) A peek in my grammar and...

 Parameter:
 ...
 BasicType Declarator
 BasicType Declarator = AssignExpression
 BasicType Declarator ...
 Type
 Type ...

 I probably should have filed bug reports back when I was going through
 the grammar. Oh well.


Hmm. On the same page, Declarator has an identifier in it. Which means I 
still couldn't parse

int function(int, int)

with it, no?
Jun 07 2010
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/07/2010 11:06 PM, Bernard Helyer wrote:

 On 08/06/10 16:00, Ellery Newcomer wrote:

 Yeah, it's wrong. (close reads of parse.c are much more useful than
 reading the spec. heh.) A peek in my grammar and...

 Parameter:
 ...
 BasicType Declarator
 BasicType Declarator = AssignExpression
 BasicType Declarator ...
 Type
 Type ...

 I probably should have filed bug reports back when I was going through
 the grammar. Oh well.


 Hmm. On the same page, Declarator has an identifier in it. Which means I
 still couldn't parse

 int function(int, int)

 with it, no?


Eh?

Parameter |= Type |= BasicType Declarator2 |= int Declarator2 |= int

wait, are you talking about the params inside the function type, or the 
whole thing as a param? I'm pretty sure it works either way.
Jun 07 2010
parent reply Bernard Helyer <b.helyer gmail.com> writes:
On 08/06/10 17:19, Ellery Newcomer wrote:

 On 06/07/2010 11:06 PM, Bernard Helyer wrote:

 On 08/06/10 16:00, Ellery Newcomer wrote:

 Yeah, it's wrong. (close reads of parse.c are much more useful than
 reading the spec. heh.) A peek in my grammar and...

 Parameter:
 ...
 BasicType Declarator
 BasicType Declarator = AssignExpression
 BasicType Declarator ...
 Type
 Type ...

 I probably should have filed bug reports back when I was going through
 the grammar. Oh well.


 Hmm. On the same page, Declarator has an identifier in it. Which means I
 still couldn't parse

 int function(int, int)

 with it, no?


 Eh?

 Parameter |= Type |= BasicType Declarator2 |= int Declarator2 |= int

 wait, are you talking about the params inside the function type, or the
 whole thing as a param? I'm pretty sure it works either way.


Parameter doesn't resolve to Type, not that I can see...
Jun 07 2010
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/08/2010 12:30 AM, Bernard Helyer wrote:

 On 08/06/10 17:19, Ellery Newcomer wrote:

 On 06/07/2010 11:06 PM, Bernard Helyer wrote:

 On 08/06/10 16:00, Ellery Newcomer wrote:

 Yeah, it's wrong. (close reads of parse.c are much more useful than
 reading the spec. heh.) A peek in my grammar and...

 Parameter:
 ...
 BasicType Declarator
 BasicType Declarator = AssignExpression
 BasicType Declarator ...
 Type
 Type ...

 I probably should have filed bug reports back when I was going through
 the grammar. Oh well.


 Hmm. On the same page, Declarator has an identifier in it. Which means I
 still couldn't parse

 int function(int, int)

 with it, no?


 Eh?

 Parameter |= Type |= BasicType Declarator2 |= int Declarator2 |= int

 wait, are you talking about the params inside the function type, or the
 whole thing as a param? I'm pretty sure it works either way.


 Parameter doesn't resolve to Type, not that I can see...


According to whom?
Jun 08 2010
prev sibling parent reply BCS <none anon.com> writes:
Right now we have two semi-official definitions of the D grammar; the docs 
(that are wrong) and the parser source (that is effectively unreadable by 
most people). I would like to propose a solution to this problem: eliminate 
one of them and derive it from the other.

I know this will be hard to do but it can be done incrementally with each 
step making improvements over the last.

The first thing to do is put all of the description of the grammar in the 
docs into one place. If the literal text of each production is replace with 
a macro reference then the definitions of these macros can be put into a 
single file and expanded everywhere.
The incremental improvement here is that having the grammar in one place 
by it's self will make it easier to check.

From here there are several things that can be done:

- copy the grammar rule macros into the parser source where they will stay 
more up to date.
- write a tool to generate the rules from the parser source.
- write a tool to generate the parser source from the rules.

I'd be willing to do that first step if I got any (semi)official indication 
that it would be used.

-- 
... <IXOYE><
Jun 13 2010
next sibling parent reply Michal Minich <michal.minich gmail.com> writes:
On Sun, 13 Jun 2010 20:15:46 +0000, BCS wrote:


 The first thing to do is put all of the description of the grammar in
 the docs into one place.


If we are at it, it would be good to enhance visual representation of 
docs by generate syntax diagram, similarly as on http://www.json.org/ it 
not only helps to learn and understand grammar, but also lowers the 
barrier to just looking at it. When I first saw this kind of diagrams on 
JSON site, it made good impression on me.

Tool used for this is.
http://dotnet.jku.at/applications/Visualizer/

Another important point is - having it visually really help find and 
enhance specs.

This tools accepts EBNF syntax that is specified on the page. D syntax is 
specified in BNF, and uses different syntax, but (E)BNF is so simple, it 
should be possible to adjust different syntaxes just using regex (I think/
hope).

If there is interest in such diagrams, and there is one file syntax file, 

generate these images. (It would be also a good idea to look at first if 
BNF diagrams look good, and if EBNF isn't necessary.)
Jun 13 2010
parent BCS <none anon.com> writes:
Hello Michal,


 On Sun, 13 Jun 2010 20:15:46 +0000, BCS wrote:
 

 The first thing to do is put all of the description of the grammar in
 the docs into one place.
 


 If we are at it, it would be good to enhance visual representation of
 docs by generate syntax diagram, similarly as on http://www.json.org/
 it not only helps to learn and understand grammar, but also lowers the
 barrier to just looking at it. When I first saw this kind of diagrams
 on JSON site, it made good impression on me.
 


[...]

Nice idea. (But I'd put them in a different set of cross linked pages.)


 If there is interest in such diagrams, and there is one file syntax

 to batch- generate these images. (It would be also a good idea to look
 at first if BNF diagrams look good, and if EBNF isn't necessary.)


You might not need to change the tool. If the grammar macros are done with 
some thought, you might be able to make them render it to whatever that tool 
already take as input.


 


-- 
... <IXOYE><
Jun 13 2010
prev sibling next sibling parent reply BCS <none anon.com> writes:
Hello BCS,


 I'd be willing to do that first step if I got any (semi)official
 indication that it would be used.


does anyone know where the SVN repo with the source for the docs is?

-- 
... <IXOYE><
Jun 13 2010
parent reply Brad Roberts <braddr puremagic.com> writes:
On 6/13/2010 5:15 PM, BCS wrote:

 Hello BCS,
 

 I'd be willing to do that first step if I got any (semi)official
 indication that it would be used.


 
 does anyone know where the SVN repo with the source for the docs is?
 


The standard phobos repo at dsource in the docsrc directory:

    http://www.dsource.org/projects/phobos/browser/trunk/docsrc

Arguably part of that could (maybe should) be split to the dmd repo now that it
exists, but... shrug.
Jun 13 2010
parent BCS <none anon.com> writes:
Hello Brad,


 On 6/13/2010 5:15 PM, BCS wrote:
 

 Hello BCS,
 

 I'd be willing to do that first step if I got any (semi)official
 indication that it would be used.
 


 does anyone know where the SVN repo with the source for the docs is?
 


 The standard phobos repo at dsource in the docsrc directory:
 
 http://www.dsource.org/projects/phobos/browser/trunk/docsrc
 


Thanks.

-- 
... <IXOYE><
Jun 13 2010
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 06/13/2010 03:15 PM, BCS wrote:

 Right now we have two semi-official definitions of the D grammar; the
 docs (that are wrong) and the parser source (that is effectively
 unreadable by most people). I would like to propose a solution to this
 problem: eliminate one of them and derive it from the other.



In the past, Walter has expressed a dislike of parser generators. I 
doubt you can get him to change his mind on that one, especially since 
most parser generators won't be able to handle D anyways, and I doubt 
any of them will be able to handle D efficiently (what are they called - 
GLR parser generators? Don't know about these).

As far as analysing parse.c goes, how much effort is it going to take to 
figure out what

while(1)
   switch(token.value){
     case x:
       parseThis();
       continue;
     case y:
       parseThat();
       break;
   }
   break;
}

should look like in a grammar?

Counterproposal: why don't we just fix the grammar?


 I know this will be hard to do but it can be done incrementally with
 each step making improvements over the last.

 The first thing to do is put all of the description of the grammar in
 the docs into one place. If the literal text of each production is
 replace with a macro reference then the definitions of these macros can
 be put into a single file and expanded everywhere.
 The incremental improvement here is that having the grammar in one place
 by it's self will make it easier to check.


I would very much like to see the entire grammar in one place. I can't 
tell you how obnoxious it is to search through the entire site to find 
one production

And as always, apologies for being a pessimist.
Jun 13 2010
parent reply BCS <none anon.com> writes:
Hello Ellery,


 On 06/13/2010 03:15 PM, BCS wrote:
 

 Right now we have two semi-official definitions of the D grammar; the
 docs (that are wrong) and the parser source (that is effectively
 unreadable by most people). I would like to propose a solution to
 this problem: eliminate one of them and derive it from the other.
 


 In the past, Walter has expressed a dislike of parser generators. I
 doubt you can get him to change his mind on that one, especially since
 most parser generators won't be able to handle D anyways, and I doubt
 any of them will be able to handle D efficiently (what are they called
 - GLR parser generators? Don't know about these).


I was very careful to leave both options open in the way I phrased that ;).


 
 As far as analysing parse.c goes, how much effort is it going to take
 to figure out what
 


void parseSomething()
{


 while(1)
 switch(token.value){
 case x:
 parseThis();
 continue;
 case y:
 parseThat();
 break;
 }
 break;
 }


}


Something ::= This* That;

The switch is just an optimization. As long as the code is regular enough, 
it can be analyzed. (I will grant that the resulting grammar might be ambiguous 
or LL(k) where k is higher than it needs to be)


 should look like in a grammar?
 
 Counterproposal: why don't we just fix the grammar?
 


That would be a start as well as a necessary step in getting a extractor 
working (to test the tools output against), but if the docs and the parser 
are maintained separately, the docs will be wrong sooner rather than later.


 I know this will be hard to do but it can be done incrementally with
 each step making improvements over the last.
 
 The first thing to do is put all of the description of the grammar in
 the docs into one place. If the literal text of each production is
 replace with a macro reference then the definitions of these macros
 can
 be put into a single file and expanded everywhere.
 The incremental improvement here is that having the grammar in one
 place
 by it's self will make it easier to check.


 I would very much like to see the entire grammar in one place. I can't
 tell you how obnoxious it is to search through the entire site to find
 one production


A while back I took a pass at pulling the grammar from the HTML via sed so 
I could stuff in into a parser generator. That it worked is the only good 
thing I can say about the experience.
 

 And as always, apologies for being a pessimist.
 


-- 
... <IXOYE><
Jun 13 2010
parent Don <nospam nospam.com> writes:
BCS wrote:

 That would be a start as well as a necessary step in getting a extractor 
 working (to test the tools output against), but if the docs and the 
 parser are maintained separately, the docs will be wrong sooner rather 
 than later.


I think that's very unlikely to be true. The existing errors in the 
grammer have formed during the long development of the language, and the 
grammer docs never really got much attention.
Now that the language has stabilised, the grammer is not going to change 
rapidly enough to be major source of errors.
The effort involved in making a tool to do it, probably exceeds the 
effort involved in just fixing it, by a factor of five or ten. There are 
many other things which could be done which would FAR more useful for D.
Jun 14 2010
prev sibling parent BCS <none anon.com> writes:
Hello BCS,


 The first thing to do is put all of the description of the grammar in
 the docs into one place. If the literal text of each production is
 replace with a macro reference then the definitions of these
 macros can be put into a single file and expanded everywhere.


[...]

 I'd be willing to do that first step if I got any (semi)official
 indication that it would be used.


Done, and because I can't attache the patch (it's to big) I started a issue.

http://d.puremagic.com/issues/show_bug.cgi?id=4308

As a side note, the formatting of the grammar is very irregular, using
different 
macros to get the same thing in different places. That would be the next 
thing I'd clean up.

-- 
... <IXOYE><
Jun 13 2010