www.digitalmars.com         C & C++   DMDScript  

D - D parsing

reply "Patrick Down" <pdown austin.rr.com> writes:
First a quote from the overview section of  "The D Programming Language"

"D's lexical analyzer and parser are totally independent of each other and
of the semantic analyzer. This means it is easy to write simple tools to
manipulate D source perfectly without having to build a full compiler."

I know I could be reading more into this than intended but I've hoped this
means that a D parser will be one other the standard modules available to D
developers.  A tool that could parse D and create an AST (abstract syntax
tree) data structure as output on the back end would be great.

One of the reasons that I've brought this up is the documentation syntax
thread that is currently going on.  The HTML stripping system that D
has now is very cool. However there are other things that people like to do
when generating source code documentation.  For example building
indexes and cross references of symbols used in a the program. Doxygen (
http://www.stack.nl/~dimitri/doxygen/ ) is a good example of a tool that
does this for C/C++/Java/IDL.  A D paring too as described above would be
helpful for building such a documentation tool for D.

Documentation is not the only reason to desire a common D parsing module.
SWIG ( http://www.swig.org/ ) is a tool that helps bind C++ code to
scripting languages like Perl and Python, etc.  These tools read C++ source
and auto generate code that bind the C++ functions and classes to the
scripting runtime. There are also tools for adding persistence to C++
classes that work by reading the C++ source and auto generating code to load
and save a class.  There are profiling tools that add timing checks into the
code.

One of the common thing about all these source analysis/transformation tools
is that they often need a way for users to embed attributes in the source to
pass on to the tool. Most often this is done via special tags in the
comments.  However it might be nice to formalize a method for tools that
work with D to accomplish this.

One way to do this might be a tool keyword.  The D parsing module would
insert this text from a tool block verbatim into the AST but the D code
generator would ignore these nodes..  Third party D tools could use to D
parsing module to find and extract this data.  For example:

class Fred
{
  int MethodA() { }
  int MethodB() { }

  tool(Script_Binding)
  {
    do_not_export: MethodA
  }
}

The tools that generates the code to bind the module to a scriping language
would know that Fred is to be exported to the scriping language but MethodA
should not be exposed.

Another way to accomplish this might be to incorporate a way to insert a
standard data markup language like XML into the source

class Fred
{
  int  A_Attr;
  char[] B_Attr;
  float C_Attr;

  xml <persistant>
    <variable name="Attr_C" no_not_save="yes"/>
  </persistant>
}

This passes information on the persistence code generator not to bother
saving C_Attr.

I know that this all could still be accomplished with comment blocks.
However I see some value in a standardized tool and language syntax to allow
third party tools to extend D to a limited extent.  As usual I'm just
throwing out an idea for the general consensus to think over.

-Patrick Down
Jan 21 2002
parent reply "Juan Carlos Arevalo Baeza" <jcab roningames.com> writes:
"Patrick Down" <pdown austin.rr.com> wrote in message
news:a2i5d3$24h9$1 digitaldaemon.com...
 First a quote from the overview section of  "The D Programming Language"

 "D's lexical analyzer and parser are totally independent of each other and
 of the semantic analyzer. This means it is easy to write simple tools to
 manipulate D source perfectly without having to build a full compiler."

 I know I could be reading more into this than intended but I've hoped this
 means that a D parser will be one other the standard modules available to
D
 developers.  A tool that could parse D and create an AST (abstract syntax
 tree) data structure as output on the back end would be great.
That'd be good. In fact, ideally, the full D front-end would be programmed in D (is it, Walter?) and made part of the library. Salutaciones, JCAB
Jan 21 2002
next sibling parent reply "Walter" <walter digitalmars.com> writes:
"Juan Carlos Arevalo Baeza" <jcab roningames.com> wrote in message
news:a2i7qj$260r$1 digitaldaemon.com...
 "Patrick Down" <pdown austin.rr.com> wrote in message
 news:a2i5d3$24h9$1 digitaldaemon.com...
 First a quote from the overview section of  "The D Programming Language"

 "D's lexical analyzer and parser are totally independent of each other
and
 of the semantic analyzer. This means it is easy to write simple tools to
 manipulate D source perfectly without having to build a full compiler."

 I know I could be reading more into this than intended but I've hoped
this
 means that a D parser will be one other the standard modules available
to
 D
 developers.  A tool that could parse D and create an AST (abstract
syntax
 tree) data structure as output on the back end would be great.
That'd be good. In fact, ideally, the full D front-end would be programmed in D (is it, Walter?) and made part of the library. Salutaciones, JCAB
Dfront doesn't exist (yet)! -Walter
Jan 21 2002
parent reply "Juan Carlos Arevalo Baeza" <jcab roningames.com> writes:
"Walter" <walter digitalmars.com> wrote in message
news:a2iaoi$27ob$1 digitaldaemon.com...
 "Juan Carlos Arevalo Baeza" <jcab roningames.com> wrote in message
 news:a2i7qj$260r$1 digitaldaemon.com...

    In fact, ideally, the full D front-end would be
 programmed in D (is it, Walter?)
Dfront doesn't exist (yet)! -Walter
Yes, I understand that. But what I meant is that there's a front-end already done, which works with the Dihgital Mars compiler back-end. Right? That's how I understand the current alpha compiler is done... O:-) So, is it in D or C++? Out of pure curiosity. Salutaciones, JCAB
Jan 21 2002
parent "Walter" <walter digitalmars.com> writes:
"Juan Carlos Arevalo Baeza" <jcab roningames.com> wrote in message
news:a2ibgr$283f$1 digitaldaemon.com...
 "Walter" <walter digitalmars.com> wrote in message
 news:a2iaoi$27ob$1 digitaldaemon.com...
 "Juan Carlos Arevalo Baeza" <jcab roningames.com> wrote in message
 news:a2i7qj$260r$1 digitaldaemon.com...

    In fact, ideally, the full D front-end would be
 programmed in D (is it, Walter?)
Dfront doesn't exist (yet)! -Walter
Yes, I understand that. But what I meant is that there's a front-end already done, which works with the Dihgital Mars compiler back-end. Right? That's how I understand the current alpha compiler is done... O:-) So, is it in D or C++? Out of pure curiosity. Salutaciones, JCAB
It's in a sort of D-style variant of C++.
Jan 21 2002
prev sibling parent reply "Patrick Down" <pdown austin.rr.com> writes:
"Juan Carlos Arevalo Baeza" <jcab roningames.com> wrote in message
news:a2i7qj$260r$1 digitaldaemon.com...
 "Patrick Down" <pdown austin.rr.com> wrote in message
 news:a2i5d3$24h9$1 digitaldaemon.com...
 First a quote from the overview section of  "The D Programming Language"

 "D's lexical analyzer and parser are totally independent of each other
and
 of the semantic analyzer. This means it is easy to write simple tools to
 manipulate D source perfectly without having to build a full compiler."

 I know I could be reading more into this than intended but I've hoped
this
 means that a D parser will be one other the standard modules available
to
 D
 developers.  A tool that could parse D and create an AST (abstract
syntax
 tree) data structure as output on the back end would be great.
That'd be good. In fact, ideally, the full D front-end would be programmed in D (is it, Walter?) and made part of the library.
Yes, in fact I've always liked the idea of a moduler compiler. The code generator could be a module too. [D Code]->D_Parser->[AST] [AST]->Transform_To_Object_File->[OBJ] This allows you to insert your own steps in the middle. [D Code]->D_Parser->[AST] [AST]->Transform_To_New_AST->[AST] [AST]->Transform_To_Object_File->[OBJ] Or even [User Defined Language]->Convert_To_D_AST->[AST] [AST]->Transform_To_Object_File->[OBJ]
Jan 21 2002
parent "J. Daniel Smith" <j_daniel_smith deja.com> writes:
If this DML idea takes root, you would have most of what you wanted - maybe
even all of it depending on how closely the XML matched the D syntax.

But instead of dealing with a very specialized piece of code (the D parser),
you would do everything using off-the-shelf XML tools such as XSLT.  The
"[AST]" in your diagram would be replaced by "[DML]" and the "->(some
operation)->" would be a XSLT stylesheet.

   Dan

"Patrick Down" <pdown austin.rr.com> wrote in message
news:a2ibo0$28d3$1 digitaldaemon.com...
 "Juan Carlos Arevalo Baeza" <jcab roningames.com> wrote in message
 news:a2i7qj$260r$1 digitaldaemon.com...
 "Patrick Down" <pdown austin.rr.com> wrote in message
 news:a2i5d3$24h9$1 digitaldaemon.com...
 First a quote from the overview section of  "The D Programming
Language"
 "D's lexical analyzer and parser are totally independent of each other
and
 of the semantic analyzer. This means it is easy to write simple tools
to
 manipulate D source perfectly without having to build a full
compiler."
 I know I could be reading more into this than intended but I've hoped
this
 means that a D parser will be one other the standard modules available
to
 D
 developers.  A tool that could parse D and create an AST (abstract
syntax
 tree) data structure as output on the back end would be great.
That'd be good. In fact, ideally, the full D front-end would be programmed in D (is it, Walter?) and made part of the library.
Yes, in fact I've always liked the idea of a moduler compiler. The code generator could be a module too. [D Code]->D_Parser->[AST] [AST]->Transform_To_Object_File->[OBJ] This allows you to insert your own steps in the middle. [D Code]->D_Parser->[AST] [AST]->Transform_To_New_AST->[AST] [AST]->Transform_To_Object_File->[OBJ] Or even [User Defined Language]->Convert_To_D_AST->[AST] [AST]->Transform_To_Object_File->[OBJ]
Jan 22 2002