www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - core.reflect becomes recursive

reply Stefan Koch <uplink.coder googlemail.com> writes:
Good Afternoon everyone,

Just now I've added a nice tool to the core_reflect druntime 
branch.
The transitive visitor.

If you have read my previous posts on core.reflect [1], then you 
know that it works by giving you a simplified and stable version 
of the AST.

Those trees are not that nice to work with using your bare hands 
only.
You need to remember to descend the right leaves in your visitor.
And most of the time you are not interested in all nodes but only 
in a certain subset.
For example maybe you just want to know about all function 
declarations in your module.
In that case you would want to customize the behavior only for 
`FunctionDeclaration` and have everything else just descend to 
it's nodes.

Using the visitor pattern this can be done by implementing a 
visitor that looks through all fields of a given AST Node, and if 
a field is a subclass of the `Node` baseclass calls the accept 
method on that field.

Of course you can write such a `TransitiveVisitor` manually.
But that is boring repetitive work and therefore one is likely to 
get distracted and make mistakes.

Whereas if you could just iterate over all fields of all ASTNodes 
and ask the question if the type of the field has the `Node` 
baseclass you could have the code write itself.

You can do this today using templates but it somewhat tricky 
since you need to use pattern-matching is-expressions.

Let's have a look at how we use core.reflect to do this.

First we need the `ClassDeclaration` of the visitor.
`static immutable ClassDeclaration visitor = cast(immutable 
ClassDeclaration)nodeFromName("Visitor");`

The visitor will have overloads of the `visit` function for every 
visit-able type.
So lets go through all the `FunctionDeclaration`s for `visit`.

```
foreach(m;visitor.members)
{
   if (auto fd = cast(FunctionDeclaration) m)
   {
```

The NodeType we are looking for should be the first and only 
parameter.
And we need to get a class declaration from that type in order to 
iterate over the fields on that Node-subclass.

```
     assert(fd.name == "visit" && fd.type.paramterTypes.length == 
1);
     auto nodeType = fd.type.parameterTypes[0].type;
     auto cDecl = (cast(TypeClass) nodeType).sym; // this better 
be a class type

```

To see if a field is a sub-class of `Node` we need two small 
helper functions and one static variable

```
private static immutable ClassDeclaration node
   = cast(immutable ClassDeclaration) nodeFromName("Node");

bool isBaseOf (const ClassDeclaration Base, const 
ClassDeclaration C)
{
   auto base = cast() C.base;
   while (base)
   {
     if (base is Base)
     {
       return true;
     }
     base = cast() base.base;
   }
   return false;
}

static bool isNodeType(Type T)
{
   if (auto CT = cast(TypeClass) T)
   {
     auto C  = CT.sym;
     return C is node || node.isBaseOf(C);
   }
   else return false;
}
```

And now we can iterate:

```
     foreach(f; cDecl.fields)
     {
       if (isNodeType(f.type))
       {
         result ~= "    if (node." ~ f.name ~ ")\n";
         result ~= "        node." ~ f.name ~ ".accept(this);\n";
       }
     }
```

And from here everything is just a mixin away.


The code actual non abbreviate code without explanations is here:
https://github.com/UplinkCoder/druntime/blob/core_reflect/src/core/reflect/transitiveVisitor.d

[0] 
https://forum.dlang.org/thread/mmrkdmhymnojmjwvrrxg forum.dlang.org
Aug 08
parent reply Tejas <notrealemail gmail.com> writes:
On Sunday, 8 August 2021 at 18:50:31 UTC, Stefan Koch wrote:
 Good Afternoon everyone,

 Just now I've added a nice tool to the core_reflect druntime 
 branch.
 The transitive visitor.

 [...]
Hello, How does this compare to DMD as a library? https://dlang.org/blog/2017/08/01/a-dub-case-study-compiling-dmd-as-a-library/ Lower barrier to entry? More functionality?
Aug 09
parent reply max haughton <maxhaton gmail.com> writes:
On Monday, 9 August 2021 at 16:38:25 UTC, Tejas wrote:
 On Sunday, 8 August 2021 at 18:50:31 UTC, Stefan Koch wrote:
 Good Afternoon everyone,

 Just now I've added a nice tool to the core_reflect druntime 
 branch.
 The transitive visitor.

 [...]
Hello, How does this compare to DMD as a library? https://dlang.org/blog/2017/08/01/a-dub-case-study-compiling-dmd-as-a-library/ Lower barrier to entry? More functionality?
Very different concepts. This work is about introspection over D code, specifically at compile time. Specifically, providing a sane and *complete* interface that actually matches the form of the thing being introspected over (recall that `__traits` is linear whereas an AST is a tree). The data structures Stefan is introducing could theoretically be used as a clean interface to D code (a la `libdparse` but in sync with the compiler and more specifically the semantic analysis), i.e. not at compile time. dmd-as-a-library is just drinking from the firehose, you get the full AST, warts and all - or at least you *can* (You have the option to use `ASTBase`). You can't run dmd as a library at compile time to parse D code, unfortunately (I tried...).
Aug 09
parent Tejas <notrealemail gmail.com> writes:
On Monday, 9 August 2021 at 17:42:46 UTC, max haughton wrote:
 On Monday, 9 August 2021 at 16:38:25 UTC, Tejas wrote:
 On Sunday, 8 August 2021 at 18:50:31 UTC, Stefan Koch wrote:
 Good Afternoon everyone,
You can't run dmd as a library at
 compile time to parse D code, unfortunately (I tried...).
Ah, I thought that package had this functionality... then yes, this library needs to exist.
Aug 09