• Prajwal S N (12/12) Oct 27 2023 Hi all,
• Imperatorn (2/14) Nov 02 2023 Are there any plans on extending ASTBase?
• Prajwal S N (15/16) Nov 03 2023 At the moment, no. `ASTBase` was created to provide a mechanism

Prajwal S N <snp dlang.org> writes:

Hi all,

Good news - the problem with using `appender!string` with the 
`extern (C++)` formatter class is no longer there. I've 
refactored the driver code to use a locking writer for both files 
and stdout. On a tangent, we've now found out that we cannot use 
`ASTBase` to build the AST - it does not store enough 
information, and also is missing some essential methods like 
`toChars()` that will inevitably be needed for the formatter to 
work. I've been working on using `ASTCodegen` instead, which 
comes with its own challenges of compiling imports. I hope to get 
it working and proceed with implementing a full traversal of the 
AST nodes soon.

Imperatorn <johan_forsberg_86 hotmail.com> writes:

On Friday, 27 October 2023 at 17:43:32 UTC, Prajwal S N wrote:
 Hi all,

 Good news - the problem with using `appender!string` with the 
 `extern (C++)` formatter class is no longer there. I've 
 refactored the driver code to use a locking writer for both 
 files and stdout. On a tangent, we've now found out that we 
 cannot use `ASTBase` to build the AST - it does not store 
 enough information, and also is missing some essential methods 
 like `toChars()` that will inevitably be needed for the 
 formatter to work. I've been working on using `ASTCodegen` 
 instead, which comes with its own challenges of compiling 
 imports. I hope to get it working and proceed with implementing 
 a full traversal of the AST nodes soon.

Are there any plans on extending ASTBase?

Prajwal S N <snp dlang.org> writes:

On Thursday, 2 November 2023 at 15:05:39 UTC, Imperatorn wrote:

 Are there any plans on extending ASTBase?

At the moment, no. `ASTBase` was created to provide a mechanism 
to build a complete AST and traverse it without having to do any 
of the semantic analysis passes. Unfortunately, this means that 
the classes used to represent the nodes in `ASTBase` are 
different from the ones used in `ASTCodegen`. There are certain 
missing features which prevent us from using it for cases where 
we need access to the tokens present in a node (e.g. the name of 
an identifier, or the body of a function). Extending it for our 
use case would not help, since the formatter needs to be able to 
walk the AST and write the tokens in each node in order to obtain 
the original source code. Since the compiler uses `ASTCodegen`, 
the nodes provided by it contain the correct tokens, and also 
contain the methods required to meaningfully generate a string 
representation of those tokens.