www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Retrieving the Scope of a statement

reply Cristian Creteanu <cristian.creteanu pm.me> writes:
Hi!

I am trying to implement autocomplete in DCD using DMD as a 
library. After parsing & the semantic analysis of a given source 
file, given a location (line & column number), I want to retrieve 
the Scope* that the statement at the given line is part of so 
that I can get the symbol table.

I noticed that every Scope has a pointer to its parent Scope 
(enclosing), but as far as I could see there's no way to access 
inner scopes with the information that there is currently in the 
Scope struct. Would it be ok to add a list of children scopes 
inside the struct? So, during semantic analysis, whenever the 
push method of the Scope structure is called I could also push 
the newly created scope inside this list of children?

What I tried to do without being able to access children directly 
was to create a visitor which goes through the entire AST until 
it reaches the statement at the given location. Once I've found 
this statement, a solution would be to find a Dsymbol inside of 
it and get its _scope, but for many of these symbols (for 
instance, local variables of functions) this field is null. I had 
a look at visit(VarDeclaration) in dsymbolsem.d and the _scope 
would be set for the variable declaration symbol if its parent 
was an aggregate declaration, but why isn't it set for variable 
declarations local to functions?

Is there any easier way to go about this?
Apr 20
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 20 April 2020 at 16:56:51 UTC, Cristian Creteanu wrote:
 Hi!

 I am trying to implement autocomplete in DCD using DMD as a 
 library. After parsing & the semantic analysis of a given 
 source file, given a location (line & column number), I want to 
 retrieve the Scope* that the statement at the given line is 
 part of so that I can get the symbol table.

 I noticed that every Scope has a pointer to its parent Scope 
 (enclosing), but as far as I could see there's no way to access 
 inner scopes with the information that there is currently in 
 the Scope struct. Would it be ok to add a list of children 
 scopes inside the struct?
No I think this is a wrong trail. You could be able to do that without modifying the compiler.
 So, during semantic analysis, whenever the push method of the 
 Scope structure is called I could also push the newly created 
 scope inside this list of children?

 What I tried to do without being able to access children 
 directly was to create a visitor which goes through the entire 
 AST until it reaches the statement at the given location.
Yes I think you can do that.
 Once I've found this statement, a solution would be to find a 
 Dsymbol inside of it and get its _scope, but for many of these 
 symbols (for instance, local variables of functions) this field 
 is null. I had a look at visit(VarDeclaration) in dsymbolsem.d 
 and the _scope would be set for the variable declaration symbol 
 if its parent was an aggregate declaration, but why isn't it 
 set for variable declarations local to functions?

 Is there any easier way to go about this?
For functions you can try to find the `Scope` when `ScopeStatement`s are visited because they have `.loc` (the begining) and `.endloc` information. So if the cursor loc is within the two values, the scope `sc` of the visitor (StatementSemanticVisitor.sc) gives you the available symbols at the cursor pos. Now technically there might be a problem because I dont see how you can plug the code to make this in `StatementSemanticVisitor`. This would require a kind of signal/callback/event system. Since the `Scope` is not attached to an `AstNode` you CANT use a custom visitor.
Apr 20
parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 21 April 2020 at 01:52:49 UTC, Basile B. wrote:
 On Monday, 20 April 2020 at 16:56:51 UTC, Cristian Creteanu 
 wrote:
 Hi!

 I am trying to implement autocomplete in DCD using DMD as a 
 library.
 [...]
[...] to an `AstNode` you CANT use a custom visitor.
Sorry maybe I've been too opiniated. in DMD, because of the lazy compilation model, visitors usually don't process a whole module so: - the scope for a statement is the scope that's passed to the `StatementSemanticVisitor` class created when `statementSemantic()` is called for this statement. - the scope for an expression is the scope that's passed to the `ExpressionSemanticVisitor` class created when `expressionSemantic()` is called for this expression. - the scope for a type is the scope that's passed to the `TypeSemanticVisitor` class created when `typeSemantic()` is called for this type. From your original message I have the impression that you want to retrieve a scope by a node, and you noticed that this is not always possible. The time where a scope is always linked to a node is when its semantic is run, and it's a class member. So maybe (and only maybe) there's something to do with that fact but as said at first glance I don't see how, without a kind of callback system.
Apr 20
parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 21 April 2020 at 06:58:47 UTC, Basile B. wrote:
 On Tuesday, 21 April 2020 at 01:52:49 UTC, Basile B. wrote:
 On Monday, 20 April 2020 at 16:56:51 UTC, Cristian Creteanu 
 wrote:
 Hi!

 I am trying to implement autocomplete in DCD using DMD as a 
 library.
 [...]
[...] to an `AstNode` you CANT use a custom visitor.
Sorry maybe I've been too opiniated. in DMD, because of the lazy compilation model, visitors usually don't process a whole module so: - the scope for a statement is the scope that's passed to the `StatementSemanticVisitor` class created when `statementSemantic()` is called for this statement. - the scope for an expression is the scope that's passed to the `ExpressionSemanticVisitor` class created when `expressionSemantic()` is called for this expression. - the scope for a type is the scope that's passed to the `TypeSemanticVisitor` class created when `typeSemantic()` is called for this type. From your original message I have the impression that you want to retrieve a scope by a node, and you noticed that this is not always possible. The time where a scope is always linked to a node is when its semantic is run, and it's a class member. So maybe (and only maybe) there's something to do with that fact but as said at first glance I don't see how, without a kind of callback system.
Maybe that adding this in the compiler globals: alias OnStatementSemanticStart = void function(Statement, Scope*); alias OnStatementSemanticDone = void function(Statement, Scope*); alias OnExpressionSemanticStart = void function(Expression, Scope*); alias OnExpressionSemanticDone = void function(Expression, Scope*); OnStatementSemanticStart onStatementSemanticStart; OnStatementSemanticDone onStatementSemanticDone; OnExpressionSemanticStart onExpressionSemanticStart; OnExpressionSemanticDone onExpressionSemanticDone; and add the handlers in statementSemantic() and expressionSemantic() could help to build a useful enough hash map but without significant memory impact on the compiler. Actually just the "Done" events would be usefull because of the many lowering done during semantics.
Apr 21
parent reply Cristian Creteanu <cristian.creteanu pm.me> writes:
On Tuesday, 21 April 2020 at 07:45:23 UTC, Basile B. wrote:
   alias OnStatementSemanticStart = void function(Statement, 
 Scope*);
   alias OnStatementSemanticDone = void function(Statement, 
 Scope*);
   alias OnExpressionSemanticStart = void function(Expression, 
 Scope*);
   alias OnExpressionSemanticDone = void function(Expression, 
 Scope*);

   OnStatementSemanticStart onStatementSemanticStart;
   OnStatementSemanticDone onStatementSemanticDone;
   OnExpressionSemanticStart onExpressionSemanticStart;
   OnExpressionSemanticDone onExpressionSemanticDone;

 and add the handlers in statementSemantic() and 
 expressionSemantic()
Thank you so much! It worked :D
May 02
parent reply Basile B. <b2.temp gmx.com> writes:
On Saturday, 2 May 2020 at 21:36:00 UTC, Cristian Creteanu wrote:
 On Tuesday, 21 April 2020 at 07:45:23 UTC, Basile B. wrote:
   alias OnStatementSemanticStart = void function(Statement, 
 Scope*);
   alias OnStatementSemanticDone = void function(Statement, 
 Scope*);
   alias OnExpressionSemanticStart = void function(Expression, 
 Scope*);
   alias OnExpressionSemanticDone = void function(Expression, 
 Scope*);

   OnStatementSemanticStart onStatementSemanticStart;
   OnStatementSemanticDone onStatementSemanticDone;
   OnExpressionSemanticStart onExpressionSemanticStart;
   OnExpressionSemanticDone onExpressionSemanticDone;

 and add the handlers in statementSemantic() and 
 expressionSemantic()
Thank you so much! It worked :D
I've seen the PR [1], so it's ok, you manage to get something to work with this ? If so in DMD I would prefer a `version(callback_API)` because at some point more callbacks might be required, added, but not necessarily tied to the dmd library version. [1] https://github.com/dlang/dmd/pull/11092
May 02
parent Cristian Creteanu <cristian.creteanu pm.me> writes:
On Sunday, 3 May 2020 at 03:10:34 UTC, Basile B. wrote:
 I've seen the PR [1], so it's ok, you manage to get something 
 to work with this ?
 [1] https://github.com/dlang/dmd/pull/11092
Yes, I did manage to get results out of this approach.
 If so in DMD I would prefer a `version(callback_API)` because 
 at some point more callbacks might be required, added, but not 
 necessarily tied to the dmd library version.
You're right, I will change the version name in the PR.
May 03
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 20 April 2020 at 16:56:51 UTC, Cristian Creteanu wrote:
 Hi!

 I am trying to implement autocomplete in DCD using DMD as a 
 library. After parsing & the semantic analysis of a given 
 source file, given a location (line & column number), I want to 
 retrieve the Scope* that the statement at the given line is 
 part of so that I can get the symbol table.
In addition to what Basile mentioned, you can have a look at VisualD [1], which now uses DMD for semantic analysis and intellisense. [1] https://github.com/rainers/visuald/blob/master/vdc/semantic.d -- /Jacob Carlborg
Apr 23
parent reply Cristian Creteanu <cristian.creteanu pm.me> writes:
On Thursday, 23 April 2020 at 09:26:22 UTC, Jacob Carlborg wrote:
 In addition to what Basile mentioned, you can have a look at 
 VisualD [1], which now uses DMD for semantic analysis and 
 intellisense.

 [1] 
 https://github.com/rainers/visuald/blob/master/vdc/semantic.d

 --
 /Jacob Carlborg
It seems that VisualD uses scopes defined internally rather than the ones used by DMD
May 02
parent Cristian Creteanu <cristian.creteanu pm.me> writes:
On Saturday, 2 May 2020 at 21:40:28 UTC, Cristian Creteanu wrote:
 On Thursday, 23 April 2020 at 09:26:22 UTC, Jacob Carlborg 
 wrote:
 In addition to what Basile mentioned, you can have a look at 
 VisualD [1], which now uses DMD for semantic analysis and 
 intellisense.

 [1] 
 https://github.com/rainers/visuald/blob/master/vdc/semantic.d

 --
 /Jacob Carlborg
It seems that VisualD uses scopes defined internally rather than the ones used by DMD
My bad, VisualD does use ScopeDsymbol instead, so I should've taken a closer look. Thanks for the suggestion, I will pay more attention to VisualD in the future.
May 03