www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Query Context at Source File and Offset in DMD

reply =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
I'm working on a feature in DMD to query lexical, syntactic and 
semantic context at a given byte offset in a given source file.

In the long run my vision is to add libclang-like functionality 
directly into DMD and a specific mode in DMD that just starts up 
DMD to query some information at a file and offset in a given 
source file typically given by an editor.

For practise and to learn the insides of DMD I've so far added 
the following to Lexer::nextToken() a long with some other stuff 
in lexer.[ch] and mars.c not shown here.

TOK Lexer::nextToken()
{
     if (token.next)
     {
         Token *t = token.next;
         memcpy(&token,t,sizeof(Token));
         t->next = freelist;
         freelist = t;
     }
     else
     {
         scan(&token);
     }

     const long begin = token.ptr - this->base; // token begin 
offset
     const long end = begin + strlen(token.toChars(token.value)); 
// token end offset
     if (begin <= global.params.queryAtOffset &&
         global.params.queryAtOffset < end)
     {

         token.printDoc();
         exit(0);
     }
     return token.value;
}

to query lexical information at point using a new DMD flag

-query=FILE:OFFSET

However, most calls to DMD parses several source files. So I need 
some way to know which file I'm currently parsing from inside the 
lexer. GDB stack is

0  in Lexer::nextToken of lexer.c:706
1  in Parser::check of parse.c:5148
2  in Parser::parseStatement of parse.c:4126
3  in Parser::parseStatement of parse.c:4289
4  in Parser::parseContracts of parse.c:3660
5  in Parser::parseDeclarations of parse.c:3480
6  in Parser::parseDeclDefs of parse.c:293
7  in Parser::parseAggregate of parse.c:1933
8  in Parser::parseDeclarations of parse.c:3303
9  in Parser::parseDeclDefs of parse.c:293
10 in Parser::parseModule of parse.c:162
11 in Module::parse of module.c:515
12 in tryMain of mars.c:1512
13 in main of mars.c:1827

The only way I see of solving this for now is to add a global 
copy of the Module::srcfile accessible to make this information 
accessible within the lexer. Does anybody have a better idea?
Apr 05 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-04-05 23:22, "Nordlöw" wrote:

 The only way I see of solving this for now is to add a global copy of
 the Module::srcfile accessible to make this information accessible
 within the lexer. Does anybody have a better idea?
The Lexer's constructor takes a Module as the first parameter [1]. A Module has the original argument name [2], I assume that's the filename. Just store the module/filename as an instance variable in the lexer and access it where you need it. [1] https://github.com/D-Programming-Language/dmd/blob/master/src/lexer.c#L245 [2] https://github.com/D-Programming-Language/dmd/blob/master/src/module.h#L76 -- /Jacob Carlborg
Apr 06 2014
parent =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
 The Lexer's constructor takes a Module as the first parameter 
 [1]. A Module has the original argument name [2], I assume 
 that's the filename. Just store the module/filename as an 
 instance variable in the lexer and access it where you need it.

 [1] 
 https://github.com/D-Programming-Language/dmd/blob/master/src/lexer.c#L245

 [2] 
 https://github.com/D-Programming-Language/dmd/blob/master/src/module.h#L76
Thx.
Apr 06 2014