www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.internals - Using phobos in conjunction with the compiler code

reply RazvanN <razvan.nitu1305 gmail.com> writes:
After I compiled the parser as a library I wanted to test it to 
see if it successfully parses the phobos code. The code I am 
using is the following:


// test_parser.d
import std.stdio;
import std.file;

import ddmd.parse;
import ddmd.astbase;

import ddmd.id;
import ddmd.globals;

void main()
{
     string path = "../../phobos/std/";
     string regex = "*.d";

     // parser initializations
     Id.initialize();
     global._init();
     global.params.isLinux = true;
     global.params.is64bit = (size_t.sizeof == 8);
     ASTBase.Type._init();

     // get recursively all the .d files
     auto dFiles = dirEntries(path, regex, SpanMode.depth);
     foreach (d; dFiles)
     {
         writeln("Processing:", d.name);

         // comment the following line and it works
         string content = readText(d.name);

         scope p = new Parser!ASTBase(null, content, false);
         p.nextToken();
         p.parseModule();

         writeln("Finished!");
     }
}

This code fails with segmentation fault. The dFiles iterator gets 
broken somehow. If the line which reads a file is commented, than 
the code works. Also, if the parser imports are commented 
(ddmd.astbase and ddmd.parse) and all the lines of code which use 
code from those modules are commented, it also works.

I compiled the lexer and the parser with the following targets:


$G/lexer.a: $(LEXER_SRCS) $(LEXER_ROOT)
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -lib -of$  $(MODEL_FLAG) 
-I/tmp/.host_dmd-2.072.2/dmd2/src/druntime/import/ -J$G 
-L-lstdc++ $(DFLAGS) $(LEXER_SRCS) $(LEXER_ROOT)


$G/parser.a: $(PARSER_SRCS) $G/lexer.a $(ROOT_SRCS)
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -lib -of$  $(MODEL_FLAG) 
-I/tmp/.host_dmd-2.072.2/dmd2/src/druntime/import/ -J$G 
-L-lstdc++ $(DFLAGS) $(PARSER_SRCS) $G/lexer.a $(ROOT_SRCS) 
-Iddmd -J../res/

Then, the test file is compiled as follows:

parser_test: $G/parser.a test_parser.d
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -of$  $(MODEL_FLAG) -vtls -J$G 
-L-lstdc++ $(DFLAGS) -I$G $G/lexer.a $G/parser.a test_parser.d 
-Iddmd -J../res/

I thought it may be a mismatch between the versions of druntime 
used for the compilation of the lexer and parser libraries and 
the one used for phobos. So I recompiled druntime (master->head) 
with the latest stable dmd (2.072.2) and then compiled phobos 
(master->head) with the aforementioned dmd and druntime, still I 
get the segfault.

I also tried to compile the lexer and the parser using -conf= 
-defaultlib= -debuglib= and then I tried to pass the path to 
druntime with -I when compiling test_parser.d, but running the 
program also ends with segmentation fault.

Does anyone have any idea what the problem might be?

Thank you,
RazvanN
May 24
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
 Does anyone have any idea what the problem might be?

 Thank you,
 RazvanN
Since my emails don't seem to go through in a timely fashion:
 You should also disable the GC.
May 24
parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Wednesday, 24 May 2017 at 10:57:40 UTC, Mathias Lang wrote:
 On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
 Does anyone have any idea what the problem might be?

 Thank you,
 RazvanN
Since my emails don't seem to go through in a timely fashion:
 You should also disable the GC.
Thank you. It does, indeed, solve the problem. But I am curious, why was it segfaulting?
May 25
parent Mathias Lang <mathias.lang sociomantic.com> writes:
On Thursday, 25 May 2017 at 16:48:00 UTC, RazvanN wrote:
 On Wednesday, 24 May 2017 at 10:57:40 UTC, Mathias Lang wrote:
 On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
 Does anyone have any idea what the problem might be?

 Thank you,
 RazvanN
Since my emails don't seem to go through in a timely fashion:
 You should also disable the GC.
Thank you. It does, indeed, solve the problem. But I am curious, why was it segfaulting?
DMD uses a mix of allocations strategy. My guess is that some references to GC-allocated memory are stored into non GC-allocated memory, and those chunks of memory are not added as GC roots. So when the GC decides to collect memory, it thinks those objects are dead and recycle them. From there, you end up with a dangling pointer.
May 26