www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Corruption of shared data with multiple threads

reply "Johan Engelen" <j j.nl> writes:
Hi all,
   I am working on adding coverage analysis (like DMD's) to LDC.
It is a work in progress, but you can test the current status 
here: https://github.com/JohanEngelen/ldc/tree/coverage
It already works nicely, but the line count is not increased for 
all types of statements yet. I've been breaking my head on a 
problem with multithreaded programs.

This test program works fine (ldc2 -cov test.d)
void main() {
    fun(...);
}
void fun(...) {
     ...
}
and outputs the same line count as dmd does.
Making the program multithreaded, like this:
import std.concurrency;
void main() {
    spawn(&fun,...);
}
void fun(...) {
     ...
}
no longer works. (it actually used to work for me, but for some 
reason with a completely rebuilt druntime, it crashes). I've 
traced the problem back to the Cover[] gdata data structure in 
druntime/cover.d becoming corrupted somewhere during program 
execution. More specifically, in struct Cover, filename.length 
becomes corrupted somehow.

Is it a known issue, that we have some bugs with shared data + 
multiple threads?

Thanks,
   Johan

(LLVM 3.6, MSVC, x64)
Mar 29 2015
next sibling parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc wrote:

 Is it a known issue, that we have some bugs with shared data + multiple
 threads?



affects migrating fibers between threads). We do optimize some things 
much more aggressively than DMD does, though, so it is possible that 
broken code works by accident on DMD but fails on LDC.


 (LLVM 3.6, MSVC, x64)


There might be some Win64-specific issues, though. Kai? You might want 
to test your changes on Linux to be sure.

  — David
Mar 29 2015
parent "Kai Nacke" <kai redstar.de> writes:
On Sunday, 29 March 2015 at 17:33:47 UTC, David Nadlinger wrote:

 On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc 
 wrote:

 (LLVM 3.6, MSVC, x64)


 There might be some Win64-specific issues, though. Kai? You 
 might want to test your changes on Linux to be sure.


I have no idea yet. Maybe you try running with the stub garbage 
collector? There are many parts in the MSVC build which are not 
well tested. I always see memory corruption if the garbage 
collector is not working correctly.

(On Linux/PPC I had some strange errors in std.string because the 
garbage collector freed memory which was still in use. It turned 
out that the TLS segment was not set up. Something similar could 
be the case on Win64. See druntime/src/rt/msvc.c for the current 
initialization of memory segments.)

Regards,
Kai
Mar 29 2015
prev sibling parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
Oh, I forgot to ask: Did you look into making use of LLVM's profiling 
data framework instead? We would likely get better performance this way, 
and might be able profit of all the tooling built around LLVM's coverage 
support down the road.

See for example:

http://llvm.org/devmtg/2014-04/PDFs/LightningTalks/EuroLLVM'14%20--%20ASan%20%2B%20Coverage.pdf
http://llvm.org/docs/CommandGuide/llvm-cov.html
http://llvm.org/docs/CoverageMappingFormat.html

But then, offering a DMD-compatible implementation might also be nice 
for users, and it's rather simple…

  — David


On 03/29/2015 07:33 PM, David Nadlinger wrote:

 On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc wrote:

 Is it a known issue, that we have some bugs with shared data + multiple
 threads?



 affects migrating fibers between threads). We do optimize some things
 much more aggressively than DMD does, though, so it is possible that
 broken code works by accident on DMD but fails on LDC.


 (LLVM 3.6, MSVC, x64)


 There might be some Win64-specific issues, though. Kai? You might want
 to test your changes on Linux to be sure.

   — David
Mar 29 2015
parent "Johan Engelen" <j j.nl> writes:
On Sunday, 29 March 2015 at 18:03:52 UTC, David Nadlinger wrote:

 Oh, I forgot to ask: Did you look into making use of LLVM's 
 profiling data framework instead? We would likely get better 
 performance this way, and might be able profit of all the 
 tooling built around LLVM's coverage support down the road.


I did not look at LLVM's profiling framework at all.
I wanted to copy DMD's coverage analysis, as indeed it seemed a 
straightforward simple task. It is nice to work on to get to know 
LLVM and LDC. :)

I hope someone can testdrive it on non-Windows, and give me some 
feedback.
Mar 29 2015