www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Corruption of shared data with multiple threads

reply "Johan Engelen" <j j.nl> writes:
Hi all,
   I am working on adding coverage analysis (like DMD's) to LDC.
It is a work in progress, but you can test the current status 
here: https://github.com/JohanEngelen/ldc/tree/coverage
It already works nicely, but the line count is not increased for 
all types of statements yet. I've been breaking my head on a 
problem with multithreaded programs.

This test program works fine (ldc2 -cov test.d)
void main() {
    fun(...);
}
void fun(...) {
     ...
}
and outputs the same line count as dmd does.
Making the program multithreaded, like this:
import std.concurrency;
void main() {
    spawn(&fun,...);
}
void fun(...) {
     ...
}
no longer works. (it actually used to work for me, but for some 
reason with a completely rebuilt druntime, it crashes). I've 
traced the problem back to the Cover[] gdata data structure in 
druntime/cover.d becoming corrupted somewhere during program 
execution. More specifically, in struct Cover, filename.length 
becomes corrupted somehow.

Is it a known issue, that we have some bugs with shared data + 
multiple threads?

Thanks,
   Johan

(LLVM 3.6, MSVC, x64)
Mar 29 2015
next sibling parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc wrote:
 Is it a known issue, that we have some bugs with shared data + multiple
 threads?
I don't know of any general threading issues besides #666 (which only affects migrating fibers between threads). We do optimize some things much more aggressively than DMD does, though, so it is possible that broken code works by accident on DMD but fails on LDC.
 (LLVM 3.6, MSVC, x64)
There might be some Win64-specific issues, though. Kai? You might want to test your changes on Linux to be sure. — David
Mar 29 2015
parent "Kai Nacke" <kai redstar.de> writes:
On Sunday, 29 March 2015 at 17:33:47 UTC, David Nadlinger wrote:
 On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc 
 wrote:
 (LLVM 3.6, MSVC, x64)
There might be some Win64-specific issues, though. Kai? You might want to test your changes on Linux to be sure.
I have no idea yet. Maybe you try running with the stub garbage collector? There are many parts in the MSVC build which are not well tested. I always see memory corruption if the garbage collector is not working correctly. (On Linux/PPC I had some strange errors in std.string because the garbage collector freed memory which was still in use. It turned out that the TLS segment was not set up. Something similar could be the case on Win64. See druntime/src/rt/msvc.c for the current initialization of memory segments.) Regards, Kai
Mar 29 2015
prev sibling parent reply David Nadlinger via digitalmars-d-ldc <digitalmars-d-ldc puremagic.com> writes:
Oh, I forgot to ask: Did you look into making use of LLVM's profiling 
data framework instead? We would likely get better performance this way, 
and might be able profit of all the tooling built around LLVM's coverage 
support down the road.

See for example:

http://llvm.org/devmtg/2014-04/PDFs/LightningTalks/EuroLLVM'14%20--%20ASan%20%2B%20Coverage.pdf
http://llvm.org/docs/CommandGuide/llvm-cov.html
http://llvm.org/docs/CoverageMappingFormat.html

But then, offering a DMD-compatible implementation might also be nice 
for users, and it's rather simple…

  — David


On 03/29/2015 07:33 PM, David Nadlinger wrote:
 On 03/29/2015 06:46 PM, Johan Engelen via digitalmars-d-ldc wrote:
 Is it a known issue, that we have some bugs with shared data + multiple
 threads?
I don't know of any general threading issues besides #666 (which only affects migrating fibers between threads). We do optimize some things much more aggressively than DMD does, though, so it is possible that broken code works by accident on DMD but fails on LDC.
 (LLVM 3.6, MSVC, x64)
There might be some Win64-specific issues, though. Kai? You might want to test your changes on Linux to be sure. — David
Mar 29 2015
parent "Johan Engelen" <j j.nl> writes:
On Sunday, 29 March 2015 at 18:03:52 UTC, David Nadlinger wrote:
 Oh, I forgot to ask: Did you look into making use of LLVM's 
 profiling data framework instead? We would likely get better 
 performance this way, and might be able profit of all the 
 tooling built around LLVM's coverage support down the road.
I did not look at LLVM's profiling framework at all. I wanted to copy DMD's coverage analysis, as indeed it seemed a straightforward simple task. It is nice to work on to get to know LLVM and LDC. :) I hope someone can testdrive it on non-Windows, and give me some feedback.
Mar 29 2015