digitalmars.D.bugs - [Issue 11862] New: Using byChunk inside a std.parallelism's parallel loop causes crash when compiled in 32 bits
- d-bugmail puremagic.com (95/95) Jan 03 2014 https://d.puremagic.com/issues/show_bug.cgi?id=11862
- d-bugmail puremagic.com (24/24) Jan 03 2014 https://d.puremagic.com/issues/show_bug.cgi?id=11862
https://d.puremagic.com/issues/show_bug.cgi?id=11862 Summary: Using byChunk inside a std.parallelism's parallel loop causes crash when compiled in 32 bits Product: D Version: D2 Platform: x86_64 OS/Version: Windows Status: NEW Severity: major Priority: P2 Component: Phobos AssignedTo: nobody puremagic.com ReportedBy: ancien9 gmail.com The following code produce some weird behavior when compiled in 32bits on my Windows 7 64b machine. Removing the parallel or compiling in 64b removes the problems. import std.md5; import std.stdio; import std.file; import std.conv; import std.getopt; import std.string; import std.process; import std.parallelism; import std.exception; struct Entry { string name; ubyte[16] md5; } void main(string[] args) { string folder1 = args[1]; string folder2 = args[2]; Entry[] entries1; foreach (name; parallel(dirEntries(folder1, SpanMode.breadth), 1)) // not working { if(name.isFile()) { entries1 ~= Entry(name ,mdFile(name)); } } writeln(folder1," has ",entries1.length, " entries"); Entry[] entries2; foreach (string name; dirEntries(folder2, SpanMode.breadth)) //working fine { if(name.isFile()) { entries2 ~= Entry(name ,mdFile(name)); } } writeln(folder2," has ", entries2.length, " entries"); } /// Digests a file and prints the result. ubyte[16] mdFile(string filename) { MD5_CTX context; ubyte[16] digest; context.start(); File f = File(filename,"r"); foreach (buffer; f.byChunk(4096)) context.update(buffer); context.finish(digest); try{ f.close(); } catch(ErrnoException e) { writeln(e.errno); } writefln("MD5 (%s) = %s", filename, digestToString(digest)); return digest; } When executed, the program starts opening files and calculating their MD5. After a short amount of files have been scanned, depending on the WorkUnitSize (between 2 and 10 if set to 1, if default to 100, around 30-40 are processed), the program begins malfunctioning - Seemingly random files throw an ErrnoException (error n° 0) when closed (not interrupting the rest of the program) - The program sometime just freeze (need to close it manually) - The program generates "Object.Error: Access violation" - The program crashes (program.exe has stopped working). The culprit seems to be the function byChunk used to calculate the MD5. Using std.stream like suggested in this question's answer ( http://stackoverflow.com/questions/20888596/opening-files-in-a-parallel-loop-crash-the-program ) makes the program work in 32 and 64bits. While this issue can be fixed by compiling in 64bit, it's still a major problem. PS : It's my first time reporting a bug, so if some information is lacking or I've submitted this wrong, please let me know and I'll try to fix it. -- Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jan 03 2014
https://d.puremagic.com/issues/show_bug.cgi?id=11862 Daniel Kozak <kozzi11 gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kozzi11 gmail.com Problem seems to be in T[] rawRead(T)(T[] buffer) version(Win32) { immutable fd = ._fileno(_p.handle); //immutable mode = ._setmode(fd, _O_BINARY); when comment this line //scope(exit) ._setmode(fd, mode); and this line it seems to works much better version(DIGITAL_MARS_STDIO) { // BUG 4243 immutable info = __fhnd_info[fd]; __fhnd_info[fd] &= ~FHND_TEXT; scope(exit) __fhnd_info[fd] = info; } } -- Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Jan 03 2014