www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ICE handling is still terrible

reply Ethan <gooberman gmail.com> writes:
Example log at the bottom of this message.

Take a quick look and scroll back up after.

Can you guess what the cause is?

I feel like I've been saying "production environment" in almost 
every post I've been making here recently. And it's exactly this 
kind of thing that someone encountering when their job is on the 
line that will make them say many obscene things while rewriting 
their code in another language.

Do you know what caused the ICE yet?

I followed my hunch. Because. For some terrible reason DMD is 
still shipped as a 32-bit binary in Windows. And I know that 
Binderoo can blow out memory usage thanks to the CTFE engine's 
naive way of handling arrays.

So I grab the 2.086.1 source. Fix the problem I was having with 
the project file. Target a 64 bit build. Got me a DMD binary the 
way it should be officially distributed. Target my compiler to 
it, build my project. No errors, no ICE.

Wahoo.

Let's see what happens if I build my own 32-bit DMD though.

COMPILED : error : out of memory

Okay. Why my custom-built binary actually gives me a good error 
message and the official build doesn't is beyond me. That in 
itself is a large problem.

Still. It's an ICE. Follow the instructions and submit a bug, 
right?

Dustmite is not a solution. Dustmite is not a solution. Dustmite 
is not a solution. Dustmite is not a solution. Dustmite is not a 
solution.

The problem here is an out of memory error. Want to get it? Just 
call some CTFE function that generates large amounts of array 
data in an infinite loop.

Let's say some random person, like the one I mentioned above with 
their job on the line, tries to use a code minimisation tool to 
find the bug. The problem in the first place was that adding code 
caused the compiler to run out of memory. Stripping code away 
will not reproduce this error.

But what's a person without my experience to do when they get a 
problem like this? Stop work?

------------------------------------------------------------------------------

COMPILED : error : This is a compiler bug.
Please report it via https://issues.dlang.org/enter_bug.cgi
with, preferably, a reduced, reproducible example and the 
information below.
DustMite (https://github.com/CyberShadow/DustMite/wiki) can help 
with the reduction.
---
DMD %.*s
predefs   ANSI SmithyEditor DigitalMars Windows 
CRuntime_Microsoft CppRuntime_Microsoft LittleEndian D_Version2 
all D_SIMD D_InlineAsm_X86_64 X86_64 Win64 D_LP64 assert 
D_ModuleInfo D_Exceptions D_TypeInfo D_HardFloat
binary    C:\Program Files 
(x86)\dlang\dmd.2.086.1.windows\dmd2\windows\bin\dmd.exe
version   v2.086.1
config    C:\Program Files 
(x86)\dlang\dmd.2.086.1.windows\dmd2\windows\bin\sc.ini
DFLAGS    '-IC:\Program Files 
(x86)\dlang\dmd.2.086.1.windows\dmd2\windows\bin\..\..\src\phobos'
'-IC:\Program Files (x86)\dlang\dmd.2.086.1.windows\dmd2\windows\bin\..\..\
rc\druntime\import' -L/OPT:NOICF
---
object.Error (0): Access Violation
----------------
0x006C9DBE
0x0052C431
0x006D6B43
0x006D6AC5
0x006D695F
0x0052C3C3
0x75938494
0x778241C8
0x77824198
Jul 27 2019
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/27/2019 3:16 PM, Ethan wrote:

 Still. It's an ICE. Follow the instructions and submit a bug, right?


Yes, file a bug.



 object.Error (0): Access Violation


https://github.com/dlang/dmd/pull/10234

I don't know if that fixes your particular issue or not. It would be nice if
you 
could compile dmd with debug on so a stack trace has symbols in it.
Jul 28 2019
parent reply Ethan <gooberman gmail.com> writes:
On Sunday, 28 July 2019 at 09:20:22 UTC, Walter Bright wrote:

 Yes, file a bug.


Will do when I get some code that reproduces the undefined error 
outside of my codebase. Unless just the generated error message 
and mentioning that it's because of an out of memory error is 
enough in this case.

On Sunday, 28 July 2019 at 09:20:22 UTC, Walter Bright wrote:

 It would be nice if you could compile dmd with debug on so a 
 stack trace has symbols in it.


Did exactly that, and copied the PDBs with the executable just to 
double make sure it could locate them.

My out of memory error didn't give me a stack trace.

This is something I wanted to mention, actually. Uploading the 
debug symbols as an optional download with every official release 
would be fantastic in circumstances like this, just so it can 
give the user more information when things go wrong and clue them 
in to what they can do to work around the problem.
Jul 28 2019
parent a11e99z <black80 bk.ru> writes:
On Sunday, 28 July 2019 at 11:23:46 UTC, Ethan wrote:

 On Sunday, 28 July 2019 at 09:20:22 UTC, Walter Bright wrote:

 Yes, file a bug.




does DMD use DRT options? then try to use
try to use https://issues.dlang.org/show_bug.cgi?id=19947#c5
Jul 28 2019
prev sibling next sibling parent reply Exil <Exil gmall.com> writes:
On Saturday, 27 July 2019 at 22:16:39 UTC, Ethan wrote:

 Okay. Why my custom-built binary actually gives me a good error 
 message and the official build doesn't is beyond me. That in 
 itself is a large problem.


They still use optlink as the linker for the "official" release, 
that's the reason why.
Jul 28 2019
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 28 July 2019 at 12:23:43 UTC, Exil wrote:

 On Saturday, 27 July 2019 at 22:16:39 UTC, Ethan wrote:

 Okay. Why my custom-built binary actually gives me a good 
 error message and the official build doesn't is beyond me. 
 That in itself is a large problem.


 They still use optlink as the linker for the "official" 
 release, that's the reason why.


No, the default build is with debug symbols and without -release 
or -O. Hence, it yields nice stacktraces.
Jul 28 2019
parent reply Exil <Exil gmall.com> writes:
On Sunday, 28 July 2019 at 12:38:14 UTC, Seb wrote:

 On Sunday, 28 July 2019 at 12:23:43 UTC, Exil wrote:

 On Saturday, 27 July 2019 at 22:16:39 UTC, Ethan wrote:

 Okay. Why my custom-built binary actually gives me a good 
 error message and the official build doesn't is beyond me. 
 That in itself is a large problem.


 They still use optlink as the linker for the "official" 
 release, that's the reason why.


 No, the default build is with debug symbols and without 
 -release or -O. Hence, it yields nice stacktraces.


He's not taking about stack traces. You get an ICE with optlink 
instead of "Error: Out of Memory".
Jul 28 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/28/2019 5:48 AM, Exil wrote:

 You get an ICE with optlink instead of 
 "Error: Out of Memory".


The error Ethan reported wasn't coming from the linker.
Jul 28 2019
parent Exil <Exil gmall.com> writes:
On Sunday, 28 July 2019 at 21:32:38 UTC, Walter Bright wrote:

 On 7/28/2019 5:48 AM, Exil wrote:

 You get an ICE with optlink instead of "Error: Out of Memory".


 The error Ethan reported wasn't coming from the linker.


Sigh. The problem isn't with what was currently being compiled 
(kind of obvious). The problem is that the "official" DMD 
executable is compiled with optlink. Sorry I wasn't clear enough 
in my first post, I thought it was obvious.
Jul 28 2019
prev sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Saturday, 27 July 2019 at 22:16:39 UTC, Ethan wrote:

 Example log at the bottom of this message.

 Take a quick look and scroll back up after.

 [...]


Yes, dmd build as 64 bit application should definitely be a 
available for download and it should be the default on windows 
too.
Unfortunately I know only 1 developer who has the knowledge in 
this area and I know his time is very limited.

What issue do you have Dustmite?

Kind regards
Andre
Jul 28 2019
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/28/2019 11:25 AM, Andre Pany wrote:

 What issue do you have Dustmite?


Dustmite is not a good tool for out of memory errors, because the error can 
disappear whenever anything is removed from the code.
Jul 28 2019
parent FeepingCreature <feepingcreature gmail.com> writes:
On Sunday, 28 July 2019 at 21:33:57 UTC, Walter Bright wrote:

 Dustmite is not a good tool for out of memory errors, because 
 the error can disappear whenever anything is removed from the 
 code.



set -e
ulimit -v 1000000
dmd test.d 2>&1 |grep "out of memory"

The main problem you'll run into is dustmite removing a load 
bearing element of the error and turning one OOM into another 
OOM. This can be partially mitigated by running dmd with "-v" and 
grepping for the last line in the ordinary failing dmd output, to 
ensure it gets "at least that far."

Dustmite is not a magical cure-all. Like any tool, it needs some 
investment of effort to use it fully, and there's tasks it's less 
good at. But reducing ooms with dustmite is certainly possible.
Jul 29 2019