www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Seemingly patternless optlink premature termination

reply pineapple <meapineapple gmail.com> writes:
I'm working on my dumb library and I have run into a case where 
when I add this line to a module, I get an optlink error 
attempting to compile it:

     unittest{}

The module compiles fine without it. So do the modules which 
depend on it.

But with that line? optlink errors everywhere.

A screencap of the error box that pops up:

     http://puu.sh/tfmBK/c5068eb2fb.png

And the contents of stdout:

     checkpoint(256)
     --- errorlevel 1

I encountered a similar error right before this and eventually 
worked out that I had introduced a cyclic dependency into my 
project some time ago - strangely not rearing its head as an 
error until I made a completely unrelated addition to the tests 
in a module incidentally importing one of the modules involved in 
that dependency cycle - and addressing that issue seemed to 
resolve the optlink error. (At least until I tried to add some 
extra unit tests to one of the modules involved in that cycle.)

I am utterly failing to find a way to make a simple repro case 
from this, or to understand what could conceivably be causing 
this error to occur in a module whose dependency graph is small 
and simple.
Jan 08
next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 8 January 2017 at 14:15:26 UTC, pineapple wrote:
 I'm working on my dumb library and I have run into a case where 
 when I add this line to a module, I get an optlink error 
 attempting to compile it:

     unittest{}
After deciding to let my inability to add some unit tests to that module I started working on some other code in the same project. At this point it seems completely arbitrary which lines produce optlink errors. Since the `unittest{}` issue I have, in other modules, had optlink errors occur depending on the presence of arbitrary method and function calls, too.
Jan 08
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 8 January 2017 at 14:42:57 UTC, pineapple wrote:
 After deciding to let my inability to add some unit tests to 
 that module I started working on some other code in the same 
 project. At this point it seems completely arbitrary which 
 lines produce optlink errors. Since the `unittest{}` issue I 
 have, in other modules, had optlink errors occur depending on 
 the presence of arbitrary method and function calls, too.
Update from IRC: Stefan was able to determine that this is a stack corruption bug with optlink. (How and why I should be the one to stumble upon it is yet to be determined.)
Jan 08
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 8 January 2017 at 17:14:13 UTC, pineapple wrote:
 On Sunday, 8 January 2017 at 14:42:57 UTC, pineapple wrote:
 After deciding to let my inability to add some unit tests to 
 that module I started working on some other code in the same 
 project. At this point it seems completely arbitrary which 
 lines produce optlink errors. Since the `unittest{}` issue I 
 have, in other modules, had optlink errors occur depending on 
 the presence of arbitrary method and function calls, too.
Update from IRC: Stefan was able to determine that this is a stack corruption bug with optlink. (How and why I should be the one to stumble upon it is yet to be determined.)
No I was not. I think! it could be one. I'll post it when there is something concrete to post.
Jan 08
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 8 January 2017 at 17:14:13 UTC, pineapple wrote:
 On Sunday, 8 January 2017 at 14:42:57 UTC, pineapple wrote:
 After deciding to let my inability to add some unit tests to 
 that module I started working on some other code in the same 
 project. At this point it seems completely arbitrary which 
 lines produce optlink errors. Since the `unittest{}` issue I 
 have, in other modules, had optlink errors occur depending on 
 the presence of arbitrary method and function calls, too.
Update from IRC: Stefan was able to determine that this is a stack corruption bug with optlink. (How and why I should be the one to stumble upon it is yet to be determined.)
No I was able to determine it not. I think, it could be one. I'll post it when there is something concrete to post.
Jan 08
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 6:15 AM, pineapple wrote:
 I'm working on my dumb library and I have run into a case where when I add this
 line to a module, I get an optlink error attempting to compile it:

     unittest{}

 The module compiles fine without it. So do the modules which depend on it.

 But with that line? optlink errors everywhere.

 A screencap of the error box that pops up:

     http://puu.sh/tfmBK/c5068eb2fb.png

 And the contents of stdout:

     checkpoint(256)
     --- errorlevel 1

 I encountered a similar error right before this and eventually worked out that
I
 had introduced a cyclic dependency into my project some time ago - strangely
not
 rearing its head as an error until I made a completely unrelated addition to
the
 tests in a module incidentally importing one of the modules involved in that
 dependency cycle - and addressing that issue seemed to resolve the optlink
 error. (At least until I tried to add some extra unit tests to one of the
 modules involved in that cycle.)

 I am utterly failing to find a way to make a simple repro case from this, or to
 understand what could conceivably be causing this error to occur in a module
 whose dependency graph is small and simple.
Please post bug reports to bugzilla. I transcribed the error here: EAX=00000010 EBX=0000001C ECX=00000000 EDX=004337CF ESI=0044AAE8 EDI=00000100 EBP=0018FF54 ESP=0018FDEC EIP=0040347E First=00402000 Try to include the code that causes the problem. The way to debug this (for anyone interested) is to get it to fail running under windbg.exe. Then examine the instructions around where it failed. Look in the source code to find where those instructions are, and hence the location in the code where it faulted. P.S. Isn't it maddening that Windows doesn't allow copying the data from a message window?
Jan 08
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 On 1/8/2017 6:15 AM, pineapple wrote:
 I'm working on my dumb library and I have run into a case 
 where when I add this
 line to a module, I get an optlink error attempting to compile 
 it:

     unittest{}

 The module compiles fine without it. So do the modules which 
 depend on it.

 But with that line? optlink errors everywhere.

 A screencap of the error box that pops up:

     http://puu.sh/tfmBK/c5068eb2fb.png

 And the contents of stdout:

     checkpoint(256)
     --- errorlevel 1

 I encountered a similar error right before this and eventually 
 worked out that I
 had introduced a cyclic dependency into my project some time 
 ago - strangely not
 rearing its head as an error until I made a completely 
 unrelated addition to the
 tests in a module incidentally importing one of the modules 
 involved in that
 dependency cycle - and addressing that issue seemed to resolve 
 the optlink
 error. (At least until I tried to add some extra unit tests to 
 one of the
 modules involved in that cycle.)

 I am utterly failing to find a way to make a simple repro case 
 from this, or to
 understand what could conceivably be causing this error to 
 occur in a module
 whose dependency graph is small and simple.
Please post bug reports to bugzilla. I transcribed the error here: EAX=00000010 EBX=0000001C ECX=00000000 EDX=004337CF ESI=0044AAE8 EDI=00000100 EBP=0018FF54 ESP=0018FDEC EIP=0040347E First=00402000 Try to include the code that causes the problem. The way to debug this (for anyone interested) is to get it to fail running under windbg.exe. Then examine the instructions around where it failed. Look in the source code to find where those instructions are, and hence the location in the code where it faulted. P.S. Isn't it maddening that Windows doesn't allow copying the data from a message window?
Any chance to get a debug build of optlink ?
Jan 08
prev sibling next sibling parent reply jkpl <jkpl nowhere.de> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 P.S. Isn't it maddening that Windows doesn't allow copying the 
 data from a message window?
Not really, because you can, Ctrl+C...
Jan 08
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 10:56 AM, jkpl wrote:
 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 P.S. Isn't it maddening that Windows doesn't allow copying the data from a
 message window?
Not really, because you can, Ctrl+C...
That frequently does not work. For example, take the "Help" window of Thunderbird, which is my email program. You cannot copy data from it.
Jan 08
prev sibling next sibling parent reply Jerry <hurricane hereiam.com> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 P.S. Isn't it maddening that Windows doesn't allow copying the 
 data from a message window?
A bit more maddening that D is still trying to support optlink, a linker that hasn't seen a commit in 2 years when it is hardly bug free.
Jan 08
parent reply jkpl <jkpl somewhere.de> writes:
On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:
 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 P.S. Isn't it maddening that Windows doesn't allow copying the 
 data from a message window?
A bit more maddening that D is still trying to support optlink, a linker that hasn't seen a commit in 2 years when it is hardly bug free.
Your comment is not pertinent. The object format doesn't change so the linker doesn't need to be updated. The problem here seems to be a bug. Until someone finds it, it can't be fixed.
Jan 08
parent reply Jerry <hurricane hereiam.com> writes:
On Sunday, 8 January 2017 at 20:36:38 UTC, jkpl wrote:
 On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:
 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 P.S. Isn't it maddening that Windows doesn't allow copying 
 the data from a message window?
A bit more maddening that D is still trying to support optlink, a linker that hasn't seen a commit in 2 years when it is hardly bug free.
Your comment is not pertinent. The object format doesn't change so the linker doesn't need to be updated. The problem here seems to be a bug. Until someone finds it, it can't be fixed.
We are discussing Optlink and a bug related to it. The fact it's 20 years old, written in assembly, and hasn't had a bug fix in years even though they have been reported is very very very much relevant. It's legacy software that shouldn't be relied on unless you also consider D to be legacy (I think most probably won't).
Jan 08
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:
 A bit more maddening that D is still trying to support optlink, a linker that
 hasn't seen a commit in 2 years when it is hardly bug free.
Optlink has had many bugs fixed in it. There are currently 7 open issues with it on bugzilla: https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=dmd&list_id=212971&query_format=advanced&short_desc=optlink&short_desc_type=allwordssubstr This includes the issue in this thread. Only 1 of the other issues looks like a bug in Optlink. If you know of unreported issues with Optlink, please report them to bugzilla.
Jan 08
parent reply Jerry <hurricane hereiam.com> writes:
On Monday, 9 January 2017 at 01:26:48 UTC, Walter Bright wrote:
 On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:
 A bit more maddening that D is still trying to support 
 optlink, a linker that
 hasn't seen a commit in 2 years when it is hardly bug free.
Optlink has had many bugs fixed in it. There are currently 7 open issues with it on bugzilla: https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=dmd&list_id=212971&query_format=advanced&short_desc=optlink&short_desc_type=allwordssubstr This includes the issue in this thread. Only 1 of the other issues looks like a bug in Optlink.
There's more, just the bugs are so obscure the people filing the bugs don't know that the problem lies with Optlink. There's problems with shared libraries not being loaded and unloaded properly. Random crashes during runtime, more often crashes when a debugger is attached. Some more that I ran into as well before I switched away from optlink. What point is there to report these bugs though, Microsoft's linker works with fewer bugs and actually supports the format of the platform. So no need to convert .lib files with it. This is my point, it's just one more thing that has to be maintained, but it's not even being maintained. Supporting MinGW would be a better option as then you don't have to maintain the linker and not have it degrade over time.
 If you know of unreported issues with Optlink, please report 
 them to bugzilla.
To what end? There isn't even an Optlink specific bug list. Look at the bug with Optlink related to data that has more than 16 MB. What happened with that? Was that known bug fixed? No, a limitation was introduced into DMD to accommodate Optlink's bug. Rather than doing the sane decision to fix the bug in Optlink. Why? Probably because no one wants to deal with assembly.
Jan 08
next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Monday, 9 January 2017 at 02:34:31 UTC, Jerry wrote:
 before I switched away from optlink. What point is there to 
 report these bugs though, Microsoft's linker works with fewer 
 bugs and actually supports the format of the platform. So no 
 need to convert .lib files with it. This is my point, it's just 
 one more thing that has to be maintained, but it's not even 
 being maintained. Supporting MinGW would be a better option as 
 then you don't have to maintain the linker and not have it 
 degrade over time.
On this subject, I spent a good chunk of my day trying to compile my code using the -m32mscoff switch and was never able to get it to work. (My most recent obstacle is this error, occurring even for a very simple test program importing no dependencies, if anyone cares to take a look: https://hastebin.com/vezojehaxo.txt) On Monday, 9 January 2017 at 01:27:27 UTC, Walter Bright wrote:
 On 1/8/2017 12:58 PM, pineapple wrote:
 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 Please post bug reports to bugzilla.
Here you go: https://issues.dlang.org/show_bug.cgi?id=17077
Thank you.
Regarding the above: Because I have not managed to get an alternative to optlink working, I currently am unable to work on mach on my Windows desktop. In order to keep up progress I've been having to write and compile code on OSX on a laptop; it is nowhere near an ideal workflow. I understand you likely have a lot on your plate, but please prioritize solving this problem. It's causing me a great deal of irritation and inconvenience, and it's embarrassing that my library now simply won't compile on Windows.
Jan 08
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 8:24 PM, pineapple wrote:
 On this subject, I spent a good chunk of my day trying to compile my code using
 the -m32mscoff switch and was never able to get it to work. (My most recent
 obstacle is this error, occurring even for a very simple test program importing
 no dependencies, if anyone cares to take a look:
 https://hastebin.com/vezojehaxo.txt)
Please post bug reports to bugzilla, not pastebins.
 Regarding the above: Because I have not managed to get an alternative to
optlink
 working, I currently am unable to work on mach on my Windows desktop. In order
 to keep up progress I've been having to write and compile code on OSX on a
 laptop; it is nowhere near an ideal workflow. I understand you likely have a
lot
 on your plate, but please prioritize solving this problem. It's causing me a
 great deal of irritation and inconvenience, and it's embarrassing that my
 library now simply won't compile on Windows.
You could try compiling it with -m64.
Jan 08
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 6:34 PM, Jerry wrote:
 There's more, just the bugs are so obscure the people filing the bugs don't
know
 that the problem lies with Optlink. There's problems with shared libraries not
 being loaded and unloaded properly. Random crashes during runtime, more often
 crashes when a debugger is attached. Some more that I ran into as well before I
 switched away from optlink. What point is there to report these bugs though,
If you don't report bugs, they are guaranteed to not get fixed. There's nothing anyone can do with the statement you made.
 If you know of unreported issues with Optlink, please report them to bugzilla.
To what end?
Complaining that somebody didn't fix problems neither you nor anyone else reported is not very sporting.
 There isn't even an Optlink specific bug list.
I found the list I posted by searching bugzilla for "optlink". If there are any I missed, please let me know.
Jan 08
next sibling parent ag0aep6g <anonymous example.com> writes:
On 01/09/2017 08:05 AM, Walter Bright wrote:
 I found the list I posted by searching bugzilla for "optlink". If there
 are any I missed, please let me know.
snn.lib is closely related to optlink, because COFF tools have no business with snn.lib. https://issues.dlang.org/buglist.cgi?quicksearch=snn
Jan 09
prev sibling next sibling parent reply Mike Wey <mike-wey example.com> writes:
On 01/09/2017 08:05 AM, Walter Bright wrote:
 There isn't even an Optlink specific bug list.
I found the list I posted by searching bugzilla for "optlink". If there are any I missed, please let me know.
https://issues.dlang.org/show_bug.cgi?id=15418 Is there anything i should add to mark it as an optlink bug? -- Mike Wey
Jan 09
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/9/2017 10:57 AM, Mike Wey wrote:
 https://issues.dlang.org/show_bug.cgi?id=15418

 Is there anything i should add to mark it as an optlink bug?
I put Optlink in the title for you. Also created an optlink keyword.
Jan 09
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/9/2017 10:57 AM, Mike Wey wrote:
 Is there anything i should add to mark it as an optlink bug?
I added "optlink" as a keyword, so you can search that way: https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&keywords=optlink&keywords_type=allwords&list_id=213005&query_format=advanced Of course, this relies on bugzilla issues about optlink to be tagged with the keyword.
Jan 09
prev sibling next sibling parent Jerry <hurricane hereiam.com> writes:
On Monday, 9 January 2017 at 07:05:27 UTC, Walter Bright wrote:
 If you don't report bugs, they are guaranteed to not get fixed. 
 There's nothing anyone can do with the statement you made.
If it's an Optlink bug it is pretty much a guarantee regardless even if the bug is reported.
 To what end?
Complaining that somebody didn't fix problems neither you nor anyone else reported is not very sporting.
Guess you missed literally the rest of that paragraph that outlines how bugs are actually handled in regards to optlink, let me repost it: Look at the bug with Optlink related to data that has more than 16 MB. What happened with that? Was that known bug fixed? No, a limitation was introduced into DMD to accommodate Optlink's bug. Rather than doing the sane decision to fix the bug in Optlink. Why? Probably because no one wants to deal with assembly. Bugs get reported, reported bugs don't get fixed, limitations are introduced to accommodate bugs. That's the way Optlink is handled. I'd more "sporting" if "please report bugs for optlink" wasn't synonymous for, "please think we care about (and might actually fix it) the bug you found with our project even though no one has touched optlink in 2 years". At some point you have to say, yes this isn't working, we should do something else. It becomes that much more difficult when the other party isn't even willing to accept there's a problem when optlink hasn't had a commit in 2 years. Stern words need to be used to get it through to the stubborn party.
 There isn't even an Optlink specific bug list.
I found the list I posted by searching bugzilla for "optlink". If there are any I missed, please let me know.
I meant a subsection of like dmd/druntime/phobos, but I guess that wouldn't help if people don't know the bugs they are reporting are for optlink.
Jan 09
prev sibling next sibling parent Jerry <hurricane hereiam.com> writes:
On Monday, 9 January 2017 at 07:05:27 UTC, Walter Bright wrote:
 Complaining that somebody didn't fix problems neither you nor 
 anyone else reported is not very sporting.
Just to clarify, I'm not complaining about a problem not being fixed. I'm advocating for the removal of Optlink, I don't care if it gets fixed or not cause I'm not using it anymore. I don't want new people to go through the problems I had to go through using it, before I realized I could do the more sane thing and just use Microsoft's linker.
Jan 09
prev sibling parent Anonymouse <asdf asdf.net> writes:
On Monday, 9 January 2017 at 07:05:27 UTC, Walter Bright wrote:
 If you don't report bugs, they are guaranteed to not get fixed. 
 There's nothing anyone can do with the statement you made.
A good quote is "unreported bugs can only be fixed by accident."
Jan 10
prev sibling parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 Please post bug reports to bugzilla.
Here you go: https://issues.dlang.org/show_bug.cgi?id=17077
Jan 08
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 12:58 PM, pineapple wrote:
 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:
 Please post bug reports to bugzilla.
Here you go: https://issues.dlang.org/show_bug.cgi?id=17077
Thank you.
Jan 08