www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 424] New: Unexpected OPTLINK Termination at EIP=0044C37B

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424

           Summary: Unexpected OPTLINK Termination at EIP=0044C37B
           Product: D
           Version: unspecified
          Platform: PC
               URL: http://www.digitalmars.com/d/archives/digitalmars/D/bugs
                    /306.html
        OS/Version: Windows
            Status: NEW
          Severity: blocker
          Priority: P1
         Component: DMD
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: someanon yahoo.com


I have some generated D code in a single source file ~ 30K LOC. On Windows I
can compile it but cannot link it. (On Linux, there's no problem.)

Some web search leads me to the following, which seems related to the problem. 
I wonder if OPTLINK can be fixed now.

http://www.digitalmars.com/d/archives/digitalmars/D/bugs/306.html

Sun, 30 May 2004 20:54:18 -0700 "Walter" <newshound xx digitalmars.com> 
writes:

This happens when there are more than 16,000 fixups in a single .obj.
Unfortunately, this isn't fixable at the moment.


http://www.digitalmars.com/d/archives/digitalmars/D/bugs/8283.html
http://www.digitalmars.com/d/archives/digitalmars/D/bugs/306.html
http://www.digitalmars.com/d/archives/digitalmars/D/41323.html


-- 
Oct 10 2006
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424


bugzilla digitalmars.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|blocker                     |normal




------- Comment #1 from bugzilla digitalmars.com  2006-10-14 23:17 -------
While important, I don't think it's a blocker. 30,000 lines of code in one
module is unusually large, and can be split into smaller source files.


-- 
Oct 14 2006
prev sibling next sibling parent reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #2 from someanon yahoo.com  2006-10-15 22:07 -------
(In reply to comment #1)
 While important, I don't think it's a blocker. 30,000 lines of code in one
 module is unusually large, and can be split into smaller source files.
 

It is a blocker at least for one user :-) OK, let me elaborate a little more where I come from, and all the problems I've encountered: When I started to use D on a hobby project, I organize the source code into modules as everyone typically does, but because of bug 386: http://d.puremagic.com/issues/show_bug.cgi?id=386 I was so frustrated that I cannot get my code compile separately, and the compiler error message is useless, each time I spent hours and hours on them and cannot get them fixed. So I write some script to concat all my source files (togather with some generated ones) into a single huge .d file, and comment out all the 'module' and 'import' statements. Then everything works. After bug 386 is fixed. I tried to compile each source file separately again, now I no longer have 'import conflicts' problems, but I still cannot get many of the file compiled, the compiler just hangs: I guess it run into some infinite loop. But unfortunately I cannot isolate the issue to send you a test case. (search "infinite loop" on digitalmars.com will show some user report: e.g. http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=42668 But I'm not sure, it's the same issue as mine.) Then I went back to my original approach to generate a huge source file (and it works); but maybe as my project grows, at certain point, it stops to link on Windows. (On Linux, it's fine). And this is a OPTLINK problem. I hope you can fix it. Or as a workaround: if the generate obj is too large (> 16,000 fixups, I really don't understand what does it mean), can the compiler split the obj into several obj files? e.g. foo1.obj, foo2.obj, ... If you ask the user the split the generated source files, first it's tedious, and it maybe run into some compiler problem, like the import one again. So please fix this issue: 1) remove the (> 16,000 fixups) constraint. It is a bug anyway. 2) or, the compiler automatically generate multiple .obj files, if (> 16,000 fixups) Thanks. --
Oct 15 2006
parent Walter Bright <newshound digitalmars.com> writes:
d-bugmail puremagic.com wrote:
 After bug 386 is fixed. I tried to compile each source file separately again,
 now I no longer have 'import conflicts' problems, but I still cannot get many
 of the file compiled, the compiler just hangs: I guess it run into some
 infinite loop.  But unfortunately I cannot isolate the issue to send you a test
 case.

Here's how to do it. Make a copy of your project. Create a .bat file that compiles it. Verify that it hangs. Now, keep deleting code until you have the minimum that produces the hang. I've never seen one yet that couldn't be isolated down this way, including from people who swore it couldn't be done <g>. Don't worry about intelligently deciding what lines to delete. Delete something, even if it's just one line.
Oct 15 2006
prev sibling next sibling parent reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #3 from someanon yahoo.com  2006-10-15 22:09 -------
Another "infinite loop" user report, maybe related:

http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=D.gnu&artnum=2134


-- 
Oct 15 2006
parent Walter Bright <newshound digitalmars.com> writes:
d-bugmail puremagic.com wrote:
 ------- Comment #3 from someanon yahoo.com  2006-10-15 22:09 -------
 Another "infinite loop" user report, maybe related:
 
 http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=D.gnu&artnum=2134

Doesn't infinite loop on current dmd.
Oct 15 2006
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #4 from braddr puremagic.com  2006-10-16 02:11 -------
Let's not clutter this report up with more than one issue.  Please make sure a
seprate bug is filed for the infinite loop issue.  This report is strictly
about th eoptlink fixup limit.

Walter, can you please clarify where the limit exists and if it's authored by
you or microsoft?  I don't do any windows development so I'm unfamiliar with
the pieces.

Filer, would you please attach to this report an example .d file that fails
with the error message?

Rather than discussing why or how this programming practice might be ill
advised, can we talk about where the problem is and how or if it can be fixed. 
Each barrier to the use of D, whatever the reason, is a barrier that should at
least be addressed in a FAQ and not dismissed as a bad practice.

Thanks,
Brad


-- 
Oct 16 2006
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #5 from someanon yahoo.com  2006-11-07 21:09 -------
any chance this will be fixed soon? it's a bug anyways.


-- 
Nov 07 2006
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #6 from someanon yahoo.com  2006-12-27 12:48 -------
can this one be fixed before 1.0 (Jan 01 2007)?


-- 
Dec 27 2006
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #7 from someanon yahoo.com  2007-03-22 22:32 -------
No update?


-- 
Mar 22 2007
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #8 from someanon yahoo.com  2007-04-06 21:28 -------
Tried the new linker:

http://ftp.digitalmars.com/Digital_Mars_C++/Patch/beta.zip 

In release mode (-d -O -inline -release), it works, and generate the exe file.
But when running the program (using wxd, 0.09 version), the main UI window
doesn't show up.

In debug mode (-d -g -gc -debug), it errors out:
Unexpected OPTLINK Termination at EIP=00412793

GDC (under MingW32) works in both debug & release mode on the same system.


-- 
Apr 06 2007
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #9 from davidl 126.com  2007-09-12 08:20 -------
I think if you can post your object file which makes optlink fails could let
Walter easier to debug this issue


-- 
Sep 12 2007
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #10 from wbaxter gmail.com  2007-11-14 18:55 -------
(In reply to comment #1)
 While important, I don't think it's a blocker. 30,000 lines of code in one
 module is unusually large, and can be split into smaller source files.

I believe I'm running into this too, and my files have a reasonable number of lines. However, the code is structured so that *everything* is templated, and doesn't get instantiated until main.d. This is a very common way to write heavily generic code. The actual library in question in which this is happening OpenMeshD (http://www.dsource.org/projects/openmeshd), which is a port of a C++ template library to D. Why I think I'm seeing this same problem: 1) Optlink is dying and popping up a MsgBox saying "Unexpected OPTLINK Termination at EIP=0044C37B" The EIP number is different that the one previously reported, but I'm guessing that doesn't matter (just a random memory address?). 2) The thing that triggered the crashing was adding a function with a big block of type-checking case statements like this: else if (ti==typeid(VectorT!(byte, 2))) { renderer_ = new PropRenderer!(VectorT!(byte, 2)); } So it's causing a lot of templates to be instantiated, which could be pushing the number of "fixups" in main.d over the 16K limit. 3) If I start commenting out those "else if" lines one by one, eventually compilation will work again. It doesn't seem to matter which lines I comment out. If I get rid of enough of them, then it compiles. 4) Compiling with -O -release instead of -g -debug allows me to uncomment a few more of the lines. 5) "main.obj" is between 1-2MB even in release mode. All that seems to add up to an OPTLINK upper limit bug. If not with # of fixups then with something. I'm using DMD 1.023 on Windows XP. I've checked the code into the OpenMesh/D repository in its buggy state so it can be checked out for debugging. To reproduce, check out revision 111 of OpenMesh from the svn repository: If you don't have dsss installed, install it first (http://www.dsource.org/projects/dsss) Then: $ mkdir scratch $ cd scratch $ svn co -r111 http://svn.dsource.org/projects/openmeshd/trunk . $ cd OpenMeshD\OpenMesh\Apps\GLViewer $ dsss net deps $ dsss build -version=OPTLINKIsMyFriend If there are complaints about Helix things, you may need to go to the Helix dir and do a "dsss build && dsss install" --
Nov 14 2007
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424


wbaxter gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |major




------- Comment #11 from wbaxter gmail.com  2007-11-14 23:37 -------
Upgrading the severity because I just ran into this again adding to an
unrelated part of the main .d file.  Moving the added code to a different file
fixed it for now but that may not always be so easy to do if the code needs to
be in the generic, templated part of things.  Then it seems it will end up
instantiating in the main .obj file no matter where I put it.

Also further rearrangement of the code started triggering the "previous
definition different blahblah_initZ" bug.

So, yeh, this is a bad one, and it affects more than just code with gigantic
source files.   It appears to affect any template library with a few tens of
thousands of lines of code.


-- 
Nov 14 2007
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #12 from wbaxter gmail.com  2008-03-04 21:25 -------
Fun fun!  Just hit this one again.

See thread digitalmars.com digitalmars.D:67093


-- 
Mar 04 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424


wbaxter gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Unexpected OPTLINK          |Unexpected OPTLINK
                   |Termination at EIP=0044C37B |Termination at EIP=0044C37B
                   |                            |(too many fixups)




------- Comment #13 from wbaxter gmail.com  2008-06-26 18:24 -------
Changed title to include keyword "fixups" for easy finding.


-- 
Jun 26 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #14 from samukha voliacable.com  2008-11-13 00:54 -------
Probably a duplicate of #2436 or/and #1439


-- 
Nov 12 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #15 from wbaxter gmail.com  2008-11-13 02:00 -------
(In reply to comment #14)
 Probably a duplicate of #2436 or/and #1439
 

Er, I think you mean #2436 and/or #1439 are probably duplicates of this one. --
Nov 13 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #16 from samukha voliacable.com  2008-11-13 03:05 -------
Exactly


-- 
Nov 13 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #17 from jarrett.billingsley gmail.com  2008-11-20 18:26 -------
Just got bitten by this one too.  Voted!


-- 
Nov 20 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #18 from wbaxter gmail.com  2008-11-20 18:53 -------
Are you (In reply to comment #17)
 Just got bitten by this one too.  Voted!
 

Unfortunately I think the only real solution for this is a new linker. Just out of curiosity, in your case is it the lots of templates getting instantiated in a single file thing? Or did you find some other way to trigger it? --
Nov 20 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #19 from jarrett.billingsley gmail.com  2008-11-20 19:23 -------
(In reply to comment #18)
 Are you (In reply to comment #17)
 Just got bitten by this one too.  Voted!
 

Unfortunately I think the only real solution for this is a new linker. Just out of curiosity, in your case is it the lots of templates getting instantiated in a single file thing? Or did you find some other way to trigger it?

I'm pretty sure that's what's causing the issue I have. It compiles with n template instantiations, and fails with n + 1. Doesn't matter which ones I remove or add. And I have a *lot* of them in one file. And they're the deeply nested variadic code-generating kind. I do horrible things. --
Nov 20 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #20 from someanon yahoo.com  2009-02-10 17:52 -------
 Unfortunately I think the only real solution for this is a new linker.

I just thought there maybe a simple solution to implement: add a switch to the DMD compiler, e.g "-mo", means output multiple object files. So for file foo.d the compiler will output foo1.o foo2.o foo3.o .... each only contains a limited number of fixups, so in this way, the user can add all these object files to the linker command line. This should work. What do you think? Walter? --
Feb 10 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #21 from someanon yahoo.com  2009-02-10 18:03 -------
Actually even without such switch, the compiler should split the object files "
when there are more than 16,000 fixups in a single .obj" -- after all why the
compiler should generate such obj file, if the linker cannot link it, while the
compiler have all the necessary info to take the right action. (I would
consider this is a minor compiler bug in this sense ;-)

And if this happened with no user option specified, the compiler should also
print a friendly message, saying: "too many fixups, multiple object file
foo1.obj, foo2.obj ... fooN.obj generated, please modify your link command".


(In reply to comment #20)
 Unfortunately I think the only real solution for this is a new linker.

I just thought there maybe a simple solution to implement: add a switch to the DMD compiler, e.g "-mo", means output multiple object files. So for file foo.d the compiler will output foo1.o foo2.o foo3.o .... each only contains a limited number of fixups, so in this way, the user can add all these object files to the linker command line. This should work. What do you think? Walter?

--
Feb 10 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #22 from 2korden gmail.com  2009-02-10 18:11 -------
I disagree about multiple output obj files from one source. This would make
linking much harder (e.g. - compilation using makefile). I wouldn't like the
build process to depend from some third party tool that would parse compiler
output and adjust my linking options.


-- 
Feb 10 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #23 from someanon yahoo.com  2009-02-10 18:18 -------
(In reply to comment #22)
 I disagree about multiple output obj files from one source. This would make
 linking much harder (e.g. - compilation using makefile). I wouldn't like the
 build process to depend from some third party tool that would parse compiler
 output and adjust my linking options.
 

Then have a compiler switch then "-mo+" means on, and "-mo-" turn it off. I just wonder if you have ever bitten by this bug, what else you can do? The user have control of his own build process, but how the user overcome this linker bug? --
Feb 10 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #24 from bugzilla digitalmars.com  2009-02-16 22:31 -------
The compiler already has a switch to generate multiple obj files from one
source file: -multiobj. I use it for debugging. But it would be a royal pain to
use if there's a lot.

A better way is to compile to a library with the -lib switch. That splits the
module into a zillion obj files, and stuffs them all into the library. Then,
link with the library (the main() program will still have to be separate).


-- 
Feb 16 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #25 from someanon yahoo.com  2009-02-17 16:59 -------
(In reply to comment #24)
 The compiler already has a switch to generate multiple obj files from one
 source file: -multiobj. I use it for debugging. But it would be a royal pain to
 use if there's a lot.

Yes, tried that: because there are sequence dependency on the order of the object file, one cannot give "foo*.obj", the linker complains lots of unresolved symbol. I just wonder if the linker can be changed to be ".obj" order insensitive. Yes, it's a royal pain to let the user try to figure out the correct ".obj" sequence, but the linker should have all the information to find the unresolved symbols in those .objs.
 
 A better way is to compile to a library with the -lib switch. That splits the
 module into a zillion obj files, and stuffs them all into the library. Then,
 link with the library (the main() program will still have to be separate).
 

Also tried, when linking with the generated .lib file, it also gives "Unexpected OPTLINK Termination" error, not sure it's the same one. But I guess so. --
Feb 17 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





------- Comment #26 from bugzilla digitalmars.com  2009-03-07 14:28 -------
If someone can attach to this an object file with too many fixups, that would
be helpful. Or email it to me.


-- 
Mar 07 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424


Jarrett Billingsley <jarrett.billingsley gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jarrett.billingsley gmail.c
                   |                            |om




--- Comment #27 from Jarrett Billingsley <jarrett.billingsley gmail.com> 
2009-06-20 23:54:07 PDT ---
Open a windows command prompt and run the following line:

for /L %i in (1,1,17000) do  echo void func%i(){} >> test.txt

This creates test.txt with 17,000 functions.

Then create test.d in the same dir:

module test;
mixin(import("test.txt"));
void main() {}

Now, compile *with the following commandline*.

dmd test.d -g -J.

This causes OPTLINK v8.00.1 to crash with EIP=00412793 as sa mentioned.  

For more testing, delete test.txt and run the shell line above with different
values.  16000 does not crash.  Interestingly, 16500 neither succeeds nor
fails; the linker apparently hangs and does not use any CPU.

If you want a real-world test, you're going to have to install some third-party
D libraries.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jun 20 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424





--- Comment #28 from Jarrett Billingsley <jarrett.billingsley gmail.com> 
2009-07-27 08:42:22 PDT ---
I know you responded with a "thanks" on the newsgroup, Walter, but.. is there
any followup?  Were you able to reproduce it?  Have you investigated the
problem?  Is there any chance of it being fixed?  Is it a fundamental limit
that can't be changed?  Is this on the back burner?  Is this going to magically
appear in the next release without warning?

Cmooon, communicate!  What is the story?  There are a lot of people who've run
into this bug, a lot who want to see it fixed, and a simple repro.  What more
do you want or need?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 27 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424



--- Comment #29 from Walter Bright <bugzilla digitalmars.com> 2009-11-03
10:38:43 PST ---
Optlink is written entirely in rather impenetrable assembler code, and is
resistant to understanding and modification. Hence, over the last few months
I've been very slowly converting it to C, function by function.

One might ask, why not convert it to D? The answer is that I don't have a good
test suite for optlink, so I have to be very very careful to not make a mistake
in the translation. That means do one function at a time, rebuild, and retest,
which means the compiled C code has to match the segment, naming and calling
conventions used in optlink. I made a custom version of the dmc compiler to do
this. Also, C can be made to work without any runtime library support at all,
and since optlink does not use the C runtime library, this is useful.

Once it is in C and working, it will be trivial to translate it to D and start
rewriting it.

Anyhow, during this process I stumbled upon what the problem was. Optlink was
apparently trying to account for some Borland obscure extension to the OMF.
Remove this, and it works, although presumably it will no longer link Borland
object files (who cares!).

The fix will go out in the next update, if you need it sooner please email me.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 03 2009
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424



--- Comment #30 from Jarrett Billingsley <jarrett.billingsley gmail.com>
2009-11-05 10:03:36 PST ---
(In reply to comment #29)

 The fix will go out in the next update, if you need it sooner please email me.

Holy-- Joy upon joys! This is REALLY good news. I can't tell you how happy you'll make a lot of people with this. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Nov 05 2009
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=424


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


--- Comment #31 from Walter Bright <bugzilla digitalmars.com> 2009-11-06
11:29:04 PST ---
Fixed dmd 1.051 and 2.036

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Nov 06 2009