www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 3372] New: optlink silently mistreats object files with more than 16384 symbols

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372

           Summary: optlink silently mistreats object files with more than
                    16384 symbols
           Product: D
           Version: 2.033
          Platform: Other
        OS/Version: Windows
            Status: NEW
          Severity: major
          Priority: P2
         Component: Optlink
        AssignedTo: nobody puremagic.com
        ReportedBy: r.sagitario gmx.de


--- Comment #0 from Rainer Schuetze <r.sagitario gmx.de> 2009-10-07 23:50:29
PDT ---
Larger projects (like qtd) are suffering from problems with large object files
with a lot of symbols, where the linker creates bad fixups, creating
executables that are calling wrong functions.

Here's a test case that creates a lot of symbols

//////////////////////////////////////

import std.stdio;

class genfuns(int n, int m)
{
    static void run() { 
        genfuns!(n - 1, m).run(); 
    }
}

class genfuns(int n : 0, int m)
{
    static void run() { writefln("=== 0, %d ===", m); }
}

void main()
{
    // dmd-release: stack overflow with n=700
    // dmd-debug:   stack overflow with n=500
    genfuns!(300, 0).run(); 
    genfuns!(300, 1).run();
    genfuns!(300, 2).run();
    genfuns!(300, 3).run();
    genfuns!(300, 4).run();
    genfuns!(300, 5).run();
    genfuns!(300, 6).run();
    genfuns!(300, 7).run();
    genfuns!(300, 8).run();
    genfuns!(300, 9).run();
    genfuns!(300, 10).run();
    genfuns!(300, 11).run();
}
////////////////////////////////////////////

The output looks like this:

=== 0, 0 ===
=== 0, 1 ===
=== 0, 2 ===
=== 0, 3 ===
=== 0, 4 ===
=== 0, 5 ===
=== 0, 6 ===
=== 0, 7 ===
=== 0, 8 ===
=== 0, 9 ===
=== 0, 0 ===
=== 0, 0 ===

That shows, that the two last functions are calling wrong functions. The
symbols don't even show up in the map file, but are still in the debug info.
There you can see, that the adress is plain wrong, and searching the symbol in
the object file dump, it seems that symbol #16384 + n gets the address of
symbol #n.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 07 2009
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
                 CC|                            |bugzilla digitalmars.com
         Resolution|                            |WORKSFORME


--- Comment #1 from Walter Bright <bugzilla digitalmars.com> 2010-09-01
18:57:27 PDT ---
I had to remove the compiler's recursion detection circuits to get this to
compile, but I can't get it to fail after linking with several versions of
optlink. In any case, the 8.00.6 linker that solves 4009 may resolve this one,
too, they sound similar.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 01 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372


Rainer Schuetze <r.sagitario gmx.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WORKSFORME                  |


--- Comment #2 from Rainer Schuetze <r.sagitario gmx.de> 2010-09-01 23:45:20
PDT ---
Sorry, but the error is still there with optlink 8.00.7 and dmd svn r638.

Here's an updated test case that does not need a compiler change:


import std.stdio;

class genfuns(int n, int m)
{
    static void run() { 
        genfuns!(n - 1, m).run(); 
    }
}

class genfuns(int n : 0, int m)
{
    static void run() { writefln("=== 0, %d ===", m); }
}

version = crash;

void main()
{
    // dmd-release: stack overflow with n=700
    // dmd-debug:   stack overflow with n=500
    // dmd-2.037: error "recursive expansion" for n=300
    genfuns!(200, 0).run(); 
    genfuns!(200, 1).run();
    genfuns!(200, 2).run();
    genfuns!(200, 3).run();
    genfuns!(200, 4).run();
    genfuns!(200, 5).run();
    genfuns!(200, 6).run();
    genfuns!(200, 7).run();
    genfuns!(200, 8).run();
    genfuns!(200, 9).run();
    genfuns!(200, 10).run();
    genfuns!(200, 11).run();
    genfuns!(200, 12).run();
    genfuns!(200, 13).run();
    genfuns!(200, 14).run();
    genfuns!(200, 15).run();
    genfuns!(200, 16).run();
    genfuns!(200, 17).run();
    genfuns!(200, 18).run();
    genfuns!(200, 19).run(); // 16224 symbols up to here
version(crash)
    genfuns!(200, 20).run(); // 17028 symbols with this line
}

The actual bad behaviour of the executable can vary a little due to changed
symbols in libraries and resulting different relocations. For me it is
currently a long loop repeating the expected output followed by access
violations or failed memory allocations.

If you uncomment the version=crash statement, you'll get the expected result.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 01 2010
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |FIXED


--- Comment #3 from Walter Bright <bugzilla digitalmars.com> 2011-02-22
14:24:24 PST ---
Fixed optlink 8.00.10

The limit on the number of symbols per object file should now be 0x7FFF. This
limit is hardcoded in the OMF object file format.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 22 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372


Andrei Alexandrescu <andrei metalanguage.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrei metalanguage.com


--- Comment #4 from Andrei Alexandrescu <andrei metalanguage.com> 2011-02-22
14:38:22 PST ---
We should mark this day as a federal holiday :o).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 22 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372


changlon <changlon gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |changlon gmail.com


--- Comment #5 from changlon <changlon gmail.com> 2011-02-22 18:50:03 PST ---
I use latest dmd 1.067, optlink  8.00.10 .

latest tango and dwtx ,  re builad all lib.

in on console run. link.exe it tell me the version is 8.00.10 .

use xfbuild for the dwtx lib .

the optlink throw a error like:


dwtx is big,  the lib is 30M .

http://gool.googlecode.com/files/optlink_bug.jpg

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 22 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #6 from Rainer Schuetze <r.sagitario gmx.de> 2011-02-22 23:29:12
PST ---
I can confirm this bug is fixed. 

Even if it's only the limit that has been pushed to 32768 without other
changes, you won't be able to stuff an object file with more symbols into
optlink because dmd 2.052 asserts with

Internal error: ..\ztc\cgobj.c 425

This could be a bit more friendly, though.

There might be projects that need more symbols still, so a work around is to
not compile into a single object file, but to multiple object files. You
currently have to do this by calling dmd twice, once for compilation and then
for linking as the output filename will otherwise also be used for the single
object file.

The problem described in comment 5 is probably not related to this bug, but the
crasher bugs, e.g. 2436.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 22 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #7 from Walter Bright <bugzilla digitalmars.com> 2011-02-23
02:42:31 PST ---
(In reply to comment #5)
 the optlink throw a error like:
 dwtx is big,  the lib is 30M .
 http://gool.googlecode.com/files/optlink_bug.jpg

This isn't 2436, which has been fixed. If you can make available the files that are passed to the linker, I can see about fixing it. I don't need source files. Just the obj's and lib's passed to the linker. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Feb 23 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #8 from changlon <changlon gmail.com> 2011-02-23 04:24:35 PST ---
http://gool.googlecode.com/files/optlink_bug.rar

lib with obj will crash the optlink 8.0.0.10 .

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 23 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #9 from Walter Bright <bugzilla digitalmars.com> 2011-02-23
12:12:23 PST ---
(In reply to comment #8)
 http://gool.googlecode.com/files/optlink_bug.rar
 lib with obj will crash the optlink 8.0.0.10 .

Thanks, I can reproduce it, though there is no tango.lib included. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Feb 23 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #10 from Walter Bright <bugzilla digitalmars.com> 2011-02-23
12:35:19 PST ---
(In reply to comment #6)
 Even if it's only the limit that has been pushed to 32768 without other
 changes, you won't be able to stuff an object file with more symbols into
 optlink because dmd 2.052 asserts with
 Internal error: ..\ztc\cgobj.c 425
 This could be a bit more friendly, though.

https://github.com/D-Programming-Language/dmd/commit/19f4b17e17b6dd742cd9edbdf5f6a4dccf5d4063 https://github.com/D-Programming-Language/dmd/commit/8151d5cd43348abcdae61e25bc7318e2ee6e7943 -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Feb 23 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #11 from Walter Bright <bugzilla digitalmars.com> 2011-02-24
17:14:04 PST ---
Try this one:

http://ftp.digitalmars.com/link.8.00.11.zip

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 24 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #12 from changlon <changlon gmail.com> 2011-02-25 15:25:21 PST ---
(In reply to comment #11)
 Try this one:
 
 http://ftp.digitalmars.com/link.8.00.11.zip

8.0.0.11 is working, thanks for the quick fix. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Feb 25 2011
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3372



--- Comment #13 from Walter Bright <bugzilla digitalmars.com> 2011-02-25
16:47:53 PST ---
You're welcome!

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 25 2011