www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 9571] New: linkage issue when using separate compilation and closures

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571

           Summary: linkage issue when using separate compilation and
                    closures
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: deadalnix gmail.com



a.d :

import std.array;

void main() {
    import b;
    foo(new A);
}

b.d :
module b;

class C {
}

class A {
    C[] cs;
}

import std.algorithm;
auto foo(A a) {
    auto b = new B;
    a.cs.map!((C c){ return b.visit(c); });
}

class B {
    C visit(C c) {
        return c;
    }
}

compilation :
dmd -op -odobj/a -c src/a.d -m64
dmd -op -odobj/b -c src/b.d -m64

dmd -ofbin/sdc `find obj/a obj/b -type f | grep '\.o$'` -m64

Error :
obj/a/src/a.o: In function
`_D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1196MFC1b1CZC1b1CTAC1b1CZ9MapResult4backMFNdZC1b1C':
src/a.d:(.text._D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1196MFC1b1CZC1b1CTAC1b1CZ9MapResult4backMFNdZC1b1C+0x64):
undefined reference to `_D1b3fooFC1b1AZv12__lambda1196MFC1b1CZC1b1C'
obj/a/src/a.o: In function
`_D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1196MFC1b1CZC1b1CTAC1b1CZ9MapResult5frontMFNdZC1b1C':
src/a.d:(.text._D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1196MFC1b1CZC1b1CTAC1b1CZ9MapResult5frontMFNdZC1b1C+0x64):
undefined reference to `_D1b3fooFC1b1AZv12__lambda1196MFC1b1CZC1b1C'
obj/a/src/a.o: In function
`_D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1196MFC1b1CZC1b1CTAC1b1CZ9MapResult7opIndexMFmZC1b1C':
src/a.d:(.text._D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1196MFC1b1CZC1b1CTAC1b1CZ9MapResult7opIndexMFmZC1b1C+0x77):
undefined reference to `_D1b3fooFC1b1AZv12__lambda1196MFC1b1CZC1b1C'

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 22 2013
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Denis Shelomovskij <verylonglogin.reg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |link-failure
                 CC|                            |verylonglogin.reg gmail.com
           Severity|normal                      |major


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 23 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Vladimir Panteleev <thecybershadow gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |thecybershadow gmail.com



17:16:46 EET ---
I'm seeing a similar error with Win64:

a.obj : error LNK2019: unresolved external symbol
_D1b3fooFC1b1AZv12__lambda1404MFC1b1CZC1b1C referenced in function
_D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1404MFC1b1CZC1b1CTAC1b1CZ9MapResult4backMFNdZC1b1C
a.exe : fatal error LNK1120: 1 unresolved externals
--- errorlevel 1120

Links fine on Win32, though.

Mr. deadalnix, does it happen without -m64? If not, please indicate the correct
platform, since it is currently set to "All".

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571






 I'm seeing a similar error with Win64:
 
 a.obj : error LNK2019: unresolved external symbol
 _D1b3fooFC1b1AZv12__lambda1404MFC1b1CZC1b1C referenced in function
 _D1b3fooFC1b1AZv59__T9MapResultS35_D1b3foo12__lambda1404MFC1b1CZC1b1CTAC1b1CZ9MapResult4backMFNdZC1b1C
 a.exe : fatal error LNK1120: 1 unresolved externals
 --- errorlevel 1120
 
 Links fine on Win32, though.
 
 Mr. deadalnix, does it happen without -m64? If not, please indicate the correct
 platform, since it is currently set to "All".


Mr. CyberShadow, I confirm the bug on 32bits as well.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Mar 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Dicebot <m.strashun gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |m.strashun gmail.com



Want to note that it works if "auto foo" is changed to "void foo", so that it
may be also somehow related to templates, exact conditions are weird though.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 02 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571




https://github.com/D-Programming-Language/dmd/pull/1882

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code dawg.eu



Could you please try to reduce the test case.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571




I was not able to reduce it any further than moving A variable from parameter
to foo itself, leaving foo signature as void foo(). Almost any change in b
module resulted in lambda marked as a weak symbol and compilation succeeding.

I have not tried to track this down really hard though as my pull solves more
fundamental issue - that symbol should even be there at all, as well as several
hundred phobos symbols. Still need to track down and fix special cases found by
test suite though. Working on it right now.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|linkage issue when using    |link error due to using
                   |separate compilation and    |unique ids in anonymous
                   |closures                    |funcliteral



cat > a.d << CODE
import b;

void main()
{
    foo();
}
CODE

cat > b.d << CODE
struct MapResult(alias fun)
{
    void bar()
    {
        fun(0);
    }
}

auto foo()
{
    version (all)
    {
        alias MapResult!(function(int c) => 0) M;
    }
    else
    {
        // works because it uses a named function literal
        auto func = function(int c) => 0;
        alias MapResult!(func) M;
    }
}
CODE

dmd -c a
dmd -c b
dmd a.o b.o
--------

During the compilation of a.d semantic3 is run on auto foo() and thus
an instantiation of MapResult!(...) is added to a.o.
The problem is that the anonymous funcliteral in a.o and the one in b.o used a
different unique id suffix.
Because semantic3 on MapResult.bar() is not run during a's compilation the
funcliteral is never referenced and missing from a.o.

nm a.o | grep funcliteral
U _D1b3fooFZv14__funcliteral5FNaNbNfiZi
nm b.o | grep funcliteral
T _D1b3fooFZv14__funcliteral1FNaNbNfiZi

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571






 Because semantic3 on MapResult.bar() is not run during a's compilation the
 funcliteral is never referenced and missing from a.o.
 


That was incorrect semantic3 for the function literal and MapResult.bar is run
and bar is added as a weak symbol to a.o which actually causes the link error
because the funcliteral remains undefined.

We need to fix the unique id issue for anonymous literals.
We shouldn't emit an unused template instance to a.o just because we ran
semantic3 on foo, note that foo returns void.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 10 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Dicebot <m.strashun gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|pull                        |



Thanks for tracking down the root cause for this specific issue.
I'll move my pull request to its own issue then.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 11 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571




The root cause of this issues got hidden / fixed by
https://github.com/D-Programming-Language/dmd/pull/2550 (as only one template
instance is emitted and thus only one literal ID gets generated).

I can't imagine any other way to manifest this issue (without using template
instances) but it may be theoretically possible.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 13 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571




---


 The root cause of this issues got hidden / fixed by
 https://github.com/D-Programming-Language/dmd/pull/2550 (as only one template
 instance is emitted and thus only one literal ID gets generated).
 
 I can't imagine any other way to manifest this issue (without using template
 instances) but it may be theoretically possible.


Able to cause the issue by combining -inline.

//a.d
import b;
void main()
{
    foo();
}

// b.d
struct MapResult(alias fun)
{
    void bar() { fun(0); }
}
auto foo()
{
    alias MapResult!(function(int c) => 0) M;
    M m;
    m.bar();
}

Command:
$ dmd -inline -c a.d
$ dmd -inline -c b.d
$ dmd -oftest.exe a.obj b.obj
OPTLINK (R) for Win32  Release 8.00.12
Copyright (C) Digital Mars 1989-2010  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
a.obj(a)
 Error 42: Symbol Undefined
_D1b51__T9MapResultS34_D1b3foo14__funcliteral4FNaNbNfiZiZ9MapResult3barMFNaNbNfZv
--- errorlevel 1

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 17 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull



---
https://github.com/D-Programming-Language/dmd/pull/2566

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 17 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571




Commits pushed to master at https://github.com/D-Programming-Language/dmd

https://github.com/D-Programming-Language/dmd/commit/225188755132e42ae97ca398be7d7f3af2b5a1e2
fix Issue 9571 - link error due to using unique ids in anonymous funcliteral

https://github.com/D-Programming-Language/dmd/commit/3395a17dfe506c7a0dcf7370e7e4de4d98391528


Issue 9571 - link error due to using unique ids in anonymous funcliteral

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 17 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571


Kenji Hara <k.hara.pg gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 19 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9571






 https://github.com/D-Programming-Language/dmd/pull/2566


Dude, I'm not in Japan for so long, but if you have one moment on Tokyo next
week, I'd buy you a beer (or whatever you prefer).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 19 2013