www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 3274] New: dmd fails to emit code for templates into object file if several files are compiled at once

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274

           Summary: dmd fails to emit code for templates into object file
                    if several files are compiled at once
           Product: D
           Version: 1.046
          Platform: Other
        OS/Version: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: nfxjfg gmail.com


If you compile several files at once, and the files instantiate the same
template, dmd outputs the code for that template only into one object file.

This breaks incremental compilation with tools like rebuild/dsss.

ldc doesn't suffer from this bug, and outputs the template code into all
corresponding object files.

Reproduction of that bug:

First put the files listed at the end of the bug report into a.d, b.d and c.d.
Then execute these steps (order of files is important):
1. dmd a.d b.d c.d -c -version=XX
2. dmd a.d -c
   (without version XX -> like a change to the source code; because b.d and c.d
don't depend from a.d, you shouldn't need to recompile it.)
3. dmd a.o b.o c.o

Step 3 gives the error:
b.o: In function `_D1b1xFZv':
b.d:(.text._D1b1xFZv+0x4): undefined reference to
`_D1c13__T6from_cTiZ6from_cFZv'
collect2: ld returned 1 exit status
--- errorlevel 1

If Step 2. is left out, the error doesn't happen. If you compile all files
separately in Step 1, the error doesn't happen.

-- file a.d:
module a;

import b;
import c;

void main() {
    version (XX) {
        from_c!(int)();
    }
}

-- file b.d:
module b;

import c;

void x() {
    from_c!(int)();
}

-- file c.d:
module c;

void from_c(T)() {
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 30 2009
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274


nfxjfg gmail.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WONTFIX


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 06 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274


Brad Roberts <braddr puremagic.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
                 CC|                            |braddr puremagic.com
         Resolution|WONTFIX                     |


-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 06 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274


Walter Bright <bugzilla digitalmars.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
                 CC|                            |bugzilla digitalmars.com
         Resolution|                            |WONTFIX



23:17:05 PDT ---
The compiler does this as a compile performance optimization. Otherwise,
template instantiations have to be done over and over, filling the object
files, and then the linker has to remove the duplicates.

The workaround for an incremental build system is straightforward - put only
one source module on a command to dmd at a time.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 04 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274


jfanatiker gmx.at changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jfanatiker gmx.at



It seems to work correctly with DMD v2.062 at least on Fedora 64 bit.

Can anyone confirm this?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 05 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code dawg.eu





 The compiler does this as a compile performance optimization. Otherwise,
 template instantiations have to be done over and over, filling the object
 files, and then the linker has to remove the duplicates.
 


It remains an incorrect optimization that just works most of the time.


 The workaround for an incremental build system is straightforward - put only
 one source module on a command to dmd at a time.


The performance impact of this is also huge. The only reason to do this in the
first place is to avoid duplicate parsing and semantic. However we can't make
any assumptions how the generated object files are linked therefor each of them
needs to get the required TypeInfo, templates et.al.

To put this into perspective, copying is only required when building multiple
object files. It is not required when building an executable, a shared library
or a single object (which may contain multiple modules, i.e. -c -of).

Maybe we can move the copying more to the backend so that we don't have to
rerun IRgen and codegen.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 07 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274




One idea to reduce the cost of copying is partial linking.
AFAIU that would allow us to generate a single temporary object containing the
template/TypeInfo instantiation and link it into every object file that
references it.

ld -r -o combined1.o template.o module1.o
mv combined1.o module1.o
ld -r -o combined2.o template.o module2.o
mv combined2.o module2.o

http://sourceware.org/binutils/docs-2.23.1/ld/Options.html#index-partial-link-88

Though I'm not sure how portable this or an equivalent dmd backend
functionality would be.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 07 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=3274


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code klickverbot.at



*** Issue 8769 has been marked as a duplicate of this issue. ***

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Apr 07 2013