www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - duplicate symbol linker errors, my fault or D's?

reply "Zach the Mystic" <reachMINUSTHISzachgmail dot.com> writes:
I'm not sure if the linker errors I'm getting are my fault or D's.

I'm trying to do incremental compilation since the size of my
program is starting to break my computer, slowing compilations
down by four or five times.

Suppose I have two libraries:
lib1.a
lib2.a

I build lib1 with a bunch of files. Then I build lib2 by linking
with lib1:

dmd -lib -oflib2.a ... file1.d file2.d lib1.a

Now file1 and file2 actually import a lot of the files built into
lib1, but neither is in lib1.

When I try to compile file3.d:

dmd file3.d -L-llib2
ld: duplicate symbol
_D53/Users/zach/source/dmd2/src/phobos/std/array.d.237__arrayZ in
./lib/lib2.a(array_17_e15.o) and ./lib/lib2.a(array_17_e82.o)
collect2: ld returned 1 exit status

I fear I'm not going to be able to use incremental compilation
because D is duplicating some symbols I can't track because I'm
not declaring them myself.

There's a lot of stuff I don't know about programming. Therefore
I hope I'm making a beginner's blunder instead.

What's your opinion?

Zach
Mar 05 2012
next sibling parent reply "Zach the Mystic" <reachMINUSTHISzachgmail dot.com> writes:
Reading the documentation about compiler options and flags here:
http://dlang.org/dmd-osx.html

led me to believe that building libraries was the right way to do 
incremental compilation. But I thought, well, can I just build 
file1.o object files instead?

So I've started doing that and I've got no problems so far. I 
guess I don't need all that fancy .lib and -L-lmyLib stuff after 
all. You can pack all sorts of stuff into a file1.o file instead.

I guess I solved my problem for now. Hopefully I'm making sense. 
:-)
Mar 05 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-03-06 02:21, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 01:53:02 Zach the Mystic wrote:
 Reading the documentation about compiler options and flags here:
 http://dlang.org/dmd-osx.html

 led me to believe that building libraries was the right way to do
 incremental compilation. But I thought, well, can I just build
 file1.o object files instead?

 So I've started doing that and I've got no problems so far. I
 guess I don't need all that fancy .lib and -L-lmyLib stuff after
 all. You can pack all sorts of stuff into a file1.o file instead.

 I guess I solved my problem for now. Hopefully I'm making sense.

Libraries are not intented for incremental compilation. They are for distributing code in a unit which can be used by programs. And in the case of a shared library, it gives the added benefit of reducing the amount of duplicate code you get in binaries (saving both memory and disk space). If you want to do incremental compilation, then use -c to generate object files that you link together when you create the actual executable. - Jonathan M Davis

Actually the -lib switch might be the answer to one of the incremental compilation problems DMD suffers from. That DMD usually does not output all symbols to all object files which can result in missing symbols when doing incremental compilation. I've heard that the when using the -lib flag DMD will output all symbols to all object files. -- /Jacob Carlborg
Mar 05 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-03-06 08:56, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 08:29:53 Jacob Carlborg wrote:
 On 2012-03-06 02:21, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 01:53:02 Zach the Mystic wrote:
 Reading the documentation about compiler options and flags here:
 http://dlang.org/dmd-osx.html

 led me to believe that building libraries was the right way to do
 incremental compilation. But I thought, well, can I just build
 file1.o object files instead?

 So I've started doing that and I've got no problems so far. I
 guess I don't need all that fancy .lib and -L-lmyLib stuff after
 all. You can pack all sorts of stuff into a file1.o file instead.

 I guess I solved my problem for now. Hopefully I'm making sense.

Libraries are not intented for incremental compilation. They are for distributing code in a unit which can be used by programs. And in the case of a shared library, it gives the added benefit of reducing the amount of duplicate code you get in binaries (saving both memory and disk space). If you want to do incremental compilation, then use -c to generate object files that you link together when you create the actual executable. - Jonathan M Davis

Actually the -lib switch might be the answer to one of the incremental compilation problems DMD suffers from. That DMD usually does not output all symbols to all object files which can result in missing symbols when doing incremental compilation. I've heard that the when using the -lib flag DMD will output all symbols to all object files.

If so, it's an implementation issue. In principle, libraries have nothing to do with incremental compilation. They're about modularizing code for reusability. - Jonathan M Davis

Yes, but Walter doesn't seem to want to fix that issue. -- /Jacob Carlborg
Mar 06 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-03-07 01:47, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 11:31:40 Jacob Carlborg wrote:
 On 2012-03-06 08:56, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 08:29:53 Jacob Carlborg wrote:
 On 2012-03-06 02:21, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 01:53:02 Zach the Mystic wrote:
 Reading the documentation about compiler options and flags here:
 http://dlang.org/dmd-osx.html

 led me to believe that building libraries was the right way to do
 incremental compilation. But I thought, well, can I just build
 file1.o object files instead?

 So I've started doing that and I've got no problems so far. I
 guess I don't need all that fancy .lib and -L-lmyLib stuff after
 all. You can pack all sorts of stuff into a file1.o file instead.

 I guess I solved my problem for now. Hopefully I'm making sense.

Libraries are not intented for incremental compilation. They are for distributing code in a unit which can be used by programs. And in the case of a shared library, it gives the added benefit of reducing the amount of duplicate code you get in binaries (saving both memory and disk space). If you want to do incremental compilation, then use -c to generate object files that you link together when you create the actual executable. - Jonathan M Davis

Actually the -lib switch might be the answer to one of the incremental compilation problems DMD suffers from. That DMD usually does not output all symbols to all object files which can result in missing symbols when doing incremental compilation. I've heard that the when using the -lib flag DMD will output all symbols to all object files.

If so, it's an implementation issue. In principle, libraries have nothing to do with incremental compilation. They're about modularizing code for reusability. - Jonathan M Davis

Yes, but Walter doesn't seem to want to fix that issue.

Well, from the sounds of it, he's going to need to be convinced one way or another - which can be frustratingly difficult at times. Sometimes, he's right even when pretty much no one agrees with him, but other times, he just doesn't seem to get it. He's hard to convince in either case though. - Jonathan M Davis

If I recall correctly, he mentioned some issues with COMDAT. Here is the full discussion: http://www.digitalmars.com/d/archives/digitalmars/D/Incremental_compilation_with_DMD_96138.html Mentions something about COMDAT: http://www.digitalmars.com/d/archives/digitalmars/D/Incremental_compilation_with_DMD_96138.html#N96197 -- /Jacob Carlborg
Mar 06 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, March 06, 2012 01:53:02 Zach the Mystic wrote:
 Reading the documentation about compiler options and flags here:
 http://dlang.org/dmd-osx.html
 
 led me to believe that building libraries was the right way to do
 incremental compilation. But I thought, well, can I just build
 file1.o object files instead?
 
 So I've started doing that and I've got no problems so far. I
 guess I don't need all that fancy .lib and -L-lmyLib stuff after
 all. You can pack all sorts of stuff into a file1.o file instead.
 
 I guess I solved my problem for now. Hopefully I'm making sense.

Libraries are not intented for incremental compilation. They are for distributing code in a unit which can be used by programs. And in the case of a shared library, it gives the added benefit of reducing the amount of duplicate code you get in binaries (saving both memory and disk space). If you want to do incremental compilation, then use -c to generate object files that you link together when you create the actual executable. - Jonathan M Davis
Mar 05 2012
prev sibling next sibling parent "Zach the Mystic" <reachMINUSTHISzachgmail dot.com> writes:
 Libraries are not intented for incremental compilation. They 
 are for
 distributing code in a unit which can be used by programs. And 
 in the case of
 a shared library, it gives the added benefit of reducing the 
 amount of
 duplicate code you get in binaries (saving both memory and disk 
 space).

 If you want to do incremental compilation, then use -c to 
 generate object files
 that you link together when you create the actual executable.

 - Jonathan M Davis

Thanks for taking the time to answer. I simply didn't realize that you could pack like 60 modules in to one ".o" object file. But I do now!
Mar 05 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, March 06, 2012 05:26:34 Zach the Mystic wrote:
 Libraries are not intented for incremental compilation. They
 are for
 distributing code in a unit which can be used by programs. And
 in the case of
 a shared library, it gives the added benefit of reducing the
 amount of
 duplicate code you get in binaries (saving both memory and disk
 space).
 
 If you want to do incremental compilation, then use -c to
 generate object files
 that you link together when you create the actual executable.
 
 - Jonathan M Davis

Thanks for taking the time to answer. I simply didn't realize that you could pack like 60 modules in to one ".o" object file. But I do now!

You could put your entire program in a .o file if you wanted to, though I don't know why you would, since if you're generating only one object file, you might as well generate an executable and save yourself the extra linking step. I don't believe that there's any limit to what you can put in an object file beyond what you can put in an executable. http://en.wikipedia.org/wiki/Object_file - Jonathan M Davis
Mar 05 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-03-06 01:04, Zach the Mystic wrote:
 I'm not sure if the linker errors I'm getting are my fault or D's.

 I'm trying to do incremental compilation since the size of my
 program is starting to break my computer, slowing compilations
 down by four or five times.

I just want to give you a warning about incremental compilation. Currently it doesn't work out that well in D due to various reasons. You can search the newsgroups archive for extended posts about this. But to mention some of the reasons: DMD doesn't output all symbols to all object files resulting in missing symbols when doing incremental compilation. I think it's mostly template symbols that cause problems. I've heard that using the -lib flag, DMD will output all symbols to all object files. Since header/import files are rarely used in D you often need to recompile quite many files even if you only do localized changes, specially compared to C/C++ using header files. -- /Jacob Carlborg
Mar 05 2012
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, March 06, 2012 08:29:53 Jacob Carlborg wrote:
 On 2012-03-06 02:21, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 01:53:02 Zach the Mystic wrote:
 Reading the documentation about compiler options and flags here:
 http://dlang.org/dmd-osx.html
 
 led me to believe that building libraries was the right way to do
 incremental compilation. But I thought, well, can I just build
 file1.o object files instead?
 
 So I've started doing that and I've got no problems so far. I
 guess I don't need all that fancy .lib and -L-lmyLib stuff after
 all. You can pack all sorts of stuff into a file1.o file instead.
 
 I guess I solved my problem for now. Hopefully I'm making sense.

Libraries are not intented for incremental compilation. They are for distributing code in a unit which can be used by programs. And in the case of a shared library, it gives the added benefit of reducing the amount of duplicate code you get in binaries (saving both memory and disk space). If you want to do incremental compilation, then use -c to generate object files that you link together when you create the actual executable. - Jonathan M Davis

Actually the -lib switch might be the answer to one of the incremental compilation problems DMD suffers from. That DMD usually does not output all symbols to all object files which can result in missing symbols when doing incremental compilation. I've heard that the when using the -lib flag DMD will output all symbols to all object files.

If so, it's an implementation issue. In principle, libraries have nothing to do with incremental compilation. They're about modularizing code for reusability. - Jonathan M Davis
Mar 05 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Tuesday, March 06, 2012 11:31:40 Jacob Carlborg wrote:
 On 2012-03-06 08:56, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 08:29:53 Jacob Carlborg wrote:
 On 2012-03-06 02:21, Jonathan M Davis wrote:
 On Tuesday, March 06, 2012 01:53:02 Zach the Mystic wrote:
 Reading the documentation about compiler options and flags here:
 http://dlang.org/dmd-osx.html
 
 led me to believe that building libraries was the right way to do
 incremental compilation. But I thought, well, can I just build
 file1.o object files instead?
 
 So I've started doing that and I've got no problems so far. I
 guess I don't need all that fancy .lib and -L-lmyLib stuff after
 all. You can pack all sorts of stuff into a file1.o file instead.
 
 I guess I solved my problem for now. Hopefully I'm making sense.

Libraries are not intented for incremental compilation. They are for distributing code in a unit which can be used by programs. And in the case of a shared library, it gives the added benefit of reducing the amount of duplicate code you get in binaries (saving both memory and disk space). If you want to do incremental compilation, then use -c to generate object files that you link together when you create the actual executable. - Jonathan M Davis

Actually the -lib switch might be the answer to one of the incremental compilation problems DMD suffers from. That DMD usually does not output all symbols to all object files which can result in missing symbols when doing incremental compilation. I've heard that the when using the -lib flag DMD will output all symbols to all object files.

If so, it's an implementation issue. In principle, libraries have nothing to do with incremental compilation. They're about modularizing code for reusability. - Jonathan M Davis

Yes, but Walter doesn't seem to want to fix that issue.

Well, from the sounds of it, he's going to need to be convinced one way or another - which can be frustratingly difficult at times. Sometimes, he's right even when pretty much no one agrees with him, but other times, he just doesn't seem to get it. He's hard to convince in either case though. - Jonathan M Davis
Mar 06 2012
prev sibling parent "Zach the Mystic" <reachMINUSTHISzachgmail dot.com> writes:
On Tuesday, 6 March 2012 at 07:29:54 UTC, Jacob Carlborg wrote:
 Actually the -lib switch might be the answer to one of the 
 incremental compilation problems DMD suffers from. That DMD 
 usually does not output all symbols to all object files which 
 can result in missing symbols when doing incremental 
 compilation. I've heard that the when using the -lib flag DMD 
 will output all symbols to all object files.

Hey you know, this could also be a problem because of my original post. Instead of missing symbols, with -lib, you could start to suffer from unintended duplicates, too.
Mar 07 2012