www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A very fast linker to replace gcc's ld

reply "Asman01" <jckj33 gmail.com> writes:
Today I've found a good alternative to slow gcc's linker: gold. 
Written by  Google in C++ it's a linker writen for large code 
bases with C++ in mind. The author claims it's about 5 times fast 
than gcc's one. It does support ELF only and UNIX-like symtem I 
think it's enough to dmd on linux or am I missing something? I 
believe it can reduce a lot dmd's compilation speed on these 
system. It's under same GPL as gcc's ld. I wonder if we can use 
it to make dmd still fastest. :)
It seems same case as Facebook needed a new C++ preprocessor for 
its large code base.

http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html
https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf
Apr 05 2014
next sibling parent "Joakim" <joakim airpost.net> writes:
On Saturday, 5 April 2014 at 21:47:33 UTC, Asman01 wrote:
 Today I've found a good alternative to slow gcc's linker: gold. 
 Written by  Google in C++ it's a linker writen for large code 
 bases with C++ in mind. The author claims it's about 5 times 
 fast than gcc's one. It does support ELF only and UNIX-like 
 symtem I think it's enough to dmd on linux or am I missing 
 something? I believe it can reduce a lot dmd's compilation 
 speed on these system. It's under same GPL as gcc's ld. I 
 wonder if we can use it to make dmd still fastest. :)
 It seems same case as Facebook needed a new C++ preprocessor 
 for its large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html
 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf

Dmd already uses whatever the system linker is and on Arch that's gold. The Android NDK also uses gold by default, though they also provide the original bfd ld and a newer llvm-based linker started by MediaTek, mclinker: https://code.google.com/p/mclinker/
Apr 05 2014
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:
 on Arch that's gold.

it is? Not for me, unless I'm missing something.
Apr 05 2014
prev sibling next sibling parent "Asman01" <jckj33 gmail.com> writes:
On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:
 On Saturday, 5 April 2014 at 21:47:33 UTC, Asman01 wrote:
 Today I've found a good alternative to slow gcc's linker: 
 gold. Written by  Google in C++ it's a linker writen for large 
 code bases with C++ in mind. The author claims it's about 5 
 times fast than gcc's one. It does support ELF only and 
 UNIX-like symtem I think it's enough to dmd on linux or am I 
 missing something? I believe it can reduce a lot dmd's 
 compilation speed on these system. It's under same GPL as 
 gcc's ld. I wonder if we can use it to make dmd still fastest. 
 :)
 It seems same case as Facebook needed a new C++ preprocessor 
 for its large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html
 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf

Dmd already uses whatever the system linker is and on Arch that's gold. The Android NDK also uses gold by default, though they also provide the original bfd ld and a newer llvm-based linker started by MediaTek, mclinker: https://code.google.com/p/mclinker/

Well, as far I know, it does invoke gcc and gcc does use the ld. I see this with -v flag. What am I missing? I'll check out this llvm-based linker. If it's the linker which I've hear they are planing to replace gcc ld it's a big step.
Apr 05 2014
prev sibling next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:
 Dmd already uses whatever the system linker is and on Arch 
 that's gold.  The Android NDK also uses gold by default, though 
 they also provide the original bfd ld and a newer llvm-based 
 linker started by MediaTek, mclinker:

 https://code.google.com/p/mclinker/

DMD emit object file that, depending on the version of Gold: - Will produce broken binaries. - Will fail with cryptic internal error; I've tried to patch Gold, to accept dmd's file, but eventually ran out of time. If one want to take over that work I can point some places in Gold's source code to look at. I'd also not be that surprised that DMD does emit somthing weird in its object files.
Apr 05 2014
prev sibling next sibling parent "Asman01" <jckj33 gmail.com> writes:
On Sunday, 6 April 2014 at 00:26:12 UTC, deadalnix wrote:
 On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:
 Dmd already uses whatever the system linker is and on Arch 
 that's gold.  The Android NDK also uses gold by default, 
 though they also provide the original bfd ld and a newer 
 llvm-based linker started by MediaTek, mclinker:

 https://code.google.com/p/mclinker/

DMD emit object file that, depending on the version of Gold: - Will produce broken binaries. - Will fail with cryptic internal error;

Why that?
 I've tried to patch Gold, to accept dmd's file, but eventually 
 ran out of time. If one want to take over that work I can point 
 some places in Gold's source code to look at. I'd also not be 
 that surprised that DMD does emit somthing weird in its object 
 files.

By weird do you mean gcc-specific?
Apr 05 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/14, 2:47 PM, Asman01 wrote:
 Today I've found a good alternative to slow gcc's linker: gold. Written
 by  Google in C++ it's a linker writen for large code bases with C++ in
 mind. The author claims it's about 5 times fast than gcc's one. It does
 support ELF only and UNIX-like symtem I think it's enough to dmd on
 linux or am I missing something? I believe it can reduce a lot dmd's
 compilation speed on these system. It's under same GPL as gcc's ld. I
 wonder if we can use it to make dmd still fastest. :)
 It seems same case as Facebook needed a new C++ preprocessor for its
 large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html

 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf

s/reduce/improve/ Yah, we've been using gold for a long time. Not on all projects though, on some it crashes. Andrei
Apr 05 2014
prev sibling next sibling parent "Joakim" <dlang joakim.airpost.net> writes:
On Saturday, 5 April 2014 at 23:56:49 UTC, John Colvin wrote:
 On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:
 on Arch that's gold.

it is? Not for me, unless I'm missing something.

Sorry, I made a mistake. Dmd simply invokes the system C compiler and the linker that it uses. I set CC to clang and build dmd trunk with it. Clang uses gold, which is what led me to assume it was the default. But I just checked and gcc, the default C compiler on Arch, uses bfd ld, so it depends on what you use as your C compiler. On Sunday, 6 April 2014 at 00:15:21 UTC, Asman01 wrote:
 On Saturday, 5 April 2014 at 22:30:28 UTC, Joakim wrote:
 Dmd already uses whatever the system linker is and on Arch 
 that's gold.  The Android NDK also uses gold by default, 
 though they also provide the original bfd ld and a newer 
 llvm-based linker started by MediaTek, mclinker:

 https://code.google.com/p/mclinker/

Well, as far I know, it does invoke gcc and gcc does use the ld. I see this with -v flag. What am I missing? I'll check out this llvm-based linker. If it's the linker which I've hear they are planing to replace gcc ld it's a big step.

See above, I was wrong. As for llvm-based linkers, there are a couple going around. The one you're probably thinking of is lld: http://lld.llvm.org/
Apr 06 2014
prev sibling parent "asman" <lol.themask gmail.com> writes:
On Sunday, 6 April 2014 at 00:50:43 UTC, Andrei Alexandrescu 
wrote:
 On 4/5/14, 2:47 PM, Asman01 wrote:
 Today I've found a good alternative to slow gcc's linker: 
 gold. Written
 by  Google in C++ it's a linker writen for large code bases 
 with C++ in
 mind. The author claims it's about 5 times fast than gcc's 
 one. It does
 support ELF only and UNIX-like symtem I think it's enough to 
 dmd on
 linux or am I missing something? I believe it can reduce a lot 
 dmd's
 compilation speed on these system. It's under same GPL as 
 gcc's ld. I
 wonder if we can use it to make dmd still fastest. :)
 It seems same case as Facebook needed a new C++ preprocessor 
 for its
 large code base.

 http://google-opensource.blogspot.com.br/2008/04/gold-google-releases-new-and-improved.html

 https://events.linuxfoundation.org/images/stories/pdf/lfcs2012_ccoutant.pdf

s/reduce/improve/ Yah, we've been using gold for a long time. Not on all projects though, on some it crashes. Andrei

haha. It's great already being using it. So, it's not a full replacement to gcc's ld because it is not stable enough, right? if so, do you think it's valuable (for DMD community) to someone put some effort to try to fix as possible this crashes? I will do some test on my linux machine later to see how fast and stable it is.
Apr 07 2014