www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - How to accelerate the linking time?

reply zoujiaqing <zoujiaqing gmail.com> writes:
zoujiaqing osx:~/projects/hunt-skeleton (*)
 time dub build                                                  
                             master [fe3d6bc] modified
Performing "debug" build using /Library/D/dmd/bin/dmd for x86_64. hunt 1.6.1: target for configuration "library" is up to date. hunt-extra 1.0.0: target for configuration "library" is up to date. hunt-net 0.5.1: building configuration "default"... hunt-time 1.1.1: building configuration "library"... hunt-proton 1.1.0-beta.3: building configuration "library"... hunt-amqp 1.1.0-beta.4: building configuration "library"... hunt-pool 1.1.1: building configuration "library"... hunt-amqp-client 1.1.0-beta.2: building configuration "library"... hunt-redis 1.2.0: building configuration "library"... hunt-cache 0.8.1: building configuration "default"... hunt-console 0.2.0: target for configuration "hunt-console" is up to date. hunt-sql 1.4.0: target for configuration "library" is up to date. hunt-database 2.1.0: building configuration "default"... hunt-validation 0.3.0: target for configuration "library" is up to date. hunt-entity 2.6.0: building configuration "library"... hunt-http 0.6.2: building configuration "default"... hunt-jwt 0.1.0-beta.1: target for configuration "library" is up to date. hunt-security 0.4.0: target for configuration "library" is up to date. hunt-shiro 1.1.3: building configuration "library"... poodinis 8.1.0-beta.2: target for configuration "library" is up to date. hunt-framework 3.2.0: building configuration "library"... hunt-skeleton 3.1.0+commit.4.gfe3d6bc: building configuration "application"... Linking... To force a rebuild of up-to-date targets, run again with --force. dub build 96.60s user 14.75s system 94% cpu 1:57.62 total My laptop is MacBook Pro 15, but linking time so long.
Jul 09 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 10 July 2020 at 05:45:03 UTC, zoujiaqing wrote:
 [...]
I think the answer is use lld instead of gnu ld
Jul 10 2020
parent Jacob Carlborg <doob me.com> writes:
On Friday, 10 July 2020 at 07:06:44 UTC, Stefan Koch wrote:
 On Friday, 10 July 2020 at 05:45:03 UTC, zoujiaqing wrote:
 [...]
I think the answer is use lld instead of gnu ld
He's running macOS, which doesn't use GNU ld. LLD does not work for D programs on macOS. At least not when compiling them using LDC. -- /Jacob Carlborg
Jul 10 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/10/20 1:45 AM, zoujiaqing wrote:
 zoujiaqing osx:~/projects/hunt-skeleton (*)
 time dub build                             master
[fe3d6bc] modified
Performing "debug" build using /Library/D/dmd/bin/dmd for x86_64. hunt 1.6.1: target for configuration "library" is up to date. hunt-extra 1.0.0: target for configuration "library" is up to date. hunt-net 0.5.1: building configuration "default"... hunt-time 1.1.1: building configuration "library"... hunt-proton 1.1.0-beta.3: building configuration "library"... hunt-amqp 1.1.0-beta.4: building configuration "library"... hunt-pool 1.1.1: building configuration "library"... hunt-amqp-client 1.1.0-beta.2: building configuration "library"... hunt-redis 1.2.0: building configuration "library"... hunt-cache 0.8.1: building configuration "default"... hunt-console 0.2.0: target for configuration "hunt-console" is up to date. hunt-sql 1.4.0: target for configuration "library" is up to date. hunt-database 2.1.0: building configuration "default"... hunt-validation 0.3.0: target for configuration "library" is up to date. hunt-entity 2.6.0: building configuration "library"... hunt-http 0.6.2: building configuration "default"... hunt-jwt 0.1.0-beta.1: target for configuration "library" is up to date. hunt-security 0.4.0: target for configuration "library" is up to date. hunt-shiro 1.1.3: building configuration "library"... poodinis 8.1.0-beta.2: target for configuration "library" is up to date. hunt-framework 3.2.0: building configuration "library"... hunt-skeleton 3.1.0+commit.4.gfe3d6bc: building configuration "application"... Linking... To force a rebuild of up-to-date targets, run again with --force. dub build  96.60s user 14.75s system 94% cpu 1:57.62 total My laptop is MacBook Pro 15, but linking time so long.
The linking time is high because of the symbol count/sizes generally. Lots of templates means the linker takes a while to sort through the mess you made ;) I have the same problem on vibe.d projects. typically for me, the compilation step takes at least 50% of the time (hard to tell, as dub does not output the timestamp at that point). -Steve
Jul 10 2020
parent reply Arun Chandrasekaran <aruncxy gmail.com> writes:
On Friday, 10 July 2020 at 12:15:56 UTC, Steven Schveighoffer 
wrote:
 On 7/10/20 1:45 AM, zoujiaqing wrote:
 [...]
The linking time is high because of the symbol count/sizes generally. Lots of templates means the linker takes a while to sort through the mess you made ;)
Mike Franklin optimized the mangling a while ago. That was a crucial change, IMO (imagine how big the original was!). Any idea why the unnecessary symbols gets generated by the compiler (foreach, etc) in release mode? I can understand for debug builds though...
Jul 10 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/10/20 8:17 PM, Arun Chandrasekaran wrote:
 On Friday, 10 July 2020 at 12:15:56 UTC, Steven Schveighoffer wrote:
 On 7/10/20 1:45 AM, zoujiaqing wrote:
 [...]
The linking time is high because of the symbol count/sizes generally. Lots of templates means the linker takes a while to sort through the mess you made ;)
Mike Franklin optimized the mangling a while ago. That was a crucial change, IMO (imagine how big the original was!).
Did you mean Rainer's change to use backreferences? That was for exponential symbols for Voldemort types with a lot of repetitive pieces. It's still quite possible to get large symbols if you wrap a ton of unrelated templates together (vibe.d and probably Hunt have quite a bit of templates that wrap each other). Apologies if Mike did something else, I don't recall what that is. There was a recent check of the basic hello world, and the number of symbols generated by format was alarming.
 
 Any idea why the unnecessary symbols gets generated by the compiler 
 (foreach, etc) in release mode? I can understand for debug builds though...
I don't know if anyone's actually done any research on what is "unnecessary". If there are a lot of unnecessary symbols in the binaries, it would be good to know where they are coming from, and how we can possibly remove them. This area of D development (binary object symbols) is not very sexy, so I don't imagine many people want to dive into it. -Steve
Jul 10 2020
parent Arun Chandrasekaran <aruncxy gmail.com> writes:
On Saturday, 11 July 2020 at 00:36:47 UTC, Steven Schveighoffer 
wrote:
 On 7/10/20 8:17 PM, Arun Chandrasekaran wrote:
 On Friday, 10 July 2020 at 12:15:56 UTC, Steven Schveighoffer 
 wrote:
 On 7/10/20 1:45 AM, zoujiaqing wrote:
 [...]
The linking time is high because of the symbol count/sizes generally. Lots of templates means the linker takes a while to sort through the mess you made ;)
Mike Franklin optimized the mangling a while ago. That was a crucial change, IMO (imagine how big the original was!).
Did you mean Rainer's change to use backreferences?
Yeah, sorry about that. It was indeed Rainer's change!
Jul 12 2020