www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - dmdcache

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
A colleague of mine has written dmdcache which may be very useful for 
some projects:

   https://github.com/seeraven/dmdcache

It drops our build time

   from 8 minutes
   to 45 seconds

on a particular build environment for about half a dozen D programs, one 
of which ends up being a 2G executable! WAT! :) And the total cache size 
is 5.5G. Wow!

This build is with dmd 2.084.1 and that one particular application uses 
tons of template instantiations most of which are in generated source 
code. If I remember correctly, 2.084.1 does not contain template symbol 
name improvements and that may be the reason for the large size.

Enjoy!

Ali
Apr 25
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote:
 A colleague of mine has written dmdcache which may be very 
 useful for some projects:

   https://github.com/seeraven/dmdcache

 It drops our build time

   from 8 minutes
   to 45 seconds

 on a particular build environment for about half a dozen D 
 programs, one of which ends up being a 2G executable! WAT! :) 
 And the total cache size is 5.5G. Wow!

 This build is with dmd 2.084.1 and that one particular 
 application uses tons of template instantiations most of which 
 are in generated source code. If I remember correctly, 2.084.1 
 does not contain template symbol name improvements and that may 
 be the reason for the large size.

 Enjoy!

 Ali
The main problem with this is that it does not take string imports into account, (or does it ???, I don't see how it could ....) Also the compilers output can depend on the timestamp at which the compilation was done.
Apr 25
parent reply bauss <jj_1337 live.dk> writes:
On Saturday, 25 April 2020 at 10:35:49 UTC, Stefan Koch wrote:
 On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote:
 A colleague of mine has written dmdcache which may be very 
 useful for some projects:

   https://github.com/seeraven/dmdcache

 It drops our build time

   from 8 minutes
   to 45 seconds

 on a particular build environment for about half a dozen D 
 programs, one of which ends up being a 2G executable! WAT! :) 
 And the total cache size is 5.5G. Wow!

 This build is with dmd 2.084.1 and that one particular 
 application uses tons of template instantiations most of which 
 are in generated source code. If I remember correctly, 2.084.1 
 does not contain template symbol name improvements and that 
 may be the reason for the large size.

 Enjoy!

 Ali
The main problem with this is that it does not take string imports into account, (or does it ???, I don't see how it could ....) Also the compilers output can depend on the timestamp at which the compilation was done.
Yeah, doesn't look like it which means it might not be useful in projects that does a lot of compile-time stuff. There is no way to determine the import statements either as those themselves can be generated at compile-time. I can see this being useful for large projects that does not use CTFE but other than that I think it might just create subtle bugs because you won't immediately know that some part of your code didn't update and isn't working as intended because the code for it was imported from another file etc.
Apr 25
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 4/25/20 9:01 AM, bauss wrote:

 On Saturday, 25 April 2020 at 10:35:49 UTC, Stefan Koch wrote:
 The main problem with this is that it does not take string imports
 into account
[...]
 Yeah, doesn't look like it which means it might not be useful in
 projects that does a lot of compile-time stuff.

 There is no way to determine the import statements either as those
 themselves can be generated at compile-time.
Luckily, dmd gives all of that information with both -v and -X command line switches. However, currently dmdcache doesn't seem to look for string imports; opportunity for improvement. :) Ali
Apr 26
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2020 1:52 AM, Ali Çehreli wrote:
 However, currently dmdcache doesn't seem to look for string imports; 
 opportunity for improvement. :)
That should not be necessary if you base it on -J instead.
Apr 26
prev sibling next sibling parent reply Johan <j j.nl> writes:
On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote:
 A colleague of mine has written dmdcache which may be very 
 useful for some projects:

   https://github.com/seeraven/dmdcache

 It drops our build time

   from 8 minutes
   to 45 seconds
Hey Ali, Did you also try with LDC's built-in object file caching (asm codegen caching) ? Cheers, Johan
Apr 25
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 4/25/20 4:39 AM, Johan wrote:
 On Saturday, 25 April 2020 at 10:17:50 UTC, Ali =C3=87ehreli wrote:
 A colleague of mine has written dmdcache which may be very useful for =
 some projects:

 =C2=A0 https://github.com/seeraven/dmdcache

 It drops our build time

 =C2=A0 from 8 minutes
 =C2=A0 to 45 seconds
=20 Hey Ali, =C2=A0 Did you also try with LDC's built-in object file caching (asm c=
odegen=20
 caching) ?
=20
 Cheers,
  =C2=A0 Johan
=20
That sounds very promising. Time to introduce ldc to the project. Ali
Apr 26
prev sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 25 April 2020 at 10:17:50 UTC, Ali Çehreli wrote:
 A colleague of mine has written dmdcache which may be very 
 useful for some projects:

   https://github.com/seeraven/dmdcache

 It drops our build time

   from 8 minutes
   to 45 seconds

 on a particular build environment for about half a dozen D 
 programs, one of which ends up being a 2G executable! WAT! :) 
 And the total cache size is 5.5G. Wow!

 This build is with dmd 2.084.1 and that one particular 
 application uses tons of template instantiations most of which 
 are in generated source code. If I remember correctly, 2.084.1 
 does not contain template symbol name improvements and that may 
 be the reason for the large size.

 Enjoy!

 Ali
Perhaps I'm being very dumb, but how does this differ from just using make?
Apr 25
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 4/25/20 5:30 AM, John Colvin wrote:

 how does this differ from just using make?
make is great and I love it (really) but it works at a coarser level. There is no way for it to know that a particular command will produce the same output. As I understand it, dmdcache is supposed to be similar to ccache, which we already use with make to speed up our C++ compilations: https://ccache.dev/ Ali
Apr 26