www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Speed up compilation of CTFE-heavy programs with this One Weird Trick!

reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
For each function that is only called during compilation (e.g. 
for code generation), at the top add:

	if (!__ctfe)
		return;

That's it. For one project, this brought compilation time (with 
optimizations) from over an hour down to seconds!

As it turns out, DMD will codegen all functions, even templated 
ones when they're instantiated, whether they're called by 
something or not. With metaprogramming-heavy code, it's easy to 
run into some optimizer performance corner cases, however this 
may make an impact regardless.
Nov 02 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/02/2016 11:43 PM, Vladimir Panteleev wrote:

 For each function that is only called during compilation (e.g. for code
 generation), at the top add:

     if (!__ctfe)
         return;

 That's it. For one project, this brought compilation time (with
 optimizations) from over an hour down to seconds!

 As it turns out, DMD will codegen all functions, even templated ones
 when they're instantiated, whether they're called by something or not.
 With metaprogramming-heavy code, it's easy to run into some optimizer
 performance corner cases, however this may make an impact regardless.


That's interesting. cc Dmitry, perhaps adding that to ctRegex would be 
helpful. -- Andrei
Nov 02 2016
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 11/3/16 5:20 AM, Andrei Alexandrescu wrote:

 On 11/02/2016 11:43 PM, Vladimir Panteleev wrote:

 For each function that is only called during compilation (e.g. for code
 generation), at the top add:

     if (!__ctfe)
         return;

 That's it. For one project, this brought compilation time (with
 optimizations) from over an hour down to seconds!

 As it turns out, DMD will codegen all functions, even templated ones
 when they're instantiated, whether they're called by something or not.
 With metaprogramming-heavy code, it's easy to run into some optimizer
 performance corner cases, however this may make an impact regardless.


 That's interesting. cc Dmitry, perhaps adding that to ctRegex would be
 helpful. -- Andrei


For ctRegex it's hardly applicable as it reuses the same code for 
run-time and compile-time version.

---
Dmitry Olshansky
Nov 03 2016
prev sibling next sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 3 November 2016 at 03:43:06 UTC, Vladimir Panteleev 
wrote:

 For each function that is only called during compilation (e.g. 
 for code generation), at the top add:

 	if (!__ctfe)
 		return;

 That's it. For one project, this brought compilation time (with 
 optimizations) from over an hour down to seconds!

 As it turns out, DMD will codegen all functions, even templated 
 ones when they're instantiated, whether they're called by 
 something or not.


  Sounds like an issue. Although, I almost think a  noctfe tag 
which would refuse to compile/use code during CTFE, or in effect 
add that line at the beginning to functions invisibly.

  Obviously having functions only voluntarily used for CTFE would 
be a pain. having a section Opt Out after a certain point seems 
more useful, and would require far less work for the programmers.
Nov 03 2016
prev sibling next sibling parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 03.11.2016 um 04:43 schrieb Vladimir Panteleev:

 For each function that is only called during compilation (e.g. for code
 generation), at the top add:

     if (!__ctfe)
         return;

 That's it. For one project, this brought compilation time (with
 optimizations) from over an hour down to seconds!

 As it turns out, DMD will codegen all functions, even templated ones
 when they're instantiated, whether they're called by something or not.
 With metaprogramming-heavy code, it's easy to run into some optimizer
 performance corner cases, however this may make an impact regardless.


Does "if (!_ctfe) assert(false);" work, too? That would also protect 
against accidential mistakes.
Nov 03 2016
parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 3 November 2016 at 12:19:09 UTC, Sönke Ludwig wrote:

 Does "if (!_ctfe) assert(false);" work, too? That would also 
 protect against accidential mistakes.


Doesn't seem like it. { assert(false); return; } works (in 
reducing compile time) but generates "statement is not reachable" 
warnings for the return statement.
Nov 03 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-11-03 04:43, Vladimir Panteleev wrote:


 As it turns out, DMD will codegen all functions, even templated ones
 when they're instantiated, whether they're called by something or not.
 With metaprogramming-heavy code, it's easy to run into some optimizer
 performance corner cases, however this may make an impact regardless.


I guess the compiler assumes that a function can end up in a library and 
be called by some other program. Sounds like a whole program 
analysis/optimization would be required. Or if the compiler is invoked 
as building an executable it can assume it can avoid generating code for 
unused functions?

To me it sounds like a  ctfe attribute would be useful. The compiler 
could enforce that functions annotated with  ctfe will only contain code 
that can be CTFE-able, can only be called in a CTFE context and will not 
generate any code.

-- 
/Jacob Carlborg
Nov 03 2016