www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CompileTime performance measurement

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys.

I recently implemented __ctfeWriteln.
Based on that experience I have now implemented another pseudo 
function called __ctfeTicksMs.
That evaluates to a uint representing the number of milliseconds 
elapsed between the start of dmd and the time of semantic 
evaluation of this expression.

This does allow meaningful compiletime performance tests to be 
written.
spanning both CTFE and template-incitations timeings.

Please tell me what you think.
Sep 03 2016
next sibling parent reply David Nadlinger <code klickverbot.at> writes:
On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 Based on that experience I have now implemented another pseudo 
 function called __ctfeTicksMs.
 […]
 Please tell me what you think.
Please don't. This makes CTFE indeterministic. To write performance tests, just measure compilation of a whole program (possibly with -o-). The variance due to the startup/shutdown overhead can trivially be controlled by just executing the CTFE code in question often enough. — David
Sep 03 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 00:08:14 UTC, David Nadlinger 
wrote:
 Please don't. This makes CTFE indeterministic.
Please elaborate on why this would have a negative impact ? if someone chooses to use a symbol called __ctfeTicksMs they shoud know what they are doing.
 To write performance tests, just measure compilation of a whole 
 program (possibly with -o-). The variance due to the 
 startup/shutdown overhead can trivially be controlled by just 
 executing the CTFE code in question often enough.
That will only allow you to tell how much overall ctfe-time you spent. It will not allow you to pinpoint and optimize the offenders.
Sep 03 2016
prev sibling parent Martin Nowak <code dawg.eu> writes:
On Sunday, 4 September 2016 at 00:08:14 UTC, David Nadlinger 
wrote:
 Please don't. This makes CTFE indeterministic.
Well we already have __TIMESTAMP__, though I think it doesn't change during compilation.
Sep 06 2016
prev sibling next sibling parent reply sarn <sarn theartofmachinery.com> writes:
On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 I recently implemented __ctfeWriteln.
Sounds like pragma msg. How does it compare? https://dlang.org/spec/pragma.html#msg
Sep 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 01:44:40 UTC, sarn wrote:
 On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 I recently implemented __ctfeWriteln.
Sounds like pragma msg. How does it compare? https://dlang.org/spec/pragma.html#msg
Pragma msg can only print compiletime constants. While __ctfeWriteln can print state while doing CTFE. Example int fn(int n) { import std.conv; __ctfeWriteln((n-10).to!string); return n; } static assert(fn(22)); will print 12; whereas int fn(int n) { import std.conv; pragma(msg, n.to!string); return n; } will tell you that the symbol n is not avilable at compiletime
Sep 03 2016
parent reply sarn <sarn theartofmachinery.com> writes:
On Sunday, 4 September 2016 at 01:53:21 UTC, Stefan Koch wrote:
 Pragma msg can only print compiletime constants.
 While __ctfeWriteln can print state while doing CTFE.
Thanks, that makes a lot of sense. Just to check, it prints to standard error, right? Also, the issue of non-deterministic compilation reminds me of the debate about allowing logging statements in pure functions. Maybe there's a similar answer (i.e., making it only work in some kind of debug mode).
Sep 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 02:03:49 UTC, sarn wrote:
 On Sunday, 4 September 2016 at 01:53:21 UTC, Stefan Koch wrote:
 Pragma msg can only print compiletime constants.
 While __ctfeWriteln can print state while doing CTFE.
Thanks, that makes a lot of sense. Just to check, it prints to standard error, right? Also, the issue of non-deterministic compilation reminds me of the debate about allowing logging statements in pure functions. Maybe there's a similar answer (i.e., making it only work in some kind of debug mode).
This works already. Anything placed in a debug {} block will be considered pure regardless.
Sep 03 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
 This works already.
 Anything placed in a debug {} block will be considered pure 
 regardless.
Opps your comment was about the debate. I would say that __ctfeWriteln and __ctfeTicksMs should not work outside of debug.
Sep 03 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 04/09/2016 2:08 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
 This works already.
 Anything placed in a debug {} block will be considered pure regardless.
Opps your comment was about the debate. I would say that __ctfeWriteln and __ctfeTicksMs should not work outside of debug.
Can we have writeln and writefln call into it if __ctfe is true? Just so that we have got some consistency between runtime and CTFE usage.
Sep 03 2016
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole 
wrote:
 On 04/09/2016 2:08 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
 This works already.
 Anything placed in a debug {} block will be considered pure 
 regardless.
Opps your comment was about the debate. I would say that __ctfeWriteln and __ctfeTicksMs should not work outside of debug.
Can we have writeln and writefln call into it if __ctfe is true? Just so that we have got some consistency between runtime and CTFE usage.
No! writeln and __ctfeWriteln are to be regarded as completely different things. __ctfeWriteln is a debugging tool only! It should not be used in any production code.
Sep 03 2016
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 04/09/2016 4:14 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole wrote:
 On 04/09/2016 2:08 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
 This works already.
 Anything placed in a debug {} block will be considered pure regardless.
Opps your comment was about the debate. I would say that __ctfeWriteln and __ctfeTicksMs should not work outside of debug.
Can we have writeln and writefln call into it if __ctfe is true? Just so that we have got some consistency between runtime and CTFE usage.
No! writeln and __ctfeWriteln are to be regarded as completely different things. __ctfeWriteln is a debugging tool only! It should not be used in any production code.
void writeln(T...)(T args) { if (__ctfe){ debug { __ctfeWriteln(args); } } else { // ... current implementation } } Are you sureeeee?
Sep 03 2016
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sunday, September 04, 2016 16:24:34 rikki cattermole via Digitalmars-d 
wrote:
 On 04/09/2016 4:14 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole wrote:
 On 04/09/2016 2:08 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
 This works already.
 Anything placed in a debug {} block will be considered pure regardless.
Opps your comment was about the debate. I would say that __ctfeWriteln and __ctfeTicksMs should not work outside of debug.
Can we have writeln and writefln call into it if __ctfe is true? Just so that we have got some consistency between runtime and CTFE usage.
No! writeln and __ctfeWriteln are to be regarded as completely different things. __ctfeWriteln is a debugging tool only! It should not be used in any production code.
void writeln(T...)(T args) { if (__ctfe){ debug { __ctfeWriteln(args); } } else { // ... current implementation } } Are you sureeeee?
He didn't say that it _couldn't_ be done. He said that it _shouldn't_ be done. - Jonathan M Davis
Sep 03 2016
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 04/09/2016 4:31 PM, Jonathan M Davis via Digitalmars-d wrote:
 On Sunday, September 04, 2016 16:24:34 rikki cattermole via Digitalmars-d
 wrote:
 On 04/09/2016 4:14 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole wrote:
 On 04/09/2016 2:08 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
 This works already.
 Anything placed in a debug {} block will be considered pure regardless.
Opps your comment was about the debate. I would say that __ctfeWriteln and __ctfeTicksMs should not work outside of debug.
Can we have writeln and writefln call into it if __ctfe is true? Just so that we have got some consistency between runtime and CTFE usage.
No! writeln and __ctfeWriteln are to be regarded as completely different things. __ctfeWriteln is a debugging tool only! It should not be used in any production code.
void writeln(T...)(T args) { if (__ctfe){ debug { __ctfeWriteln(args); } } else { // ... current implementation } } Are you sureeeee?
He didn't say that it _couldn't_ be done. He said that it _shouldn't_ be done. - Jonathan M Davis
I wrote that code to prove that it could be used only for debugging purposes, which is Stefan's argument.
Sep 03 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 04:35:15 UTC, rikki cattermole 
wrote:
 void writeln(T...)(T args) {
   if (__ctfe){
       debug {
           __ctfeWriteln(args);
       }
   } else {
       // ... current implementation
   }
 }
That will not work. The signature is void __ctfeWriteln(const string s)
Sep 03 2016
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 04:31:09 UTC, Jonathan M Davis 
wrote:
 He didn't say that it _couldn't_ be done. He said that it 
 _shouldn't_ be done.

 - Jonathan M Davis
Yes exactly.
Sep 03 2016
prev sibling parent reply timepp <tongjunhui live.cn> writes:
On Sunday, 4 September 2016 at 04:24:34 UTC, rikki cattermole 
wrote:
 void writeln(T...)(T args) {
 	if (__ctfe){
 		debug {
 			__ctfeWriteln(args);
 		}
 	} else {
 		// ... current implementation
 	}
 }

 Are you sureeeee?
any usage example? consider a normal usage: writeln("done."); I just want a runtime output. how can I tell the compiler not to print "done." at compile time?
Sep 05 2016
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 06 Sep 2016 05:02:54 +0000
schrieb timepp <tongjunhui live.cn>:

 On Sunday, 4 September 2016 at 04:24:34 UTC, rikki cattermole 
 wrote:
 void writeln(T...)(T args) {
 	if (__ctfe){
 		debug {
 			__ctfeWriteln(args);
 		}
 	} else {
 		// ... current implementation
 	}
 }

 Are you sureeeee?  
any usage example? consider a normal usage: writeln("done."); I just want a runtime output. how can I tell the compiler not to print "done." at compile time?
If you actually call a function during compile-time that uses writeln and want it to be silent you would write: if (!__ctfe) writeln("done."); -- Marco
Sep 09 2016
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/4/16 6:14 AM, Stefan Koch wrote:
 writeln and __ctfeWriteln are to be regarded as completely different
 things.
 __ctfeWriteln is a debugging tool only!
 It should not be used in any production code.
Well I'm not sure how that would be reasonably enforced. -- Andrei
Sep 04 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 4 September 2016 at 12:38:05 UTC, Andrei Alexandrescu 
wrote:
 On 9/4/16 6:14 AM, Stefan Koch wrote:
 writeln and __ctfeWriteln are to be regarded as completely 
 different
 things.
 __ctfeWriteln is a debugging tool only!
 It should not be used in any production code.
Well I'm not sure how that would be reasonably enforced. -- Andrei
One could enforce it by defining it inside a version or debug block. The reason I do not want to see this in production code is as follows: In the engine I am working on, communication between it and the rest of dmd is kept to a minimum, because : "The new CTFE engine abstracts away everything into bytecode, there is no guarantee that the bytecode-evaluator is run in the same process or even on the same machine."
Sep 04 2016
parent Martin Tschierschke <mt smartdolphin.de> writes:
On Sunday, 4 September 2016 at 19:36:16 UTC, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 12:38:05 UTC, Andrei 
 Alexandrescu wrote:
 On 9/4/16 6:14 AM, Stefan Koch wrote:
 writeln and __ctfeWriteln are to be regarded as completely 
 different
 things.
 __ctfeWriteln is a debugging tool only!
 It should not be used in any production code.
Well I'm not sure how that would be reasonably enforced. -- Andrei
One could enforce it by defining it inside a version or debug block. The reason I do not want to see this in production code is as follows: In the engine I am working on, communication between it and the rest of dmd is kept to a minimum, because : "The new CTFE engine abstracts away everything into bytecode, there is no guarantee that the bytecode-evaluator is run in the same process or even on the same machine."
An alternative might be, to save your ctfe values in an static array and output them on startup of the compiled program. Same idea is used in vibe.d to make a caching of the templates evaluation possible. See: http://code.dlang.org/packages/diet-ng Experimental HTML template caching
Sep 06 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/4/16 6:10 AM, rikki cattermole wrote:
 On 04/09/2016 2:08 PM, Stefan Koch wrote:
 On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
 This works already.
 Anything placed in a debug {} block will be considered pure regardless.
Opps your comment was about the debate. I would say that __ctfeWriteln and __ctfeTicksMs should not work outside of debug.
Can we have writeln and writefln call into it if __ctfe is true? Just so that we have got some consistency between runtime and CTFE usage.
Yes, that's the natural way. __ctfeWriteln is really removing a limitation of writeln. -- Andrei
Sep 04 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 I recently implemented __ctfeWriteln.
Nice, is it only for your interpreter or can we move https://trello.com/c/6nU0lbl2/24-ctfewrite to done? I think __ctfeWrite would be a better primitive. And we could actually consider to specialize std.stdio.write* for CTFE.
Sep 06 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 6 September 2016 at 10:42:00 UTC, Martin Nowak wrote:
 On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 I recently implemented __ctfeWriteln.
Nice, is it only for your interpreter or can we move https://trello.com/c/6nU0lbl2/24-ctfewrite to done? I think __ctfeWrite would be a better primitive. And we could actually consider to specialize std.stdio.write* for CTFE.
It's only for the current engine and only for Strings! See: https://github.com/dlang/druntime/pull/1643 and https://github.com/dlang/dmd/pull/6101
Sep 06 2016
parent reply Rory McGuire via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, Sep 6, 2016 at 7:42 PM, Stefan Koch via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 6 September 2016 at 10:42:00 UTC, Martin Nowak wrote:

 On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:

 I recently implemented __ctfeWriteln.
Nice, is it only for your interpreter or can we move https://trello.com/c/6nU0lbl2/24-ctfewrite to done? I think __ctfeWrite would be a better primitive. And we could actually consider to specialize std.stdio.write* for CTFE.
It's only for the current engine and only for Strings! See: https://github.com/dlang/druntime/pull/1643 and https://github.com/dlang/dmd/pull/6101
Seriously Stefan, you make my day! My libraries will be so much easier to write!
Sep 06 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 7 September 2016 at 06:49:09 UTC, Rory McGuire 
wrote:
 Seriously Stefan, you make my day!

 My libraries will be so much easier to write!
I am glad the time was not wasted. Let's hope it gets merged :)
Sep 07 2016
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 Hi Guys.

 I recently implemented __ctfeWriteln.
 Based on that experience I have now implemented another pseudo 
 function called __ctfeTicksMs.
 That evaluates to a uint representing the number of 
 milliseconds elapsed between the start of dmd and the time of 
 semantic evaluation of this expression.
For bigger CTFE programs it might be helpful. Milliseconds are a fairly low resolution, would think with hnsec or so makes a better unit. Using core.time.TickDuration for that would make sense.
Sep 06 2016
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tuesday, September 06, 2016 10:46:11 Martin Nowak via Digitalmars-d wrote:
 On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 Hi Guys.

 I recently implemented __ctfeWriteln.
 Based on that experience I have now implemented another pseudo
 function called __ctfeTicksMs.
 That evaluates to a uint representing the number of
 milliseconds elapsed between the start of dmd and the time of
 semantic evaluation of this expression.
For bigger CTFE programs it might be helpful. Milliseconds are a fairly low resolution, would think with hnsec or so makes a better unit. Using core.time.TickDuration for that would make sense.
If you're doing to do that use core.time.Duration. TickDuration is slated to be deprecated once the functionality in Phobos that uses it has been deprecated. Duration replaces its functionality as a duration, and MonoTime replaces its functionality as a timestamp of the monotonic clock. - Jonathan M Davis
Sep 06 2016
prev sibling parent reply safety0ff <safety0ff.dev gmail.com> writes:
On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 ... I have now implemented another pseudo function called 
 __ctfeTicksMs.
 [Snip]

 This does allow meaningful compiletime performance tests to be 
 written.
 spanning both CTFE and template-incitations timeings.

 Please tell me what you think.
I think automated ctfe profiling would be much better and the byte-code interpreter seems like a great platform to build this onto. For example, using a command line switch to enable profiling which outputs something similar to gprof's flat profile. Skimming the byte-code work it seems like it is too early to add this yet. My thoughts on __ctfeTicksMs: - it isn't very meaningful for users without intimate compiler knowledge - it requires writing boilerplate code over and over for profiling - doesn't seem like it would work well for functions that get executed multiple times While it might be a useful compiler developer hack, I do not think it should become a user primitive.
Sep 08 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 8 September 2016 at 16:52:47 UTC, safety0ff wrote:
 On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
 ... I have now implemented another pseudo function called 
 __ctfeTicksMs.
 [Snip]

 This does allow meaningful compiletime performance tests to be 
 written.
 spanning both CTFE and template-incitations timeings.

 Please tell me what you think.
I think automated ctfe profiling would be much better and the byte-code interpreter seems like a great platform to build this onto. For example, using a command line switch to enable profiling which outputs something similar to gprof's flat profile.
I thought of the same thing a while back. However I have had the time to decipher the gprof data-format yet. Is there another profile-format for decent visualization tools exist ?
Sep 08 2016
parent reply safety0ff <safety0ff.dev gmail.com> writes:
On Thursday, 8 September 2016 at 17:03:30 UTC, Stefan Koch wrote:
 I thought of the same thing a while back.
 However I have had the time to decipher the gprof data-format 
 yet.
 Is there another profile-format for decent visualization tools 
 exist ?
I was just using that as an example of what we might want to output as text. e.g. https://sourceware.org/binutils/docs/gprof/Flat-Profile.html I wasn't saying that we should mimic gmon.out file format, I don't think that buys us much.
Sep 08 2016
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 8 September 2016 at 17:15:54 UTC, safety0ff wrote:
 On Thursday, 8 September 2016 at 17:03:30 UTC, Stefan Koch 
 wrote:
 I thought of the same thing a while back.
 However I have had the time to decipher the gprof data-format 
 yet.
 Is there another profile-format for decent visualization tools 
 exist ?
I was just using that as an example of what we might want to output as text. e.g. https://sourceware.org/binutils/docs/gprof/Flat-Profile.html I wasn't saying that we should mimic gmon.out file format, I don't think that buys us much.
I disagree anything which allows to use existing visualization and correlation will be a major win. If I am going to write a profiler it should have pretty charts. Also the gnu guys probably put a lot thought into their format.
Sep 08 2016