www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Request for Comment assert(__ctfe)

reply Stefan Koch <uplink.coder googlemail.com> writes:
Hi Guys,

I am currently working on speeding up dmd. In the presence of 
templates and ctfe.
and one thing that stood out is that we do codegen for symbols 
which are never used.
(and hopefully eliminated by a smart linker but eh ...)

I've seen the following pattern in the wild.


string GenerateMixin(string[] params)
{
     assert(__ctfe);
     .... more code ....
}

that means if we see assert(__ctfe) inside the outermost scope of 
a function body ( not in a branch) then we know that this 
function would assert if called at runtime.
And therefore no correct code can be calling it.
which also means we can forgo having code-gen or optimization for 
the function body.
and reduce the count of names which has to get mangled.

They only downside to this, is that giving assert(__ctfe) the 
special meaning to skip codegen might be confusing to some people 
.... then again you wouldn't use assert(__ctfe) unless you expect 
that function to not be available at run-time

What do you guys think?

Cheers,

Stefan
Apr 05 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,
 ....
 What do you guys think?


One more thing.
The implementation of this is not that hard, and almost done 
already.
(just in case you're getting trigger happy Walter. )
Apr 05 2020
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/5/2020 5:13 AM, Stefan Koch wrote:

 (just in case you're getting trigger happy Walter. )


I'll try not to shoot myself in the foot.
Apr 06 2020
prev sibling next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,
 ....
 What do you guys think?


I had a similar thought some time ago:
https://github.com/dlang/DIPs/pull/177#issuecomment-539894029

;)
Apr 05 2020
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 What do you guys think?


good enough for me.
Apr 05 2020
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 [...]

 What do you guys think?

 Cheers,

 Stefan


I'd prefer a new func attribute ` ctfe`.
Also I find the current implementation unplesant. Cant you just 
check the assertion when they are already visited elsewhere e.g 
in dmd.expressionsem.ExpressionSemanticVisitor.visit(AssertExp) ? 
Also what if we are in a contract ?
Apr 05 2020
next sibling parent reply Stefan Koch <uplink.coder gmail.com> writes:
On Sunday, 5 April 2020 at 13:26:22 UTC, Basile B. wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 [...]

 What do you guys think?

 Cheers,

 Stefan


 I'd prefer a new func attribute ` ctfe`.
 Also I find the current implementation unplesant. Cant you just 
 check the assertion when they are already visited elsewhere e.g 
 in dmd.expressionsem.ExpressionSemanticVisitor.visit(AssertExp) 
 ? Also what if we are in a contract ?


Don't we have enough attributes already?

I _could_ but then it's hard to know if I am in the top-level of 
the function body.
Apr 05 2020
parent Basile B. <b2.temp gmx.com> writes:
On Sunday, 5 April 2020 at 14:04:12 UTC, Stefan Koch wrote:

 On Sunday, 5 April 2020 at 13:26:22 UTC, Basile B. wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 [...]

 What do you guys think?

 Cheers,

 Stefan


 I'd prefer a new func attribute ` ctfe`.
 Also I find the current implementation unplesant. Cant you 
 just check the assertion when they are already visited 
 elsewhere e.g in 
 dmd.expressionsem.ExpressionSemanticVisitor.visit(AssertExp) ? 
 Also what if we are in a contract ?


 Don't we have enough attributes already?


I don't know. The criticism I have aginst func attributes is more 
that their style are mixed (i.e w/ or w/o ' '). Anyway I'm not 
strongly against your way, it's just that in a perfect world I 
imagine  ctfe rather.
Apr 05 2020
prev sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Sunday, 5 April 2020 at 13:26:22 UTC, Basile B. wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 [...]

 What do you guys think?

 Cheers,

 Stefan


 I'd prefer a new func attribute ` ctfe`.
 Also I find the current implementation unplesant. Cant you just 
 check the assertion when they are already visited elsewhere e.g 
 in dmd.expressionsem.ExpressionSemanticVisitor.visit(AssertExp) 
 ? Also what if we are in a contract ?


I agree that a function attribute makes more sense for a feature 
like this, however, this solution doesn't solve the general case 
(i.e. elide functions that are only called at compile-time but 
don't have assert(__ctfe) because they are available to be called 
at runtime).  Given that, I believe solving the general case is 
only a matter of implementation rather than a language change.  
So I would hate to add a new language feature that would become 
obselete once the tooling is better.

So I say yes, I'd be fine with this "hack" to temporarily provide 
a path for programs to create CTFE-ONLY functions that won't go 
to code generation.  It's better than the status-quo and can be 
removed if we ever solve the general case.
Apr 05 2020
parent Johannes Pfau <nospam example.com> writes:
Am Sun, 05 Apr 2020 16:00:42 +0000 schrieb Jonathan Marler:


 
 So I say yes, I'd be fine with this "hack" to temporarily provide a path
 for programs to create CTFE-ONLY functions that won't go to code
 generation.  It's better than the status-quo and can be removed if we
 ever solve the general case.


The compiler can really not know if a function is supposed to be callable 
at runtime, see https://forum.dlang.org/post/
r6hotc$28au$2 digitalmars.com for details. It could probably work if we 
decide to introduce 'export', but there was no consensus on that for 
years.

-- 
Johannes
Apr 07 2020
prev sibling next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 They only downside to this, is that giving assert(__ctfe) the 
 special meaning to skip codegen might be confusing to some 
 people .... then again you wouldn't use assert(__ctfe) unless 
 you expect that function to not be available at run-time

 What do you guys think?


Ideally it should only skip codegen if there are no statements 
with side-effects before the assert(__ctfe). However, I think 
it's fine to do it your way if that's easier to implement in the 
compiler, as long as normal code gen and optimization is still 
enabled for functions containing this construct, as a workaround:

if(!__ctfe) assert(0);
Apr 05 2020
parent Stefan Koch <uplink.coder gmail.com> writes:
On Sunday, 5 April 2020 at 13:54:19 UTC, tsbockman wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 They only downside to this, is that giving assert(__ctfe) the 
 special meaning to skip codegen might be confusing to some 
 people .... then again you wouldn't use assert(__ctfe) unless 
 you expect that function to not be available at run-time

 What do you guys think?


 Ideally it should only skip codegen if there are no statements 
 with side-effects before the assert(__ctfe). However, I think 
 it's fine to do it your way if that's easier to implement in 
 the compiler, as long as normal code gen and optimization is 
 still enabled for functions containing this construct, as a 
 workaround:

 if(!__ctfe) assert(0);


So it's much easier to just scan for assert(__ctfe);

I've just implemented it and the projects on the dlang buildkite 
are green.
Apr 05 2020
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/5/20 8:11 AM, Stefan Koch wrote:

 What do you guys think?


It seems like a good idea, except what you will see is a linker error if 
you call it. Which is not very friendly or gives the right impression.

Is there a way we can have the compiler give an error when it sees this 
during compile-time?

-Steve
Apr 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 5 April 2020 at 14:25:27 UTC, Steven Schveighoffer 
wrote:

 On 4/5/20 8:11 AM, Stefan Koch wrote:

 What do you guys think?


 It seems like a good idea, except what you will see is a linker 
 error if you call it. Which is not very friendly or gives the 
 right impression.

 Is there a way we can have the compiler give an error when it 
 sees this during compile-time?

 -Steve


Yes. that is possible.

However actually it never happens :)
Apr 05 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/5/20 10:26 AM, Stefan Koch wrote:

 On Sunday, 5 April 2020 at 14:25:27 UTC, Steven Schveighoffer wrote:

 On 4/5/20 8:11 AM, Stefan Koch wrote:

 What do you guys think?


 It seems like a good idea, except what you will see is a linker error 
 if you call it. Which is not very friendly or gives the right impression.

 Is there a way we can have the compiler give an error when it sees 
 this during compile-time?


 
 Yes. that is possible.


I'd be fully on board then.


 
 However actually it never happens :)


What do you mean it never happens?

I get that code that builds and runs correctly today will build with 
this change. But development is full of writing code that doesn't work, 
running the compiler, and fixing the errors shown. When the error is a 
link error, this will be completely confusing.

-Steve
Apr 05 2020
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,
 ...
 What do you guys think?


Another, more radical and interesting option is to not perform 
codegen unless it's actually necessary. Instead of blacklisting 
` ctfe` or `assert(__ctfe)` functions, have a whitelist of what 
functions are allowed to codegen-ed.

Specifically, do not perform codegen for any function unless that 
function:

A) is marked, or inferred as `export` or called from such 
function (transitively)
B) is part of an aggregate (module, union, class, struct, 
interface, template) marked as `export`, or called from a 
function part of such
(By module marked as `export` I mean `export module foo;`, as 
opposed to `module foo; export:`)
C) is `main`, or it's called from `main` (in these cases it's 
inferred as `export`)
D) is a `unittest`, or called from one (under `version(unittest)` 
`unittest`s (and functions they call) are inferred as `export`)
E) is marked as `extern($lang)` or called from such function 
(again inferred as `export` in such cases)
F) is virtual, or called from such function (again inferred as 
`export` in those cases)

`export` must be a function storage class orthogonal to 
protection modifiers, just like `extern`.

Yeah, it will break the world, so this new compilation strategy 
should be opt-in via a switch (e.g. `-ix`, `-lazy`, 
`--as-needed`, etc.). I suspect it will play nicely with the 
existing `-i` ;)

Implications? From top of my head:
1. No need for a new ` cttfe` function attribute, or to treat 
`assert(__ctfe)` in special way. Though, on second thoughts, 
perhaps it should be an error to have non-conditional 
`assert(__ctfe)` inside `export`-ed functions.
2. Regular functions gain some of the special powers of template 
ones - specifically, they're codegen-ed only if needed. Before, 
to make druntime/phobos functions accessible from DasBetterC we 
had to make them templates. Now this awkward technique (what's 
the point of having a template with no parameters? Non-template 
auto-returning functions also have attribute inference) is no 
longer needed.
3. Compilation speed and code bloat are reduced
4. D libraries become kind of like header-only C++ libraries.
5. Until now object files (or static libraries) were a primary 
build caching technique. Now a per-function caching technique 
would be much more helpful.
Apr 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 5 April 2020 at 18:25:10 UTC, Petar Kirov [ZombineDev] 
wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 [...]


 Another, more radical and interesting option is to not perform 
 codegen unless it's actually necessary. Instead of blacklisting 
 ` ctfe` or `assert(__ctfe)` functions, have a whitelist of what 
 functions are allowed to codegen-ed.

 [...]


Seems like you are reading my mind.
Indeed this is one of next steps ...
assert(__ctfe) did not break a single package, in out CI!
Therefore I do now have enough confidence in crawling the dynamic 
dependencies outwards from the entry point and exported functions.

That however is for the future and will require me to do some 
theoretical work, in order to proof to myself that it actually is 
feasible and won't end up in a nightmare of subtly broken builds.
Apr 05 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 5 April 2020 at 18:30:25 UTC, Stefan Koch wrote:

 On Sunday, 5 April 2020 at 18:25:10 UTC, Petar Kirov 
 [ZombineDev] wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 [...]


 Another, more radical and interesting option is to not perform 
 codegen unless it's actually necessary. Instead of 
 blacklisting ` ctfe` or `assert(__ctfe)` functions, have a 
 whitelist of what functions are allowed to codegen-ed.

 [...]


 Seems like you are reading my mind.
 Indeed this is one of next steps ...
 assert(__ctfe) did not break a single package, in out CI!
 Therefore I do now have enough confidence in crawling the 
 dynamic dependencies outwards from the entry point and exported 
 functions.

 That however is for the future and will require me to do some 
 theoretical work, in order to proof to myself that it actually 
 is feasible and won't end up in a nightmare of subtly broken 
 builds.


Great! Perhaps we should write a DIP to specify the semantics of 
such a lazy compilation model? An even more extreme option would 
be to not perform semantic analysis on non-`export`-ed functions. 
This should bring some really high gains, though it would be the 
trickiest to adopt option for users.
Apr 05 2020
parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 05 Apr 2020 18:40:31 +0000 schrieb Petar Kirov [ZombineDev]:


 On Sunday, 5 April 2020 at 18:30:25 UTC, Stefan Koch wrote:

 On Sunday, 5 April 2020 at 18:25:10 UTC, Petar Kirov [ZombineDev]
 wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 [...]


 Another, more radical and interesting option is to not perform codegen
 unless it's actually necessary. Instead of blacklisting ` ctfe` or
 `assert(__ctfe)` functions, have a whitelist of what functions are
 allowed to codegen-ed.

 [...]


 Seems like you are reading my mind.
 Indeed this is one of next steps ...
 assert(__ctfe) did not break a single package, in out CI! Therefore I
 do now have enough confidence in crawling the dynamic dependencies
 outwards from the entry point and exported functions.

 That however is for the future and will require me to do some
 theoretical work, in order to proof to myself that it actually is
 feasible and won't end up in a nightmare of subtly broken builds.


 
 Great! Perhaps we should write a DIP to specify the semantics of such a
 lazy compilation model? An even more extreme option would be to not
 perform semantic analysis on non-`export`-ed functions.
 This should bring some really high gains, though it would be the
 trickiest to adopt option for users.


How would such a lazy compilation model work without explicit 'export' 
anotations? If I wanted to build a proprietary library libfoo.so and 
provide only .di files with stripped function bodies, all functions have 
to be compiled into the .so file. But how does the compiler know the 
result is going to be a shared library in a separate compilation model, 
where the compiler produces only an object file?

I guess such a lazy compilation model might be quite interesting, but as 
it is a quite radical idea, it needs lots of thinking, proper 
specification and testing first. (Testing as you'll probably have to emit 
everything as weak symbols then, and you probably want to test how 
linkers deal with that if you make excessive use of it)


-- 
Johannes
Apr 07 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 7 April 2020 at 17:33:03 UTC, Johannes Pfau wrote:

 On Sunday, 5 April 2020 at 18:25:10 UTC, Petar Kirov 
 [ZombineDev] wrote:

 Another, more radical and interesting option is to not 
 perform codegen unless it's actually necessary. Instead of 
 blacklisting ` ctfe` or `assert(__ctfe)` functions, have a 
 whitelist of what functions are allowed to codegen-ed.

 [...]






 How would such a lazy compilation model work without explicit 
 'export' animations?


In my post [1] I said that the `export` storage class is the key 
to making this work. The way I envision this is similar to a 
copying garbage collector:
1. [MARK] Starting from the set of ~GC~ CG roots [2] the compiler 
traverses the static call graph [3] and marks any callees as 
`export` (i.e. the `export` inference step in my proposal). All 
functions determined to have the `export` attribute are added to 
the code generation queue
2. [COPY] Do code generation by poping function from the queue 
until it becomes empty.
3. (Optional) produce a report of all functions that were not 
added to the queue.

[1]: 
https://forum.dlang.org/post/xbllqrpvflazfpowizwj forum.dlang.org
[2]: CG (call graph) roots: all functions marked explicitly as 
`export`, or implicitly under rules B) - F) in my post [1] (e.g. 
`main()`, `unittest`s, virtual functions, etc.)
[3]: After statements inserted from `static`, `version` and 
`mixin` declarations are resolved, the set of all functions that 
may be called from a given function. Even if a function is called 
conditionally at runtime, it's still part of the static call 
graph.


 If I wanted to build a proprietary library libfoo.so and 
 provide only .di files with stripped function bodies, all 
 functions have to be compiled into the .so file. But how does 
 the compiler know the result is going to be a shared library in 
 a separate compilation model, where the compiler produces only 
 an object file?


To produce the libfoo.{so,dll,dylib}, I assume that you have a 
whole package of D modules that you want to compile.

---

(The case where libfoo is produced from a single D module is the 
trivial base case: only functions determined to be `export` will 
be part of the object file, from which libfoo will be produced.)

Also, as in C++ (AFAIU), templates can't be part of your dynamic 
library ABI, only specific template instances can be. If type 
templates participate in the function signature (the parameter 
types, or return type) of an `export`-ed function, then those 
type templates are inferred to be `export` as well (and all of 
their members). I suspect we can do better here, but it's easier 
to start pessimistically to relax the rules later when we have a 
better formal model an empirical experience.

---

So, finally, how would we produce a dynamic library?

1. In our implementation files (*.d) we would mark as `export` 
all public functions that we want to be part of our dynamic 
library API/ABI.

2. When the compiler produces *.di header files it would include 
the declarations of only public, protected or package symbols 
that are determined to be export.

3. When the object files are produced all `export` functions will 
be compiled (including private ones, i.e. private symbols, called 
from public exported ones).

4. The rest of the symbols that are not determined to be `export` 
are discarded.

---

All in all, what we're left with is:
1. A strict compilation mode in which only symbols determined to 
be `export` are included in libraries. This way we have a much 
more precise definition of ABI.
2. A lazy compilation mode, based on (dmd -i) in which the 
compiler automatically imports and compiles functions as needed, 
starting from `main()`. In a package-wise compilation (not 
separate compilation), regular free functions are treated like 
template functions - they're not compiled unless used.

Actually 1. and 2. are exactly the same compilation strategy, 
just very different use cases ;)


 I guess such a lazy compilation model might be quite 
 interesting, but as it is a quite radical idea, it needs lots 
 of thinking, proper specification and testing first. (Testing 
 as you'll probably have to emit everything as weak symbols 
 then, and you probably want to test how linkers deal with that 
 if you make excessive use of it)


I don't think we should rely that much on linkers for object 
file-level GC. The compiler shouldn't produce functions that are 
not determined to be `export` in the first place.


Some examples:

--------------------
module app;

string generateMixin(T)(string b)
{
     assert(__ctfe);
     return T.stringof ~ " " ~ b ~ ";";
}

void main()  nogc
{
     mixin(generateMixin!string("b"));
}
--------------------

`dmd -c app.d` should produce an object file with a single 
(empty) function - `main()`;

---------------------------
module foo;

private void privateFunc() {}

public void publicTemplate(T)()
{
     privateFunc();
}
---------------------------

Q: How would this work?
A: Uner my proposal, the compiler should produce an **empty** 
object file.

What could work is the following:

---------------------------
module foo;

private void privateFunc() {}

public void publicTemplate(T)()
{
     privateFunc();
}

// the obvious way:
export void publicFunc() { publicTemplate!int(); }

// the syntax sugar way 1a:
export alias publicFunc = publicTemplate!int;

// the syntax sugar way 1b:
export
{
     alias publicFunc = publicTemplate!int;
     alias publicFunc = publicTemplate!string;
}

// the syntax sugar way 2:
export
{
     publicTemplate!int;
     publicTemplate!string;
}

// Note: the syntax sugar is not part of my proposal,
// as it's not important for now.
---------------------------

Or:

---------------------------
module foo;

private void privateFunc() {}

public void publicTemplate(T)()
{
     privateFunc();
}

----
module bar;

import foo;

export void actuallyPublicInterfaceFunction()
{
     publicTemplate!int();
}

---

dmd -i bar.d

---------------------------

In this way we tell the compiler to compile the exported function 
in `bar` as well as any other functions that they may call.
Apr 08 2020
parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 08 Apr 2020 15:07:32 +0000 schrieb Petar Kirov [ZombineDev]:


 ---------------------------
 
 In this way we tell the compiler to compile the exported function in
 `bar` as well as any other functions that they may call.


Thanks for taking the time to write such a detailed answer. I think I'll 
have to spend some more time thinking about this to convince myself that 
it really covers all cases.

Anyway, a very interesting idea.

-- 
Johannes
Apr 08 2020
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 8 April 2020 at 20:07:49 UTC, Johannes Pfau wrote:

 Am Wed, 08 Apr 2020 15:07:32 +0000 schrieb Petar Kirov 
 [ZombineDev]:


 ---------------------------
 
 In this way we tell the compiler to compile the exported 
 function in `bar` as well as any other functions that they may 
 call.


 Thanks for taking the time to write such a detailed answer. I 
 think I'll have to spend some more time thinking about this to 
 convince myself that it really covers all cases.

 Anyway, a very interesting idea.


Thanks! I do realize that D compilation is a way more complicated 
process than my simplified model. My idea needs to be better 
formalized and to endure more scrutiny ;)
Apr 08 2020
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05.04.20 14:11, Stefan Koch wrote:

 Hi Guys,
 
 I am currently working on speeding up dmd. In the presence of templates 
 and ctfe.
 and one thing that stood out is that we do codegen for symbols which are 
 never used.
 (and hopefully eliminated by a smart linker but eh ...)
 
 I've seen the following pattern in the wild.
 
 
 string GenerateMixin(string[] params)
 {
      assert(__ctfe);
      .... more code ....
 }
 
 that means if we see assert(__ctfe) inside the outermost scope of a 
 function body ( not in a branch) then we know that this function would 
 assert if called at runtime.


Make sure it works with `in` contracts:

string generateMixin(string[] params)in{
     assert(__ctfe);
}do{
     .... more code ....
}

(The plain assertion slightly hurts my eyes, because if it fails, that 
failure would be attributable to the GenerateMixin function, whereas 
with the `in` contract it is clear that the caller is at fault.)
Apr 05 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 5 April 2020 at 20:31:15 UTC, Timon Gehr wrote:

 On 05.04.20 14:11, Stefan Koch wrote:

 [...]


 Make sure it works with `in` contracts:

 string generateMixin(string[] params)in{
     assert(__ctfe);
 }do{
     .... more code ....
 }

 (The plain assertion slightly hurts my eyes, because if it 
 fails, that failure would be attributable to the GenerateMixin 
 function, whereas with the `in` contract it is clear that the 
 caller is at fault.)


Oh good idea.
I don't use contracts myself but that's easy to add.
Apr 05 2020
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05.04.20 22:39, Stefan Koch wrote:

 On Sunday, 5 April 2020 at 20:31:15 UTC, Timon Gehr wrote:

 On 05.04.20 14:11, Stefan Koch wrote:

 [...]


 Make sure it works with `in` contracts:

 string generateMixin(string[] params)in{
     assert(__ctfe);
 }do{
     .... more code ....
 }

 (The plain assertion slightly hurts my eyes, because if it fails, that 
 failure would be attributable to the GenerateMixin function, whereas 
 with the `in` contract it is clear that the caller is at fault.)


 
 Oh good idea.
 I don't use contracts myself but that's easy to add.


Great, thanks! :)
Apr 05 2020
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Sunday, 5 April 2020 at 20:31:15 UTC, Timon Gehr wrote:

 On 05.04.20 14:11, Stefan Koch wrote:

 Hi Guys,
 
 I am currently working on speeding up dmd. In the presence of 
 templates and ctfe.
 and one thing that stood out is that we do codegen for symbols 
 which are never used.
 (and hopefully eliminated by a smart linker but eh ...)
 
 I've seen the following pattern in the wild.
 
 
 string GenerateMixin(string[] params)
 {
      assert(__ctfe);
      .... more code ....
 }
 
 that means if we see assert(__ctfe) inside the outermost scope 
 of a function body ( not in a branch) then we know that this 
 function would assert if called at runtime.


 Make sure it works with `in` contracts:

 string generateMixin(string[] params)in{
     assert(__ctfe);
 }do{
     .... more code ....
 }

 (The plain assertion slightly hurts my eyes, because if it 
 fails, that failure would be attributable to the GenerateMixin 
 function, whereas with the `in` contract it is clear that the 
 caller is at fault.)


We likely get the shorthand version for free as well:
string generateMixin(...)
in(__ctfe)
{
    ...
}
Apr 05 2020
parent reply Stefan Koch <uplink.coder gmail.com> writes:
On Sunday, 5 April 2020 at 20:54:03 UTC, Meta wrote:

 On Sunday, 5 April 2020 at 20:31:15 UTC, Timon Gehr wrote:

 On 05.04.20 14:11, Stefan Koch wrote:

 [...]


 Make sure it works with `in` contracts:

 string generateMixin(string[] params)in{
     assert(__ctfe);
 }do{
     .... more code ....
 }

 (The plain assertion slightly hurts my eyes, because if it 
 fails, that failure would be attributable to the GenerateMixin 
 function, whereas with the `in` contract it is clear that the 
 caller is at fault.)


 We likely get the shorthand version for free as well:
 string generateMixin(...)
 in(__ctfe)
 {
    ...
 }


No. the in contract was already a pain in the butt.
Apr 05 2020
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Sunday, 5 April 2020 at 22:12:02 UTC, Stefan Koch wrote:

 On Sunday, 5 April 2020 at 20:54:03 UTC, Meta wrote:

 We likely get the shorthand version for free as well:
 string generateMixin(...)
 in(__ctfe)
 {
    ...
 }


 No. the in contract was already a pain in the butt.


That IS an in contract. idk about the implementation but from the 
user level this is identical to in { assert(__ctfe);}
Apr 05 2020
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06.04.20 00:12, Stefan Koch wrote:

 On Sunday, 5 April 2020 at 20:54:03 UTC, Meta wrote:

 On Sunday, 5 April 2020 at 20:31:15 UTC, Timon Gehr wrote:

 On 05.04.20 14:11, Stefan Koch wrote:

 [...]


 Make sure it works with `in` contracts:

 string generateMixin(string[] params)in{
     assert(__ctfe);
 }do{
     .... more code ....
 }

 (The plain assertion slightly hurts my eyes, because if it fails, 
 that failure would be attributable to the GenerateMixin function, 
 whereas with the `in` contract it is clear that the caller is at fault.)


 We likely get the shorthand version for free as well:
 string generateMixin(...)
 in(__ctfe)
 {
    ...
 }


 
 No. the in contract was already a pain in the butt.


I don't understand how you would implement it in a way where the above 
does not work. Have you tested it? I am literally rewriting `in(__ctfe)` 
to `in{ assert(__ctfe); }` in the parser:

https://github.com/dlang/dmd/blob/master/src/dmd/parse.d#L5060
Apr 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 6 April 2020 at 01:28:19 UTC, Timon Gehr wrote:

 I don't understand how you would implement it in a way where 
 the above does not work. Have you tested it? I am literally 
 rewriting `in(__ctfe)` to `in{ assert(__ctfe); }` in the parser:

 https://github.com/dlang/dmd/blob/master/src/dmd/parse.d#L5060


I see. Well in that case it'll work out of the box.
Although it does make me shudder to change the Tree in the parser 
in such a way.
When was this merged?
Apr 05 2020
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06.04.20 04:25, Stefan Koch wrote:

 On Monday, 6 April 2020 at 01:28:19 UTC, Timon Gehr wrote:

 I don't understand how you would implement it in a way where the above 
 does not work. Have you tested it? I am literally rewriting 
 `in(__ctfe)` to `in{ assert(__ctfe); }` in the parser:

 https://github.com/dlang/dmd/blob/master/src/dmd/parse.d#L5060


 
 I see. Well in that case it'll work out of the box.
 Although it does make me shudder to change the Tree in the parser in 
 such a way.


For no reason at all. It's the appropriate place to expand simple 
short-hand notation, e.g., see template functions.

It's just good engineering. Let's consider the evidence:

What you expected to be the case:

- `in(expr)` is a special kind of construct in the AST, burdening the 
entire compiler code base from frontend to backend.
- `in{ assert(__ctfe); }` and `in(__ctfe)` have diverging behavior, a 
unfortunate development caused by butt pain.

What was actually the case:

- `in(expr)` is simple shorthand notation and is treated by the compiler 
precisely like `in{ assert(__ctfe); }`
- `in(__ctfe)` works precisely like `in{ assert(__ctfe); }` even after 
the latter has been special-cased.

You might want to recalibrate your guts. :P


 When was this merged?


Just when it should have been, right after the corresponding DIP was 
accepted.
Apr 05 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 6 April 2020 at 03:11:41 UTC, Timon Gehr wrote:

 On 06.04.20 04:25, Stefan Koch wrote:

 [...]


 For no reason at all. It's the appropriate place to expand 
 simple short-hand notation, e.g., see template functions.

 [...]


I don't mind the what. I just mind the when. But it's not a big 
deal either way. I would have done it in sema.
Apr 06 2020
parent Johan <j j.nl> writes:
On Monday, 6 April 2020 at 07:33:19 UTC, Stefan Koch wrote:

 On Monday, 6 April 2020 at 03:11:41 UTC, Timon Gehr wrote:

 On 06.04.20 04:25, Stefan Koch wrote:

 [...]


 For no reason at all. It's the appropriate place to expand 
 simple short-hand notation, e.g., see template functions.

 [...]


 I don't mind the what. I just mind the when. But it's not a big 
 deal either way. I would have done it in sema.


Offtopic.
This triggers me, so a quick response: the tree rewriting that 
happens all over the place in the compiler frontend is very 
problematic with anything that has to report back to the user and 
more: debug location information, error reporting, coverage 
reporting, syntax highlighting, source formatting, etc., etc. [*]

-Johan

[*] As an example, even the harmless looking replacement of 
__FILE__ by the parser, turns out to be very problematic for LDC 
for deterministically generating symbol names (linking) when 
combined with crossmodule inlining :(
Apr 06 2020
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 I am currently working on speeding up dmd. In the presence of 
 templates and ctfe.
 and one thing that stood out is that we do codegen for symbols 
 which are never used.
 (and hopefully eliminated by a smart linker but eh ...)

 [...]


Wouldn't it be easier to skip codegen for private functions that 
are never called from non CTFE contexts?
Apr 06 2020
next sibling parent reply Stefan Koch <uplink.coder gmail.com> writes:
On Monday, 6 April 2020 at 10:14:24 UTC, Atila Neves wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 I am currently working on speeding up dmd. In the presence of 
 templates and ctfe.
 and one thing that stood out is that we do codegen for symbols 
 which are never used.
 (and hopefully eliminated by a smart linker but eh ...)

 [...]


 Wouldn't it be easier to skip codegen for private functions 
 that are never called from non CTFE contexts?


Easier from the user-perspective yes.
 From the compiler perspective,
That's another step which may take quite a while to do correctly.
The easy thing would be (Essentially an (N*M) loop over all calls 
and functions), which I doubt will help compile-times much.

For now assert(__ctfe); enables people to manually mark their 
ctfe-only, functions.
Apr 06 2020
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 6 April 2020 at 13:41:52 UTC, Stefan Koch wrote:

 On Monday, 6 April 2020 at 10:14:24 UTC, Atila Neves wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 [...]


 Wouldn't it be easier to skip codegen for private functions 
 that are never called from non CTFE contexts?


 Easier from the user-perspective yes.
 From the compiler perspective,
 That's another step which may take quite a while to do 
 correctly.
 The easy thing would be (Essentially an (N*M) loop over all 
 calls and functions),


Where N and M are all calls and private functions in one module, 
not all code in a project.


 For now assert(__ctfe); enables people to manually mark their 
 ctfe-only, functions.


What about `in(__ctfe)` instead?
Apr 06 2020
next sibling parent Stefan Koch <uplink.coder gmail.com> writes:
On Monday, 6 April 2020 at 13:49:19 UTC, Atila Neves wrote:

 On Monday, 6 April 2020 at 13:41:52 UTC, Stefan Koch wrote:

 On Monday, 6 April 2020 at 10:14:24 UTC, Atila Neves wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 [...]


 Wouldn't it be easier to skip codegen for private functions 
 that are never called from non CTFE contexts?


 Easier from the user-perspective yes.
 From the compiler perspective,
 That's another step which may take quite a while to do 
 correctly.
 The easy thing would be (Essentially an (N*M) loop over all 
 calls and functions),


 Where N and M are all calls and private functions in one 
 module, not all code in a project.


 For now assert(__ctfe); enables people to manually mark their 
 ctfe-only, functions.


 What about `in(__ctfe)` instead?


in (__ctfe) is already supported.
Apr 06 2020
prev sibling next sibling parent reply Dominikus Dittes Scherkl <dominikus.scherkl continental-corporation.com> writes:
On Monday, 6 April 2020 at 13:49:19 UTC, Atila Neves wrote:

 On Monday, 6 April 2020 at 13:41:52 UTC, Stefan Koch wrote:

 For now assert(__ctfe); enables people to manually mark their 
 ctfe-only, functions.


 What about `in(__ctfe)` instead?


It was already explained, this will expand to the same code, so 
yes, both will work.
Apr 06 2020
parent Stefan Koch <uplink.coder gmail.com> writes:
On Monday, 6 April 2020 at 14:14:48 UTC, Dominikus Dittes Scherkl 
wrote:

 On Monday, 6 April 2020 at 13:49:19 UTC, Atila Neves wrote:

 On Monday, 6 April 2020 at 13:41:52 UTC, Stefan Koch wrote:

 For now assert(__ctfe); enables people to manually mark their 
 ctfe-only, functions.


 What about `in(__ctfe)` instead?


 It was already explained, this will expand to the same code, so 
 yes, both will work.


Well technically I had to do a bit of work for InContracts 
("Requires" in dmd lingo), because for them you need to unwrap 
the scopeStatements.
Apr 06 2020
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 06 Apr 2020 13:49:19 +0000 schrieb Atila Neves:


 On Monday, 6 April 2020 at 13:41:52 UTC, Stefan Koch wrote:

 On Monday, 6 April 2020 at 10:14:24 UTC, Atila Neves wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 [...]


 Wouldn't it be easier to skip codegen for private functions that are
 never called from non CTFE contexts?


 Easier from the user-perspective yes.
 From the compiler perspective,
 That's another step which may take quite a while to do correctly.
 The easy thing would be (Essentially an (N*M) loop over all calls and
 functions),


 
 Where N and M are all calls and private functions in one module,
 not all code in a project.
 


In fact, you can't do that in current D. I thought about it some time 
ago, consider this valid D code:

---------------------------
module foo;

private void privateFunc() {}

public void publicTemplate(T)()
{
    privateFunc();
}
---------------------------

Let's say I compile module foo into libfoo. libfoo internally never uses 
publicTemplate. An external user instantiates publicTemplate ==> 
privateFunc needs to be emitted in libfoo.

Now you can't even check that privateFunc is never called, cause you 
can't analyze the template without instantiating it. And in general, you 
can't instantiate it without knowing the template instance type 
parameters. (In this simple example you might think this should work. But 
just imagine that I can pass in a type with an enum string member, which 
I can then mixin in the template, ....)


Generally the only way to solve this seems to finally implement export 
properly, as it was proposed years ago. Then privateFunc should be 
flagged export (needed for windows DLL support and linux .so selective 
exports anyway) and you could probably strip uncalled, non-exported 
private functions.


TLDR: It's certainly not easy to implement.

-- 
Johannes
Apr 07 2020
parent Jacob Carlborg <doob me.com> writes:
On 2020-04-07 13:41, Johannes Pfau wrote:

 Am Mon, 06 Apr 2020 13:49:19 +0000 schrieb Atila Neves:
 

 On Monday, 6 April 2020 at 13:41:52 UTC, Stefan Koch wrote:

 On Monday, 6 April 2020 at 10:14:24 UTC, Atila Neves wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 [...]


 Wouldn't it be easier to skip codegen for private functions that are
 never called from non CTFE contexts?


 Easier from the user-perspective yes.
  From the compiler perspective,
 That's another step which may take quite a while to do correctly.
 The easy thing would be (Essentially an (N*M) loop over all calls and
 functions),


 Where N and M are all calls and private functions in one module,
 not all code in a project.


 
 In fact, you can't do that in current D. I thought about it some time
 ago, consider this valid D code:
 
 ---------------------------
 module foo;
 
 private void privateFunc() {}
 
 public void publicTemplate(T)()
 {
      privateFunc();
 }
 ---------------------------
 
 Let's say I compile module foo into libfoo. libfoo internally never uses
 publicTemplate. An external user instantiates publicTemplate ==>
 privateFunc needs to be emitted in libfoo.
 
 Now you can't even check that privateFunc is never called, cause you
 can't analyze the template without instantiating it. And in general, you
 can't instantiate it without knowing the template instance type
 parameters. (In this simple example you might think this should work. But
 just imagine that I can pass in a type with an enum string member, which
 I can then mixin in the template, ....)
 
 
 Generally the only way to solve this seems to finally implement export
 properly, as it was proposed years ago. Then privateFunc should be
 flagged export (needed for windows DLL support and linux .so selective
 exports anyway) and you could probably strip uncalled, non-exported
 private functions.
 
 
 TLDR: It's certainly not easy to implement.
 


You can also bypass "private" using __traits(getMemeber) or using an 
extern(D) declaration.

-- 
/Jacob Carlborg
Apr 07 2020
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/6/2020 3:14 AM, Atila Neves wrote:

 Wouldn't it be easier to skip codegen for private functions that are never 
 called from non CTFE contexts?


There's no reason the CTFE-only function can't be global.
Apr 06 2020
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 7 April 2020 at 06:53:18 UTC, Walter Bright wrote:

 On 4/6/2020 3:14 AM, Atila Neves wrote:

 Wouldn't it be easier to skip codegen for private functions 
 that are never called from non CTFE contexts?


 There's no reason the CTFE-only function can't be global.


No, there isn't. But in my experience nearly all functions I 
write that are only ever called from CTFE returning a string to 
be mixed in are private.
Apr 07 2020
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 7 April 2020 at 09:16:34 UTC, Atila Neves wrote:

 On Tuesday, 7 April 2020 at 06:53:18 UTC, Walter Bright wrote:

 There's no reason the CTFE-only function can't be global.


 No, there isn't. But in my experience nearly all functions I 
 write that are only ever called from CTFE returning a string to 
 be mixed in are private.


I've done it both ways. If it can be accomplished easily enough 
might as well support both equally.
Apr 07 2020
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Sounds ok to me, as long as the assert(__ctfe) is the first statement in the 
function body.

Though I would check the linker map file to see if the linker didn't already 
omit functions that were never called.
Apr 06 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 7 April 2020 at 06:55:59 UTC, Walter Bright wrote:

 Sounds ok to me, as long as the assert(__ctfe) is the first 
 statement in the function body.

 Though I would check the linker map file to see if the linker 
 didn't already omit functions that were never called.


In order to support contracts we need to go through all the 
statements in the merged contract .... but I can split that out.
Apr 07 2020
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/7/2020 12:33 AM, Stefan Koch wrote:

 On Tuesday, 7 April 2020 at 06:55:59 UTC, Walter Bright wrote:

 Sounds ok to me, as long as the assert(__ctfe) is the first statement in the 
 function body.

 Though I would check the linker map file to see if the linker didn't already 
 omit functions that were never called.


 
 In order to support contracts we need to go through all the statements in the 
 merged contract .... but I can split that out.


You could move it up earlier in the semantic analysis. When semantic analysis 
starts, simply check the first statement, and set a flag "no code gen" in the 
FuncDeclaration.
Apr 07 2020
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 7 April 2020 at 06:55:59 UTC, Walter Bright wrote:

 Sounds ok to me, as long as the assert(__ctfe) is the first 
 statement in the function body.

 Though I would check the linker map file to see if the linker 
 didn't already omit functions that were never called.


I think the problem with relying on the linker is that the user 
still pays with longer compile times for codegen that isn't 
needed.
Apr 07 2020
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/7/2020 2:17 AM, Atila Neves wrote:

 I think the problem with relying on the linker is that the user still pays
with 
 longer compile times for codegen that isn't needed.


True
Apr 07 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/7/20 5:17 AM, Atila Neves wrote:

 On Tuesday, 7 April 2020 at 06:55:59 UTC, Walter Bright wrote:

 Sounds ok to me, as long as the assert(__ctfe) is the first statement 
 in the function body.

 Though I would check the linker map file to see if the linker didn't 
 already omit functions that were never called.


 
 I think the problem with relying on the linker is that the user still 
 pays with longer compile times for codegen that isn't needed.


Isn't there an issue for better C as well? If a function can't be 
compiled in betterC but is valid for CTFE, then this could be a 
potential solution.

I don't know how this works today.

-Steve
Apr 07 2020
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 7 April 2020 at 21:11:11 UTC, Steven Schveighoffer 
wrote:

 On 4/7/20 5:17 AM, Atila Neves wrote:

 On Tuesday, 7 April 2020 at 06:55:59 UTC, Walter Bright wrote:

 Sounds ok to me, as long as the assert(__ctfe) is the first 
 statement in the function body.

 Though I would check the linker map file to see if the linker 
 didn't already omit functions that were never called.


 
 I think the problem with relying on the linker is that the 
 user still pays with longer compile times for codegen that 
 isn't needed.


 Isn't there an issue for better C as well? If a function can't 
 be compiled in betterC but is valid for CTFE, then this could 
 be a potential solution.

 I don't know how this works today.

 -Steve


Good question, me neither.
Apr 08 2020
parent Johannes Pfau <nospam example.com> writes:
Am Wed, 08 Apr 2020 09:53:24 +0000 schrieb Atila Neves:



 Isn't there an issue for better C as well? If a function can't be
 compiled in betterC but is valid for CTFE, then this could be a
 potential solution.

 I don't know how this works today.

 -Steve


 
 Good question, me neither.


The answer is simple. It simply does not work today ;-)

-- 
Johannes
Apr 08 2020
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Mon, 06 Apr 2020 23:55:59 -0700 schrieb Walter Bright:


 Sounds ok to me, as long as the assert(__ctfe) is the first statement in
 the function body.
 
 Though I would check the linker map file to see if the linker didn't
 already omit functions that were never called.



Very nice work, Stefan!

This has multiple benefits which have not been mentioned yet:

Yes, the linker might eliminate such functions. In some cases it can't do 
that though, most importantly when building shared libraries. So you may 
end up with useless mixin helpers in shared libraries. Now when we ever 
get a proper export implementation, the linker might be able to remove 
these as well.


As mentioned earlier, no doing codegen might reduce compile times. I 
guess this can be more important for GDC and LDC than for DMD, as we 
spent quite some time in backend optimizer phases. I remember Mike doing 
template heavy embedded stuff which lead to compile times of minutes for 
a hello world with gdc. And IIRC the main issue was the optimizer working 
on hundreds of unused template instances.


And the most important thing: This will vastly improve betterC / nogc 
code:

--------------------
string generateMixin(T)(string b)
{
    assert(__ctfe);
    return T.stringof ~ " " ~ b ~ ";";
}

void main()  nogc
{
    mixin(generateMixin!string("b"));
}
--------------------


Here emitting generateMixin means that at least some of the array 
appending infrastructure needs to be there. If we don't emit the function 
at all, we can probably get rid of all related issues.


However, to also enable these benefits for betterC and not only for 
 nogc, we have to disable the betterC semantic check for such CTFE-only 
functions:

test.d(4): Error: array concatenation of expression "string " ~ b ~ ";" 
requires the GC which is not available with -betterC

 Stefan can you add that to your PR?




Also a comment regarding  ctfe UDA vs assert(__ctfe): Although  ctfe adds 
clutter to function signatures, that actually has it's benefits. With 
 ctfe it would be trivial to check that  ctfe functions are only called 
from other  ctfe functions or a CTFE context instead of letting users run 
into linker errors. Also the information would be available early, I'm 
not sure if the check in semantic3 might be too late for some things 
(e.g. to disable betterC checks)?

This is especially a problem if you link to external libraries where you 
don't have source code: The .di and docs could only include the signature 
and the compiler and user just can't know that this is a ctfe-only 
function. OTOH it doesn't make sense to have only the signature of such a 
function, so you'd probably include the function body for CTFE in the .di 
file anyway. But this would still require the compiler to do an complete 
semantic on all imported modules, only to warn the user if he tried to use 
a  ctfe function in non-ctfe context.


So I guess all things considered I'd prefer  ctfe. But as long as 
whatever syntax is chosen works with -betterC, I'll be very happy either 
way :-)

-- 
Johannes
Apr 07 2020
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 7 April 2020 at 11:32:27 UTC, Johannes Pfau wrote:

 And the most important thing: This will vastly improve betterC 
 / nogc code:

 --------------------
 string generateMixin(T)(string b)
 {
     assert(__ctfe);
     return T.stringof ~ " " ~ b ~ ";";
 }

 void main()  nogc
 {
     mixin(generateMixin!string("b"));
 }
 --------------------



I can confirm with the patch there are no template-instances in 
the object file

with patch:
0000000000000000 R _D6t_ctfe12__ModuleInfoZ
                  U _d_dso_registry
0000000000000000 W _Dmain
                  U _Dmain
                  U _d_run_main
                  U _GLOBAL_OFFSET_TABLE_
0000000000000000 W main
                  U _main
                  U __start_minfo
                  U __stop_minfo

without patch:
                  U _D12TypeInfo_Aya6__initZ
0000000000000000 R _D6t_ctfe12__ModuleInfoZ
0000000000000000 W _D6t_ctfe__T13generateMixinTAyaZQuFNaNbNfQnZQq
                  U _d_arraycatnTX
                  U _d_assertp
                  U _d_dso_registry
0000000000000000 W _Dmain
                  U _Dmain
                  U _d_run_main
                  U _GLOBAL_OFFSET_TABLE_
0000000000000000 W main
                  U _main
                  U __start_minfo
                  U __stop_minfo
0000000000000000 r _TMP0
0000000000000009 r _TMP1
000000000000000b r _TMP2
Apr 07 2020
next sibling parent Guillaume Piolat <firstname.lastname gmail.com> writes:
On Tuesday, 7 April 2020 at 12:09:48 UTC, Stefan Koch wrote:

 I can confirm with the patch there are no template-instances in 
 the object file


Very cool! Thanks.
Apr 07 2020
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 07 Apr 2020 12:09:48 +0000 schrieb Stefan Koch:


 On Tuesday, 7 April 2020 at 11:32:27 UTC, Johannes Pfau wrote:

 And the most important thing: This will vastly improve betterC / nogc
 code:

 --------------------
 string generateMixin(T)(string b)
 {
     assert(__ctfe);
     return T.stringof ~ " " ~ b ~ ";";
 }

 void main()  nogc {
     mixin(generateMixin!string("b"));
 }
 --------------------


 
 
 I can confirm with the patch there are no template-instances in the
 object file


The example above actually already compiles with -betterC, as the check 
is done in codegen phase! But some betterC checks seem to be done in 
semantic. This doesn't work yet:

------------------------
string generateMixin(T)(string b)
{
    assert(__ctfe);
    return typeid(T).stringof ~ " " ~ b ~ ";";
}

void main()  nogc
{
    mixin(generateMixin!string("b"));
}
------------------------
dmd -betterC test.d -c
test.d(4): Error: TypeInfo cannot be used with -betterC


-- 
Johannes
Apr 07 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/7/20 12:53 PM, Johannes Pfau wrote:

 The example above actually already compiles with -betterC, as the check
 is done in codegen phase! But some betterC checks seem to be done in
 semantic. This doesn't work yet:
 
 ------------------------
 string generateMixin(T)(string b)
 {
      assert(__ctfe);
      return typeid(T).stringof ~ " " ~ b ~ ";";
 }
 
 void main()  nogc
 {
      mixin(generateMixin!string("b"));
 }
 ------------------------
 dmd -betterC test.d -c
 test.d(4): Error: TypeInfo cannot be used with -betterC


Ironically, ctfe can't deal with TypeInfo today.

In the past, the issue was for functions that use TypeInfo is that the 
non-CTFE branch uses TypeInfo, and that can't be used in betterC, but 
the CTFE branch doesn't use TypeInfo, but the compiler wants to build 
the whole function.

So while this is still a problem, I don't think this fix will change that.

-Steve
Apr 07 2020
parent reply Johannes Pfau <nospam example.com> writes:
Am Tue, 07 Apr 2020 17:13:54 -0400 schrieb Steven Schveighoffer:


 On 4/7/20 12:53 PM, Johannes Pfau wrote:

 The example above actually already compiles with -betterC, as the check
 is done in codegen phase! But some betterC checks seem to be done in
 semantic. This doesn't work yet:
 
 ------------------------
 string generateMixin(T)(string b)
 {
      assert(__ctfe);
      return typeid(T).stringof ~ " " ~ b ~ ";";
 }
 
 void main()  nogc {
      mixin(generateMixin!string("b"));
 }
 ------------------------
 dmd -betterC test.d -c test.d(4): Error: TypeInfo cannot be used with
 -betterC


 
 Ironically, ctfe can't deal with TypeInfo today.
 
 In the past, the issue was for functions that use TypeInfo is that the
 non-CTFE branch uses TypeInfo, and that can't be used in betterC, but
 the CTFE branch doesn't use TypeInfo, but the compiler wants to build
 the whole function.
 
 So while this is still a problem, I don't think this fix will change
 that.
 
 -Steve


I'm not sure what exactly you're referring to, but this code compiles 
perfactly fine:
--------------------------
 string generateMixin(T)(string b)
 {
      assert(__ctfe);
      return "auto a = " ~typeid(T).stringof ~ ";";
 }
 
 void main()  nogc
 {
      mixin(generateMixin!string("b"));
      pragma(msg, generateMixin!string("b"));
 }
--------------------------


But it does not copile with -betterC. The point here is that the "Error: 
TypeInfo cannot be used with -betterC" message does not apply to CTFE-
only functions, so this PR should be able to disable the check.

My primary fear here is that the current implementation checks the 
function too late in semantic and therefore such checks (or the "ensure 
no non-ctfe function calls ctfe-only functions" check are much more 
difficult to implement than they should be. Using a pragma or attribute 
would easily solve this.

Maybe you'd have to introduce something analog to "safetyInProgress" 
which is "ctfeOnlyInProgress" to make this assert(__ctfe) detection fully 
usable? But I think this adds way to much complexity. Even right now, the 
code of the PR could probably be reduced by 1/2 when just using a pragma.

-- 
Johannes
Apr 08 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 8 April 2020 at 07:09:11 UTC, Johannes Pfau wrote:

 Am Tue, 07 Apr 2020 17:13:54 -0400 schrieb Steven Schveighoffer:


 On 4/7/20 12:53 PM, Johannes Pfau wrote:

 [...]


 
 Ironically, ctfe can't deal with TypeInfo today.
 
 In the past, the issue was for functions that use TypeInfo is 
 that the non-CTFE branch uses TypeInfo, and that can't be used 
 in betterC, but the CTFE branch doesn't use TypeInfo, but the 
 compiler wants to build the whole function.
 
 So while this is still a problem, I don't think this fix will 
 change that.
 
 -Steve


 I'm not sure what exactly you're referring to, but this code 
 compiles perfactly fine:
 --------------------------
  string generateMixin(T)(string b)
  {
       assert(__ctfe);
       return "auto a = " ~typeid(T).stringof ~ ";";
  }

  void main()  nogc
  {
       mixin(generateMixin!string("b"));
       pragma(msg, generateMixin!string("b"));
  }
 --------------------------


 But it does not copile with -betterC. The point here is that 
 the "Error: TypeInfo cannot be used with -betterC" message does 
 not apply to CTFE- only functions, so this PR should be able to 
 disable the check.

 My primary fear here is that the current implementation checks 
 the function too late in semantic and therefore such checks (or 
 the "ensure no non-ctfe function calls ctfe-only functions" 
 check are much more difficult to implement than they should be. 
 Using a pragma or attribute would easily solve this.

 Maybe you'd have to introduce something analog to 
 "safetyInProgress" which is "ctfeOnlyInProgress" to make this 
 assert(__ctfe) detection fully usable? But I think this adds 
 way to much complexity. Even right now, the code of the PR 
 could probably be reduced by 1/2 when just using a pragma.


The detection of assert __ctfe in the function body _can_ be done 
very early in the process
I do not want to introduce a pragma or magic annotatation.
When existing the language is perfectly able to express (only 
runs at ctfe).

Also the PR I have open is _only_ for that.
I or someone else, might write code which does the detection 
earlier and then disables betterC checks, but this is only about 
turning off codegen.
Apr 08 2020
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 8 April 2020 at 08:42:52 UTC, Stefan Koch wrote:

 When existing the language is perfectly able to express (only 
 runs at ctfe).


assert(__ctfe) does not express that you STATICALLY know it only 
runs at ctfe, it is a RUNTIME error and it is impossible to 
statically decide whether the function may actually get called 
outside ctfe context.
Apr 08 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 8 April 2020 at 09:11:45 UTC, Dennis wrote:

 On Wednesday, 8 April 2020 at 08:42:52 UTC, Stefan Koch wrote:

 When existing the language is perfectly able to express (only 
 runs at ctfe).


 assert(__ctfe) does not express that you STATICALLY know it 
 only runs at ctfe, it is a RUNTIME error and it is impossible 
 to statically decide whether the function may actually get 
 called outside ctfe context.


The only reason why I used assert (__ctfe) not static assert 
(__ctfe) is because that's impossible to express.
since __ctfe is a magic runtime variable that's hardwired to be 
zero in the non-ctfe case.
Apr 08 2020
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 8 April 2020 at 09:48:25 UTC, Stefan Koch wrote:

 The only reason why I used assert (__ctfe) not static assert 
 (__ctfe) is because that's impossible to express.
 since __ctfe is a magic runtime variable that's hardwired to be 
 zero in the non-ctfe case.


Indeed, and that should make you think how you can properly 
express that instead of adding special cases to established 
language concepts making them mean an entirely different thing.

Imagine if D could not express quick sort, so programmers just 
use bubble sort instead out of necessity. Then to fix that, you 
let the compiler detect a common bubble-sort implementation, and 
replace it with a quick-sort implementation.

It is backwards-compatible and automatically works with existing 
code. What a fix!

Except even the slightest deviation of that pattern will break 
the optimization.
Also the quick-sort implementation is not stable unlike bubble 
sort, so the behavior is actually different making this an 
illegal optimization. What a hack!

Problems like this should be attacked at the root. This 
assert(__ctfe) hack may fix one thing in the short-term, but you 
gain a lot of technical debt. Next thing you'll stumble upon this:

```
import core.stdc.stdlib;
int* newIntPtr()  nogc {
     if (__ctfe) {
         return new int(); // Error: cannot use new in  nogc 
function
     } else {
         return cast(int*) malloc(4);
     }
}
```

What now, another hack? Or should we finally think of something 
to make __ctfe work with `static if` / `static assert` proper?
Apr 08 2020
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 8 April 2020 at 10:27:02 UTC, Dennis wrote:

 What now, another hack? Or should we finally think of something 
 to make __ctfe work with `static if` / `static assert` proper?


I have tried doing that for a long time and gave up.
the problem is that static if and static assert get done before 
ctfe.
and therefore would always see the __ctfe being false.

A language which only allows you to express bubble-sort may have 
a good reason for that limitation.
Though I suspect it would come down to religious believes in 
mathematical concepts.
Apr 08 2020
parent Dennis <dkorpel gmail.com> writes:
On Wednesday, 8 April 2020 at 10:47:07 UTC, Stefan Koch wrote:

 A language which only allows you to express bubble-sort may 
 have a good reason for that limitation.
 Though I suspect it would come down to religious believes in 
 mathematical concepts.


Let me be clear in case you're implying my critique is for the 
sake of satisfying some mathematical principle.

I'm objecting this because *actual D users* constantly complain 
about D garnering more and more half-baked features that don't 
integrate well with the rest of the language, and this is like 
the epitome of that. It does not work at all with `version`, 
`mixin`, constant folding among others.
Apr 09 2020
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08.04.20 12:27, Dennis wrote:

 
 What now, another hack? Or should we finally think of something to make 
 __ctfe work with `static if` / `static assert` proper?


It's obvious how to make that work, but is it really worth the effort 
and compile time overhead? You'd have to perform semantic analysis twice 
for each function.
Apr 09 2020
parent Dennis <dkorpel gmail.com> writes:
On Thursday, 9 April 2020 at 09:30:14 UTC, Timon Gehr wrote:

 On 08.04.20 12:27, Dennis wrote:

 What now, another hack? Or should we finally think of 
 something to make __ctfe work with `static if` / `static 
 assert` proper?


 It's obvious how to make that work, but is it really worth the 
 effort and compile time overhead? You'd have to perform 
 semantic analysis twice for each function.


That could be a good POC implementation to start with, from there 
we can see how far it can be optimized.

However, what I meant with that statement was not that we 
literally should support `static if (__ctfe)`, just some way of 
actually statically recognizing ctfe-only code.

pragma(ctfe) looks like a promising candidate to me: 
https://github.com/dlang/dmd/pull/11014
Apr 09 2020
prev sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Wednesday, 8 April 2020 at 09:48:25 UTC, Stefan Koch wrote:

 On Wednesday, 8 April 2020 at 09:11:45 UTC, Dennis wrote:

 On Wednesday, 8 April 2020 at 08:42:52 UTC, Stefan Koch wrote:

 When existing the language is perfectly able to express (only 
 runs at ctfe).


 assert(__ctfe) does not express that you STATICALLY know it 
 only runs at ctfe, it is a RUNTIME error and it is impossible 
 to statically decide whether the function may actually get 
 called outside ctfe context.


 The only reason why I used assert (__ctfe) not static assert 
 (__ctfe) is because that's impossible to express.
 since __ctfe is a magic runtime variable that's hardwired to be 
 zero in the non-ctfe case.


This may be a naive suggestion, but how hard would it be to make 
version(ctfe) work?

version (ctfe) string generateMixin(...) { ... }

Using version means you would be able to know earlier whether a 
function is ctfe-only, but I'm not sure how you could propogate 
that information through the rest of semantic-analysis and what 
effects that could have.
Apr 08 2020
parent Johannes Pfau <nospam example.com> writes:
Am Wed, 08 Apr 2020 19:16:15 +0000 schrieb Jonathan Marler:


 On Wednesday, 8 April 2020 at 09:48:25 UTC, Stefan Koch wrote:

 On Wednesday, 8 April 2020 at 09:11:45 UTC, Dennis wrote:

 On Wednesday, 8 April 2020 at 08:42:52 UTC, Stefan Koch wrote:

 When existing the language is perfectly able to express (only runs at
 ctfe).


 assert(__ctfe) does not express that you STATICALLY know it only runs
 at ctfe, it is a RUNTIME error and it is impossible to statically
 decide whether the function may actually get called outside ctfe
 context.


 The only reason why I used assert (__ctfe) not static assert (__ctfe)
 is because that's impossible to express.
 since __ctfe is a magic runtime variable that's hardwired to be zero in
 the non-ctfe case.


 
 This may be a naive suggestion, but how hard would it be to make
 version(ctfe) work?
 
 version (ctfe) string generateMixin(...) { ... }
 
 Using version means you would be able to know earlier whether a function
 is ctfe-only, but I'm not sure how you could propogate that information
 through the rest of semantic-analysis and what effects that could have.


I'd guess very hard. To be consistent with the rest of the language, you 
then have to support this:

void foo()
{
    version (ctfe)
    {
    }
    else
    {
    }
}

You can end up with two completely different function implementations. 
And you can do this at top-level as well, where you have the same function 
in version(ctfe) and in the else block.


Maybe you can somehow cheat and simplify this. But in the most general 
case, you'd have to keep two versions of the AST of each function.

-- 
Johannes
Apr 08 2020
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, April 8, 2020 3:11:45 AM MDT Dennis via Digitalmars-d wrote:

 On Wednesday, 8 April 2020 at 08:42:52 UTC, Stefan Koch wrote:

 When existing the language is perfectly able to express (only
 runs at ctfe).


 assert(__ctfe) does not express that you STATICALLY know it only
 runs at ctfe, it is a RUNTIME error and it is impossible to
 statically decide whether the function may actually get called
 outside ctfe context.


Sure, but since you're asserting that it should never run except during
CTFE, turning that into a compiler or linker error shouldn't be a problem.
It's about as clear a case as you can get where the compiler can effectively
turn a runtime condition into a compile-time one, because it has enough
information to do that optimization, and if assert(__ctfe) is used where it
shouldn't be, then it will just result in an error when compiling/linking
your program instead of at runtime. Insisting that the compiler not do that
optimization doesn't really buy us anything, and having it do that
optimization could allow us to reduce compilation times in code with a lot
of CTFE-only functions.

It would arguably be cleaner if __ctfe were done in a way that it were a
static condition rather than a runtime one (and thus static assert could be
used instead), but that's not how it works, and from what I understand of
how CTFE in general works, it's highly unlikely that that will ever change.

- Jonathan M Davis
Apr 08 2020
parent Dennis <dkorpel gmail.com> writes:
On Wednesday, 8 April 2020 at 23:45:43 UTC, Jonathan M Davis 
wrote:

 because it has enough information to do that optimization, and 
 if assert(__ctfe) is used where it shouldn't be, then it will 
 just result in an error when compiling/linking your program 
 instead of at runtime.


It's not an optimization, it is a breaking change that can turn 
valid code into compile errors or linker errors. Minimal examples 
are:

```
int ctfeOnly() {
     assert(__ctfe);
     return 3;
}

int alsoCtfeOnly() {
     version(all) {
         assert(__ctfe); // ctfe detection is defeated by the 
slightest deviation
         return ctfeOnly();
     }
}
```
This will become a compile error.

```
immutable ctfeOnly = function int() {
     assert(__ctfe);
     return 3;
};
```
This will become a linker error.


 It would arguably be cleaner if __ctfe were done in a way that 
 it were a static condition rather than a runtime one (and thus 
 static assert could be used instead), but that's not how it 
 works, and from what I understand of how CTFE in general works, 
 it's highly unlikely that that will ever change.


I don't agree there. First of all, this seems like an 
implementation difficulty, not a fundamental limitation. Second 
of all, there are plenty of alternatives to consider, such as 
version(__ctfe) or pragma(ctfe). Johannes Pfau is working on the 
latter right now: https://github.com/dlang/dmd/pull/11014
Apr 09 2020
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 08, 2020 at 05:45:43PM -0600, Jonathan M Davis via Digitalmars-d
wrote:
[...]

 It would arguably be cleaner if __ctfe were done in a way that it were
 a static condition rather than a runtime one (and thus static assert
 could be used instead), but that's not how it works, and from what I
 understand of how CTFE in general works, it's highly unlikely that
 that will ever change.


[...]

Yeah, static assert is run at template expansion time, which comes
before CTFE, so __ctfe cannot be evaluated by static assert:

	https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


T

-- 
WINDOWS = Will Install Needless Data On Whole System -- CompuMan
Apr 08 2020
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 08 Apr 2020 08:42:52 +0000 schrieb Stefan Koch:



 My primary fear here is that the current implementation checks the
 function too late in semantic and therefore such checks (or the "ensure
 no non-ctfe function calls ctfe-only functions" check are much more
 difficult to implement than they should be. Using a pragma or attribute
 would easily solve this.

 Maybe you'd have to introduce something analog to "safetyInProgress"
 which is "ctfeOnlyInProgress" to make this assert(__ctfe) detection
 fully usable? But I think this adds way to much complexity. Even right
 now, the code of the PR could probably be reduced by 1/2 when just
 using a pragma.


 
 The detection of assert __ctfe in the function body _can_ be done very
 early in the process I do not want to introduce a pragma or magic
 annotatation.
 When existing the language is perfectly able to express (only runs at
 ctfe).
 
 Also the PR I have open is _only_ for that.
 I or someone else, might write code which does the detection earlier and
 then disables betterC checks, but this is only about turning off
 codegen.


The important point is that we do not bake in semantics now which might 
turn out to cause problems later. If we can do the assert detection before 
all betterC checking, nogc checking, and related checks, this is fine. 
But it should not be an afterthought, we should really be pretty sure 
that this will not cause issues later on and that this integrates well 
with all other language features. Especially betterC, nogc (if you have a 
 nogc: at module top, do you want to still enforce GC checks in CTFE only 
code, ...)

-- 
Johannes
Apr 08 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/8/20 3:09 AM, Johannes Pfau wrote:

 Am Tue, 07 Apr 2020 17:13:54 -0400 schrieb Steven Schveighoffer:
 

 On 4/7/20 12:53 PM, Johannes Pfau wrote:

 The example above actually already compiles with -betterC, as the check
 is done in codegen phase! But some betterC checks seem to be done in
 semantic. This doesn't work yet:

 ------------------------
 string generateMixin(T)(string b)
 {
       assert(__ctfe);
       return typeid(T).stringof ~ " " ~ b ~ ";";
 }

 void main()  nogc {
       mixin(generateMixin!string("b"));
 }
 ------------------------
 dmd -betterC test.d -c test.d(4): Error: TypeInfo cannot be used with
 -betterC


 Ironically, ctfe can't deal with TypeInfo today.

 In the past, the issue was for functions that use TypeInfo is that the
 non-CTFE branch uses TypeInfo, and that can't be used in betterC, but
 the CTFE branch doesn't use TypeInfo, but the compiler wants to build
 the whole function.

 So while this is still a problem, I don't think this fix will change
 that.


 
 I'm not sure what exactly you're referring to, but this code compiles
 perfactly fine:
 --------------------------
   string generateMixin(T)(string b)
   {
        assert(__ctfe);
        return "auto a = " ~typeid(T).stringof ~ ";";
   }
   
   void main()  nogc
   {
        mixin(generateMixin!string("b"));
        pragma(msg, generateMixin!string("b"));
   }
 --------------------------
 


I misread your example! I thought you were doing typeid(T).toString. I 
had to fix an issue very recently that was similar to this.

TypeInfo does not work inside ctfe, but here you aren't actually using 
the TypeInfo object. If you switch to typeid(T).toString, even without 
-betterC you get:

Error: static variable typeid(string) cannot be read at compile time

But your example is indeed a bug with betterC. But even if that was 
fixed, the code would still not work. It seems more like a diagnostic error:

return "auto a = typeid(" ~ T.stringof ~ ");";

This is equivalent to your code and will work in betterC, but the mixin 
of it won't.


 But it does not copile with -betterC. The point here is that the "Error:
 TypeInfo cannot be used with -betterC" message does not apply to CTFE-
 only functions, so this PR should be able to disable the check.


In fact, there is a further error that TypeInfo isn't being used AT ALL, 
and it's still complaining.


 
 My primary fear here is that the current implementation checks the
 function too late in semantic and therefore such checks (or the "ensure
 no non-ctfe function calls ctfe-only functions" check are much more
 difficult to implement than they should be. Using a pragma or attribute
 would easily solve this.


This is a legitimate concern. Indeed, it might not fix this issue just 
to not emit the code to the binary.

-Steve
Apr 08 2020
parent Johannes Pfau <nospam example.com> writes:
Am Wed, 08 Apr 2020 09:26:36 -0400 schrieb Steven Schveighoffer:


 

 My primary fear here is that the current implementation checks the
 function too late in semantic and therefore such checks (or the "ensure
 no non-ctfe function calls ctfe-only functions" check are much more
 difficult to implement than they should be. Using a pragma or attribute
 would easily solve this.


 
 This is a legitimate concern. Indeed, it might not fix this issue just
 to not emit the code to the binary.
 
 -Steve



I've posted a proof-of-concept PR using pragma(ctfe) here:
https://github.com/dlang/dmd/pull/11014

Unlike the `assert(__ctfe)` check, this is done very early in the 
semantic analysis. I added support to bypass all -betterC checks I could 
find in the DMD source code. If one is missing, it should be trivial to 
add. I've also added examples to the tests in the PR.

I also tried to backport the commit which introduces the betterC bypass 
to Stefan's PR. But as expected it does not work, as the assert detection 
is done too late. I'm not saying that we must go for pragma(ctfe), but 
whatever implementation we merge, it should at least support those tests 
as well. Especially considering the pragma implementation is also way 
less complex.


Another thing we could think about is how to handle this code:

 nogc:

pragma(ctfe)
string mixinHelper2(string a)
{
     return "int " ~ a ~ ";";
}

void main()  nogc
{
     mixin(mixinHelper("a"));
}


I'd probably disable nogc checks in ctfe-only functions for convenience. 
But I guess that is debatable.

-- 
Johannes
Apr 08 2020
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-04-05 14:11, Stefan Koch wrote:

 Hi Guys,
 
 I am currently working on speeding up dmd. In the presence of templates 
 and ctfe.
 and one thing that stood out is that we do codegen for symbols which are 
 never used.
 (and hopefully eliminated by a smart linker but eh ...)
 
 I've seen the following pattern in the wild.
 
 
 string GenerateMixin(string[] params)
 {
      assert(__ctfe);
      .... more code ....
 }
 
 that means if we see assert(__ctfe) inside the outermost scope of a 
 function body ( not in a branch) then we know that this function would 
 assert if called at runtime.
 And therefore no correct code can be calling it.
 which also means we can forgo having code-gen or optimization for the 
 function body.
 and reduce the count of names which has to get mangled.
 
 They only downside to this, is that giving assert(__ctfe) the special 
 meaning to skip codegen might be confusing to some people .... then 
 again you wouldn't use assert(__ctfe) unless you expect that function to 
 not be available at run-time
 
 What do you guys think?


For many years I wanted a feature that could indicate a function is only 
used at compile time.

BTW, this is a breaking change. Consider this code:

import core.exception;

void foo()
{
     assert(__ctfe);
}

void main()  system
{
     try
         foo();
     catch (AssertError)
     {}
}

-- 
/Jacob Carlborg
Apr 07 2020
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Tuesday, 7 April 2020 at 17:52:44 UTC, Jacob Carlborg wrote:

 BTW, this is a breaking change. Consider this code:

 import core.exception;

 void foo()
 {
     assert(__ctfe);
 }

 void main()  system
 {
     try
         foo();
     catch (AssertError)
     {}
 }


Catching an AssertError thrown by an assert statement is 
officially undefined behavior:
https://dlang.org/spec/expression.html#assert_expressions
Apr 07 2020
parent Jacob Carlborg <doob me.com> writes:
On 2020-04-07 21:33, tsbockman wrote:


 Catching an AssertError thrown by an assert statement is officially 
 undefined behavior:
 https://dlang.org/spec/expression.html#assert_expressions


Ok, let me tweak the example a bit:

void foo()
{
     assert(__ctfe);
}

unittest  system
{
     try
         foo();
     catch (AssertError)
     {}
}

Then if you read the third item in the list:

"AssertExpression has different semantics if it is in a unittest or in 
contract."

Then you read the unittest spec [1], the third item, the second sentence:

"Unlike AssertExpressions used elsewhere, the assert is not assumed to 
hold, and upon assert failure the program is still in a defined state."

Now it's not undefined anymore.

[1] https://dlang.org/spec/unittest.html

-- 
/Jacob Carlborg
Apr 08 2020
prev sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 7 April 2020 at 17:52:44 UTC, Jacob Carlborg wrote:

 void foo()
 {
     assert(__ctfe);
 }

 void main()  system
 {
     try
         foo();
     catch (AssertError)
     {}
 }


firstly Does anybody actually do this?
secondly your program either complies with no change (i.e. the 
assert never fired), or you get a message telling you that 
ctfe-only functions can't be called at runtime.
Apr 07 2020
parent reply Jacob Carlborg <doob me.com> writes:
On 2020-04-07 22:43, Stefan Koch wrote:

 On Tuesday, 7 April 2020 at 17:52:44 UTC, Jacob Carlborg wrote:

 void foo()
 {
     assert(__ctfe);
 }

 void main()  system
 {
     try
         foo();
     catch (AssertError)
     {}
 }


 
 firstly Does anybody actually do this?


Ok, let me tweak the example a bit:

void foo()
{
     assert(__ctfe);
}

unittest
{
     foo();
}

druntime catches _all_ exceptions:

https://github.com/dlang/druntime/blob/46867186035fdf5ec2596efbb770627518ff0919/src/core/runtime.d#L617

Regardless if anyone does it or not, it's still a breaking change. Then 
we can of course argue the impact of the breaking change, how much code 
will actually break.

-- 
/Jacob Carlborg
Apr 08 2020
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/8/20 2:53 PM, Jacob Carlborg wrote:

 On 2020-04-07 22:43, Stefan Koch wrote:

 On Tuesday, 7 April 2020 at 17:52:44 UTC, Jacob Carlborg wrote:

 void foo()
 {
     assert(__ctfe);
 }

 void main()  system
 {
     try
         foo();
     catch (AssertError)
     {}
 }


 firstly Does anybody actually do this?


 
 Ok, let me tweak the example a bit:
 
 void foo()
 {
      assert(__ctfe);
 }
 
 unittest
 {
      foo();
 }
 
 druntime catches _all_ exceptions:
 
 https://github.com/dlang/druntime/blob/46867186035fdf5ec2596efbb770627518ff0919/src
core/runtime.d#L617 
 
 
 Regardless if anyone does it or not, it's still a breaking change. Then 
 we can of course argue the impact of the breaking change, how much code 
 will actually break.
 


I think it's a breaking change, just like flagging any obvious error 
case as a non-compilable error is a breaking change for code that wasn't 
aware of the problem.

I'm sure there was lots of code that did something equivalent to this 
before it was disallowed:

ref int foo()
{
   int x;
   return x;
}

That doesn't mean we shouldn't do it.

-Steve
Apr 08 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/8/20 2:53 PM, Jacob Carlborg wrote:

 Regardless if anyone does it or not, it's still a breaking change. Then 
 we can of course argue the impact of the breaking change, how much code 
 will actually break.


another interesting case:

void foo()
{
    assert(__ctfe);
}

void main(string[] args)
{
    if(args.length == 0)
        foo(0);
}

Arguably, the call will never happen. But it won't compile now.

As I said a minute ago, this is a breaking change, and it's one we 
should make.

-Steve
Apr 08 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/8/20 3:04 PM, Steven Schveighoffer wrote:

         foo(0);


Ugh, that should have been foo();, forgot to edit that.

-Steve
Apr 08 2020
prev sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 [...]

 What do you guys think?

 Cheers,

 Stefan


Other and new thought.

This will have a positive impact on coverage maybe ?

I opened this a while ago: 
https://issues.dlang.org/show_bug.cgi?id=15590

(0 coverage should be ignored in __ctfe branches)

The problem would be partially solved when splitting the function 
in two versions.
Apr 09 2020
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 9 April 2020 at 10:14:21 UTC, Basile B. wrote:

 On Sunday, 5 April 2020 at 12:11:23 UTC, Stefan Koch wrote:

 Hi Guys,

 [...]

 What do you guys think?

 Cheers,

 Stefan


 Other and new thought.

 This will have a positive impact on coverage maybe ?

 I opened this a while ago: 
 https://issues.dlang.org/show_bug.cgi?id=15590

 (0 coverage should be ignored in __ctfe branches)

 The problem would be partially solved when splitting the 
 function in two versions.


well ... splitting the function in two versions comes with the 
oblivious overhead of having to run semantic two times.
As well as having to maintain that in a multi-pass compilation 
model which we have to avoid having headers and that gives use 
the power to declare anywhere.
Apr 09 2020