www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - CTFE casts of delegates

reply Eyal Lotem <eyal.lotem gmail.com> writes:
Hi,

According to [1], we can use an escape hatch in D to assume 
purity or assume  nogc.

We use this escape hatch for 2 purposes:

1) Wrapping standard library functions that forgot the correct 
annotations

2) Aborting assertion mechanisms - where we don't care about 
 nogc or "pure" in the case that we abort the program.

-----

However, a big drawback of this escape hatch is that if it is 
ever used (even very indirectly) in the context of CTFE, it 
yields a DMD error:

Error: pointer cast from void function()  safe to void function() 
 nogc  safe is not supported at compile time

-----

Losing the ability to call this assertion mechanism from any 
function that is used in CTFE context is very problematic.

How can I work around this issue?

Should it be considered a DMD bug? Is there any reason for 
attribute-only delegate/function casts in CTFEe to be disallowed?

[1] http://klickverbot.at/blog/2012/05/purity-in-d/
Jan 02
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 3 January 2017 at 07:58:05 UTC, Eyal Lotem wrote:
 Hi,

 we can use an escape hatch in D to assume purity or assume 
  nogc.


 Losing the ability to call this assertion mechanism from any 
 function that is used in CTFE context is very problematic.

 How can I work around this issue?

 Should it be considered a DMD bug? Is there any reason for 
 attribute-only delegate/function casts in CTFEe to be 
 disallowed?
The reason for them being is disallowed is, that casts are quite tricky at CTFE in general. Therefore everything that is not explicitly allowed is forbidden, also CTFE is supposed to exhibit no undefined behavior whatsoever. Which is a very nice property of CTFE. I think that I can provide a dmd patch that would allow the casts you want at ctfe. However I would not propose it for inclusion myself.
Jan 03
parent reply Eyal Lotem <eyal weka.io> writes:
On Tuesday, 3 January 2017 at 09:44:38 UTC, Stefan Koch wrote:

 I think that I can provide a dmd patch that would allow the 
 casts you want at ctfe.
 However I would not propose it for inclusion myself.
A cast that only fiddles with function attributes like nogc should be perfectly safe, shouldn't it?
Jan 04
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 4 January 2017 at 08:45:59 UTC, Eyal Lotem wrote:
 On Tuesday, 3 January 2017 at 09:44:38 UTC, Stefan Koch wrote:

 I think that I can provide a dmd patch that would allow the 
 casts you want at ctfe.
 However I would not propose it for inclusion myself.
A cast that only fiddles with function attributes like nogc should be perfectly safe, shouldn't it?
Honestly I am not sure about it. I guess I could enable this back-door for you and others. Patch will follow in the next days, I am not sure if it will be merged though.
Jan 04
parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 4 January 2017 at 16:06:11 UTC, Stefan Koch wrote:
 On Wednesday, 4 January 2017 at 08:45:59 UTC, Eyal Lotem wrote:
 On Tuesday, 3 January 2017 at 09:44:38 UTC, Stefan Koch wrote:

 I think that I can provide a dmd patch that would allow the 
 casts you want at ctfe.
 However I would not propose it for inclusion myself.
A cast that only fiddles with function attributes like nogc should be perfectly safe, shouldn't it?
Honestly I am not sure about it. I guess I could enable this back-door for you and others.
This back-door already exists, see e.g.: https://github.com/nordlow/phobos-next/blob/master/src/dbgio.d#L13 and http://forum.dlang.org/post/nq4eol$2h34$1 digitalmars.com
 Patch will follow in the next days, I am not sure if it will be 
 merged though.
There is assumeWontThrow (https://dlang.org/phobos/std_exception.html#.assumeWontThrow) in std.exception, so I think the chances for assumeNogc or assumePure aren't that bad ;-)
Jan 04
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Wednesday, 4 January 2017 at 16:19:52 UTC, Seb wrote:

 There is assumeWontThrow 
 (https://dlang.org/phobos/std_exception.html#.assumeWontThrow) 
 in std.exception, so I think the chances for assumeNogc or 
 assumePure aren't that bad ;-)
I understand that it did not work at ctfe.
Jan 04
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wednesday, January 04, 2017 16:19:52 Seb via Digitalmars-d wrote:
 On Wednesday, 4 January 2017 at 16:06:11 UTC, Stefan Koch wrote:
 On Wednesday, 4 January 2017 at 08:45:59 UTC, Eyal Lotem wrote:
 On Tuesday, 3 January 2017 at 09:44:38 UTC, Stefan Koch wrote:
 I think that I can provide a dmd patch that would allow the
 casts you want at ctfe.
 However I would not propose it for inclusion myself.
A cast that only fiddles with function attributes like nogc should be perfectly safe, shouldn't it?
Honestly I am not sure about it. I guess I could enable this back-door for you and others.
This back-door already exists, see e.g.: https://github.com/nordlow/phobos-next/blob/master/src/dbgio.d#L13 and http://forum.dlang.org/post/nq4eol$2h34$1 digitalmars.com
 Patch will follow in the next days, I am not sure if it will be
 merged though.
There is assumeWontThrow (https://dlang.org/phobos/std_exception.html#.assumeWontThrow) in std.exception, so I think the chances for assumeNogc or assumePure aren't that bad ;-)
Well, assumeWontThrow is a completely different animal. There is a perfectly legitimate way to make a function be nothrow while still call functions that can throw exceptions - you have a catch(Exception) block in it and don't throw any exceptons or call any non-nothrow functions outside of it. No such thing exists for nogc or pure. Those are type system issues, and in order to make something nogc that's calling functions that aren't nogc or to make something pure that's calling functions that aren't pure, you have to subvert the type system. If the programmer does it, they have to make sure that they do so in a way that doesn't violate the guarantees that the compiler makes about nogc or pure - which isn't necessarily easy. And in general, it's a big red flag when code uses casts to make anything nogc or pure. So, putting something in the standard library specifically to support such an idiom really doesn't seem like a good idea. Only experts should be doing it, and even then, they should reconsider. Now, as to having the ability to have casting muck with pure or nogc with CTFE, I don't know. It would be nice to have the same capabilties at compile time that you have at runtime, and if a function is legitimately and correctly using casts to deal with nogc or pure, then it sucks that it can't be done with CTFE. However, since code that legitimately casts a function to alter its attributes should be _very_ rare, it should also be quite rare that the problem even comes up.
From an  safety perspective though, I don't see why it would be an
implementation issue to support such casts durintg CTFE. IIRC, you can't mess with global or static variables in CTFE if they're mutable, and you can't call C functions. So, in essence, _everything_ is pure during CTFE. If I understand correctly, any attempt to call anything that was not legitimately pure would fail for not being CTFE-able. And since AFAIK, you pretty much _have_ to use the GC for everything during CTFE (you certainly can't call malloc, and most pointer operations are forbidden, making stack allocators and whatnot infeasible), I don't think that having a function be nogc when it really isn't nogc would actually matter during CTFE. So, I would think that making casts for attributes legal during CTFE wouldn't be a problem (though I could be missing something), but at the same time, if someone is actally asking for such a capability, that's very worrisome. That implies that pure and nogc are not being used correctly and that casts are being used to try and force the compiler into submission, which risks nasty bugs when the compiler is then making assumptions about the code that are wrong thanks to the fact that it was lied to with casts. - Jonathan M Davis
Jan 04
parent reply Eyal Lotem <eyal.lotem gmail.com> writes:
On Thursday, 5 January 2017 at 03:05:21 UTC, Jonathan M Davis 
wrote:
 However, since code that legitimately casts a function to alter 
 its attributes should be _very_ rare, it should also be quite 
 rare that the problem even comes up.
It is very rare to have it in code, but the low-level function that does this rare thing (custom assertion function) may be very frequently called, indirectly.
 So, I would think that making casts for attributes legal during 
 CTFE wouldn't be a problem (though I could be missing 
 something), but at the same time, if someone is actally asking 
 for such a capability, that's very worrisome. That implies that 
 pure and  nogc are not being used correctly and that casts are 
 being used to try and force the compiler into submission, which 
 risks nasty bugs when the compiler is then making assumptions 
 about the code that are wrong thanks to the fact that it was 
 lied to with casts.
Well, I did explain the reason for this cast. An aborting-assertion that discontinues execution (hard kill the process). We expect these assertions to never happen. If they do, we don't really care about any GC or impure things that they do just before terminating the process. We don't really want *all* code to be marked gc/impure because it directly or indirectly uses such assertions.
Jan 04
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, January 05, 2017 07:35:52 Eyal Lotem via Digitalmars-d wrote:
 On Thursday, 5 January 2017 at 03:05:21 UTC, Jonathan M Davis

 wrote:
 However, since code that legitimately casts a function to alter
 its attributes should be _very_ rare, it should also be quite
 rare that the problem even comes up.
It is very rare to have it in code, but the low-level function that does this rare thing (custom assertion function) may be very frequently called, indirectly.
 So, I would think that making casts for attributes legal during
 CTFE wouldn't be a problem (though I could be missing
 something), but at the same time, if someone is actally asking
 for such a capability, that's very worrisome. That implies that
 pure and  nogc are not being used correctly and that casts are
 being used to try and force the compiler into submission, which
 risks nasty bugs when the compiler is then making assumptions
 about the code that are wrong thanks to the fact that it was
 lied to with casts.
Well, I did explain the reason for this cast. An aborting-assertion that discontinues execution (hard kill the process). We expect these assertions to never happen. If they do, we don't really care about any GC or impure things that they do just before terminating the process. We don't really want *all* code to be marked gc/impure because it directly or indirectly uses such assertions.
Well, you appear to have a use case where such a cast makes sense, but that's rare. So, while it sucks for you that it doesn't work, it wouldn't affect most folks, and in most cases, they probably shouldn't be doing it. Regardless, I have no problem with making such casts legal in CTFE if it can be done safely (and I think that it can be, since I don't see how pure can actually be violated during CTFE, and if an nogc ends up allocating during CTFE, I don't think that that's actually a problem), since then you're just in essentially the same boat with CTFE that you are with runtime in that if you're doing it wrong, you're shooting yourself in the foot, but you're free to do it if you really want to. I don't know exactly what you're doing with your custom assertion function, but my suggestion would be to just use the normal assertion mechanism during CTFE. e.g. void customAssert(bool result, lazy msg) { if(__ctfe) assert(result, msg); else (cast(...)&myCustomAssert)(result, msg); } The bigger problem is if you're trying to use functions with CTFE which aren't marked properly as pure or nogc, and I don't think that there's really a solution for that right now other than providing an alternate implementation which doesn't have the problem. In general, the real solution is to fix it so that the functions are appropriately pure or nogc rather than doing any casting (the casting is either a hack/workaround or a misunderstanding of how the function works), but obviously, that's not always in your control, and you're either forced to give up on pure or nogc for that piece of code or be very sure of what the function actually does and do the cast as a workaround. So, if you can't fix the function in question, and you're forced to cast it to use it in the way that you're trying to use it, then I think that the only solution for CTFE right now would be to replace it with different functionality during CTFE, which is far from ideal. Personally, when something doesn't work with nogc or pure, and I want to make my function nogc or pure, I just comment out the nogc or pure on the function and live with it not being nogc or pure for the moment rather than casting, since casting seems like a serious risk, especially if the functionality changes later, and it really can't be nogc or pure. Then I try and fix it so that whatever isn't properly nogc or pure is properly nogc or pure, and if I can't, I just put up with the problem until it gets sorted out. I can understand wanting to go with the casting solution. I just don't think that it's worth the risk. In any case, assuming that there's no technical barrier to making the attribute casts work in CTFE (and it doesn't sound like there is), I don't see any reason why such casts shouldn't be legal in CTFE (at least with pure and nogc). If anything, from what I can tell they're _more_ safe with CTFE than runtime. - Jonathan M Davis
Jan 05