www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - safe D requires which DIP switches?

reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
As far as I know ` safe` is only really safe with both DIP25 and 
DIP1000 in effect, and I thought that DIP25 has been integrated 
for a while now. Nonetheless, DIP25 still appears in the list of 
previews (`dmd -preview=help`) *as well as* being revertible 
(`dmd -revert=help`) which adds to the confusion.

What is in the way of integrating DIP1000 by default?

-- Bastiaan.
Jan 03 2022
next sibling parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Monday, 3 January 2022 at 14:27:55 UTC, Bastiaan Veelo wrote:

 As far as I know ` safe` is only really safe with both DIP25 
 and DIP1000 in effect, and I thought that DIP25 has been 
 integrated for a while now. Nonetheless, DIP25 still appears in 
 the list of previews (`dmd -preview=help`) *as well as* being 
 revertible (`dmd -revert=help`) which adds to the confusion.

 What is in the way of integrating DIP1000 by default?

 -- Bastiaan.


Bugs, and lots of them.

- Alex
Jan 03 2022
parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Monday, 3 January 2022 at 15:19:22 UTC, 12345swordy wrote:

 On Monday, 3 January 2022 at 14:27:55 UTC, Bastiaan Veelo wrote:

 As far as I know ` safe` is only really safe with both DIP25 
 and DIP1000 in effect, and I thought that DIP25 has been 
 integrated for a while now. Nonetheless, DIP25 still appears 
 in the list of previews (`dmd -preview=help`) *as well as* 
 being revertible (`dmd -revert=help`) which adds to the 
 confusion.

 What is in the way of integrating DIP1000 by default?

 -- Bastiaan.


 Bugs, and lots of them.

 - Alex


But are these bugs in the rigidity of the checks, or do other 
problems arise when DIP1000 is enabled? If it is just the former, 
there is little reason to not switch it on by default? Does it 
produce false negatives (requiring ` trusted` where unnecessary)?

And does `-preview=dip25` make a difference or not? Still 
confused.

--Bastiaan.
Jan 03 2022
next sibling parent Paul Backus <snarwin gmail.com> writes:
On Monday, 3 January 2022 at 16:06:38 UTC, Bastiaan Veelo wrote:

 On Monday, 3 January 2022 at 15:19:22 UTC, 12345swordy wrote:

 On Monday, 3 January 2022 at 14:27:55 UTC, Bastiaan Veelo 
 wrote:

 What is in the way of integrating DIP1000 by default?

 -- Bastiaan.


 Bugs, and lots of them.

 - Alex


 But are these bugs in the rigidity of the checks, or do other 
 problems arise when DIP1000 is enabled? If it is just the 
 former, there is little reason to not switch it on by default? 
 Does it produce false negatives (requiring ` trusted` where 
 unnecessary)?


The problem is usually that the checks are too loose--they allow 
undefined behavior in ` safe` code when `-preview=dip1000` is 
enabled.

You can browse the list yourself here: 
https://issues.dlang.org/buglist.cgi?quicksearch=dip1000
Jan 03 2022
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Monday, 3 January 2022 at 16:06:38 UTC, Bastiaan Veelo wrote:

 But are these bugs in the rigidity of the checks, or do other 
 problems arise when DIP1000 is enabled?


This is the situation on dmd's master branch:
✅ = correct, considering the current language design
🆗 = correct, but is more strict than necessary
💀 = incorrect, allows memory corruption

| Action                         | no dip1000 | dip1000           
                                  |
|--------------------------------|------------|----------------------------------------------------|
| Take address of local variable | 🆗 Error   | ✅ Allowed using 
`scope` when type has no pointers  |
| Make slice of local variable   | 💀 Allowed | ✅ Allowed using 
`scope` when type has no pointers  |
| Take address of `ref` return   | 💀 Allowed | 🆗 Allowed using 
`scope` when type has no pointers |
| Make slice of `ref` return     | 💀 Allowed | 💀 Allowed even 
when type has pointers             |

I'm trying to turn dip1000's 💀 into an 🆗, but Walter wants to go 
straight to a ✅:
https://github.com/dlang/dmd/pull/13362#issuecomment-981181277

On top of this, dip1000 currently has a few more 💀s because:

- parameters of `pure` functions can be incorrectly inferred 
`scope`, even when you can escape them with a thrown Exception. 
The fix is easy, but it requires updating Phobos and excel-d to 
pass the test suite, which is cumbersome.
https://issues.dlang.org/show_bug.cgi?id=22221

- `inout` on struct member functions currently gives the 
permissions of `return` without the associated lifetime 
restrictions.
https://issues.dlang.org/show_bug.cgi?id=20149

- Some checks are missing when using closures:
https://issues.dlang.org/show_bug.cgi?id=22298


 And does `-preview=dip25` make a difference or not? Still 
 confused.


dip25 prevents returning a `ref` parameter by `ref`, unless you 
annotate it `return ref`. It's enabled by default, but only as a 
deprecation. `-preview=dip25` turns the deprecation into an 
error. `-preview=dip1000` implies `-preview=dip25`, so there's no 
need to use both switches.
Jan 04 2022
next sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Tuesday, 4 January 2022 at 11:57:41 UTC, Dennis wrote:

 This is the situation on dmd's master branch:
 ✅ = correct, considering the current language design
 🆗 = correct, but is more strict than necessary
 💀 = incorrect, allows memory corruption

 | Action                         | no dip1000 | dip1000         
                                    |
 |--------------------------------|------------|----------------------------------------------------|
 | Take address of local variable | 🆗 Error   | ✅ Allowed using 
 `scope` when type has no pointers  |
 | Make slice of local variable   | 💀 Allowed | ✅ Allowed using 
 `scope` when type has no pointers  |
 | Take address of `ref` return   | 💀 Allowed | 🆗 Allowed using 
 `scope` when type has no pointers |
 | Make slice of `ref` return     | 💀 Allowed | 💀 Allowed even 
 when type has pointers             |

 I'm trying to turn dip1000's 💀 into an 🆗, but Walter wants to 
 go straight to a ✅:
 https://github.com/dlang/dmd/pull/13362#issuecomment-981181277

 On top of this, dip1000 currently has a few more 💀s because:

 - parameters of `pure` functions can be incorrectly inferred 
 `scope`, even when you can escape them with a thrown Exception. 
 The fix is easy, but it requires updating Phobos and excel-d to 
 pass the test suite, which is cumbersome.
 https://issues.dlang.org/show_bug.cgi?id=22221

 - `inout` on struct member functions currently gives the 
 permissions of `return` without the associated lifetime 
 restrictions.
 https://issues.dlang.org/show_bug.cgi?id=20149

 - Some checks are missing when using closures:
 https://issues.dlang.org/show_bug.cgi?id=22298



 dip25 prevents returning a `ref` parameter by `ref`, unless you 
 annotate it `return ref`. It's enabled by default, but only as 
 a deprecation. `-preview=dip25` turns the deprecation into an 
 error. `-preview=dip1000` implies `-preview=dip25`, so there's 
 no need to use both switches.


This is really helpful, thanks!

-- Bastiaan.
Jan 04 2022
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 4 January 2022 at 11:57:41 UTC, Dennis wrote:

 On Monday, 3 January 2022 at 16:06:38 UTC, Bastiaan Veelo wrote:

 But are these bugs in the rigidity of the checks, or do other 
 problems arise when DIP1000 is enabled?


 I'm trying to turn dip1000's 💀 into an 🆗, but Walter wants to 
 go straight to a ✅:
 https://github.com/dlang/dmd/pull/13362#issuecomment-981181277

 On top of this, dip1000 currently has a few more 💀s because:

 - parameters of `pure` functions can be incorrectly inferred 
 `scope`, even when you can escape them with a thrown Exception. 
 The fix is easy, but it requires updating Phobos and excel-d to 
 pass the test suite, which is cumbersome.
 https://issues.dlang.org/show_bug.cgi?id=22221


I talked to Walter about this and I don't think it's the correct 
fix. I've been looking at how to do it otherwise.
Jan 05 2022
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 5 January 2022 at 10:05:02 UTC, Atila Neves wrote:

 I talked to Walter about this and I don't think it's the 
 correct fix. I've been looking at how to do it otherwise.


Why not? Given a signature like:
```D
int fun(string s) pure nothrow;
```
You can assume `s` is `scope`, because there's no channel to 
escape it: return value has no pointers, global variables are not 
accessible in a pure function, there are no other parameters to 
assign it to.

But when the signature is this:
```D
int fun(string s) pure;
```
You can assign `s` to an Exception that gets thrown. How would 
you prevent / detect that?
Jan 05 2022
next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 5 January 2022 at 10:47:50 UTC, Dennis wrote:

 On Wednesday, 5 January 2022 at 10:05:02 UTC, Atila Neves wrote:

 I talked to Walter about this and I don't think it's the 
 correct fix. I've been looking at how to do it otherwise.


 Why not? Given a signature like:
 ```D
 int fun(string s) pure nothrow;
 ```
 You can assume `s` is `scope`, because there's no channel to 
 escape it: return value has no pointers, global variables are 
 not accessible in a pure function, there are no other 
 parameters to assign it to.

 But when the signature is this:
 ```D
 int fun(string s) pure;
 ```
 You can assign `s` to an Exception that gets thrown. How would 
 you prevent / detect that?


You *can*, but it's likely that the vast majority of code 
*won't*. Changing this would break a lot of code, as seen by your 
attempts to fix Phobos.

Another option is making it so pure functions can't throw, but 
that also probably introduces a lot of breakage.
Jan 07 2022
next sibling parent reply bauss <jj_1337 live.dk> writes:
On Friday, 7 January 2022 at 08:39:25 UTC, Atila Neves wrote:

 Another option is making it so pure functions can't throw, but 
 that also probably introduces a lot of breakage.


It would be a somewhat correct approach, because while in theory 
you can throw from pure functions, then you violate the purity by 
observing the thrown exception.

So it doesn't really make much sense to throw in a pure function 
in itself, as using the exception will be a direct violation of 
purity.
Jan 07 2022
next sibling parent Dennis <dkorpel gmail.com> writes:
On Friday, 7 January 2022 at 08:46:27 UTC, bauss wrote:

 So it doesn't really make much sense to throw in a pure 
 function in itself, as using the exception will be a direct 
 violation of purity.


Printing the stack trace of an Exception can only happen when you 
catch it in an impure function. In that case the side effects 
happen there, that doesn't make the function that threw the 
Exception impure.

Catching the Exception in a `pure` function is not a violation 
either, whatever you do with the Exception object in the catch 
block is the same as what you could do if the Exception was 
returned instead of thrown.
Jan 07 2022
prev sibling parent reply Elronnd <elronnd elronnd.net> writes:
On Friday, 7 January 2022 at 08:46:27 UTC, bauss wrote:

 It would be a somewhat correct approach, because while in 
 theory you can throw from pure functions, then you violate the 
 purity by observing the thrown exception.

 So it doesn't really make much sense to throw in a pure 
 function in itself, as using the exception will be a direct 
 violation of purity.


Exceptions can be modeled as return values. This is a strictly 
local rewrite.  So it is no more a violation of purity than is 
writing to memory pointed to by passed-in pointers (and arguably 
less).
Jan 08 2022
parent reply Nick Treleaven <nick geany.org> writes:
On Saturday, 8 January 2022 at 13:33:26 UTC, Elronnd wrote:

 Exceptions can be modeled as return values. This is a strictly 
 local rewrite.  So it is no more a violation of purity than is 
 writing to memory pointed to by passed-in pointers (and 
 arguably less).


They can 'return' to a higher scope than the caller of the 
function. Any stack memory from the caller's frame will be 
corrupted.  safe cannot allow this.
Jan 08 2022
parent Nick Treleaven <nick geany.org> writes:
On Saturday, 8 January 2022 at 18:19:47 UTC, Nick Treleaven wrote:

  safe cannot allow this.


To be clear, I was talking about inferred scope parameters that 
actually escape via a throwable, not about throwing from pure 
functions.
Jan 08 2022
prev sibling parent Dennis <dkorpel gmail.com> writes:
On Friday, 7 January 2022 at 08:39:25 UTC, Atila Neves wrote:

 You *can*, but it's likely that the vast majority of code 
 *won't*. Changing this would break a lot of code, as seen by 
 your attempts to fix Phobos.

 Another option ...


I don't see a first option in your first paragraph. "it's likely 
that the vast majority of code *won't*", sure, but scope 
inference needs to be 100%, 'likely `scope`' is not useful 
information for DMD.


 Another option is making it so pure functions can't throw, but 
 that also probably introduces a lot of breakage.


`pure` `nothrow` functions don't break, it's only `pure` 
functions that throw exceptions that lose their free `scope` 
inference. Disallowing `pure` `throw` functions entirely results 
in a strict superset of the breakage, on top of adding an 
arbitrary restriction to users.
Jan 07 2022
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 5 January 2022 at 10:47:50 UTC, Dennis wrote:

 On Wednesday, 5 January 2022 at 10:05:02 UTC, Atila Neves wrote:

 I talked to Walter about this and I don't think it's the 
 correct fix. I've been looking at how to do it otherwise.


 Why not? Given a signature like:
 ```D
 int fun(string s) pure nothrow;
 ```
 You can assume `s` is `scope`, because there's no channel to 
 escape it: return value has no pointers, global variables are 
 not accessible in a pure function, there are no other 
 parameters to assign it to.


There is one problem with this. The string could still be 
assigned to a non-`Exception` `Throwable` and thrown. If the 
function implementation is also ` safe`, the compiler is going to 
wrongly prevent throwing the exception due to the automatically 
added `scope`.

When you think of it, it means that `assert`s using `s` as the 
message would be forbidden. Or worse, they would be allowed but 
the assert failure printer would then try to access expired 
memory after such an assert has gone boom.
Jan 07 2022
parent Dennis <dkorpel gmail.com> writes:
On Saturday, 8 January 2022 at 00:13:38 UTC, Dukc wrote:

 There is one problem with this. The string could still be 
 assigned to a non-`Exception` `Throwable` and thrown.


Yikes, that's another annoying oversight. I hope we don't have to 
do away with `pure` based `scope` inference entirely. Since 
Errors are only thrown once per program, it could be okay to make 
the assert handler's message string `scope` forcing it to copy it 
when it wants to back trace, but when throwing a custom 
`Throwable` class this is harder to enforce.
Jan 08 2022
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On Monday, 3 January 2022 at 14:27:55 UTC, Bastiaan Veelo wrote:

 As far as I know ` safe` is only really safe with both DIP25 
 and DIP1000 in effect


Conceptually, DIP 1000 does not make your existing  safe code any 
safer. It allows some code to be  safe that had to be  system 
before.

If you just want to write  safe code, and you don't care for 
`scope`, you can ignore DIP 1000. Any safety holes are just bugs, 
with and without `-preview=dip1000`.
Jan 03 2022
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 3 January 2022 at 14:27:55 UTC, Bastiaan Veelo wrote:

 As far as I know ` safe` is only really safe with both DIP25 
 and DIP1000 in effect, and I thought that DIP25 has been 
 integrated for a while now. Nonetheless, DIP25 still appears in 
 the list of previews (`dmd -preview=help`) *as well as* being 
 revertible (`dmd -revert=help`) which adds to the confusion.

 What is in the way of integrating DIP1000 by default?

 -- Bastiaan.


 safe is really safe - what DIPs 25 and 1000 do is make it so 
more code can be  safe.

I'm currently working on making DIP1000 the default.
Jan 04 2022
parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Tuesday, 4 January 2022 at 08:58:04 UTC, Atila Neves wrote:

  safe is really safe - what DIPs 25 and 1000 do is make it so 
 more code can be  safe.


Thanks, that clears things up considerably.


 I'm currently working on making DIP1000 the default.


Great!

-- Bastiaan.
Jan 04 2022