www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - trusted attribute should be replaced with trusted blocks

reply Ogi <ogion.art gmail.com> writes:
There was a discussion a few years ago [1], but now it came up 
again in the light of  safe-by-default DIP. I’ve created a new 
thread for this subject because it’s off-topic to DIP 1028 but 
still deserves attention, especially now when there’s a shift 
towards memory safety.

This proposal is independent to DIP 1028. We can have  trusted 
blocks without  safe by default and vice versa. But they go along 
nicely.

The idea is to remove  trusted as a function attribute and 
instead introduce  trusted blocks:

 safe fun() {
     //safe code here
      trusted {
         //those few lines that require manual checking
     }
     //we are safe again
}

Generally, only a few lines inside  trusted are actually unsafe. 
With  trusted blocks parts that require manual checking will 
stand out. Why should we leave the task of identifying 
problematic places to the reader?

Once the dangerous parts are enclosed inside  trusted blocks, 
memory safety of the rest of the function is guaranteed by a 
compiler. Why should we refuse from the additional checks?

Actually, there’s already a workaround for injecting unsafe 
section to  safe code: just put it inside  trusted lambda that’s 
called immediately. It’s featured in the “How to Write  trusted 
Code” in D blog. Even Walter admits that he’s using this trick in 
Phobos. So why should we resort to dirty hacks that could also 
harm performance and compilation time if we can make it part of a 
language?

Some would say that we need  trusted attribute to distinguish 
functions that are checked by compiler and functions that are 
checked manually. But is  safe actually safe? You can’t tell that 
a “safe” function doesn’t call  trusted functions somewhere along 
the road, or doesn’t use the aforementioned lambda hack. Why 
pretend that  safe guarantees safety if it’s nothing more than a 
pinky promise?

If you think of it, it makes no sense for  trusted to be a 
function attribute. It doesn’t describe its behavior but its 
implementation. If you call a function, it should not be your 
concern;  safe and  trusted are the same thing for you. But these 
two attributes result in two different signatures. If a function 
expects a  safe callback, you can’t pass a  trusted function to 
it without  safe wrapping. If some library makes a function that 
used to be  safe  trusted, that’s a breaking change. Etc, etc. 
Why should we introduce complexity out of nowhere?

Without  trusted the safety design will be much simpler to grasp. 
Currently the are three vague keywords, complex rules on which 
functions can call which and best practices on which attribute 
you should choose. Without  trusted, it’s simple as potato: there 
are  safe and  system functions,  safe require putting unsafe 
parts inside  trusted blocks and can only call other  safe 
functions. We should not disregard the learning experience, 
because with every overcomplicated aspect of the language we lose 
some potential users.

With  safe as default upon us, it’s especially important to get 
safety right.

There is a risk that users would slap  trusted on everything just 
to make it work. While the new syntax wouldn’t stop them all, 
putting a whole function (or an entire module) inside of a block 
would scream code smell.

We can’t expect all users to be world-class programmers. The 
proposed design is more foolproof than the current safety 
trichotomy.
A user could create a  trusted function to do something unsafe 
and do it properly, but also do something that he expects to be 
safe while it’s not (like, calling a  system function thinking 
that it’s  safe). In a  safe function with  trusted blocks it 
wouldn’t be possible to do something unsafe if you don’t mean it.

To sum up,  trusted blocks will make it easier to write safe 
code, read safe code and learn the language. This proposal aligns 
with the current direction of the language towards memory safety.

How much will this change break? Probably not that much, because 
 trusted functions are not common.

To upgrade your code, the proper thing to do would be to put only 
the unsafe parts inside  trusted blocks. It would require some 
manual work, but it’s not for naught, since it would re-enable 
compiler checks outside of  trusted blocks. You can even find 
some bugs your eye could miss!

Of course, you can also just put the whole function body inside a 
 trusted block and call it a day. Not great, but just as bad as 
the current  trusted attribute.

Additional thoughts.

Obviously,  trusted block must not introduce a new scope. If you 
need a new scope, just use double curly.

We could also allow applying  trusted to a single line:
 trusted unsafeFunction();
But I don’t think that’s a good idea, as it would make it just 
too easy to call unsafe functions from  safe code.

With only  safe and  system attributes left, it would make 
perfect sense to rename  system to  unsafe, as suggested by Manu. 
The only ones who will protest are those who use “I write  system 
code” as a pickup line.

It would be possible to write a tool that analyzes code safety 
and e.g. shows how many  trusted lines are in the dub package.

The   should be probably dropped.

[1] 
https://forum.dlang.org/thread/blrglebkzhrilxkbprgh forum.dlang.org
Jan 15
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
 The idea is to remove  trusted as a function attribute and 
 instead introduce  trusted blocks:

  safe fun() {
     //safe code here
      trusted {
         //those few lines that require manual checking
     }
     //we are safe again
 }

 Generally, only a few lines inside  trusted are actually unsafe.
So here's the problem with this approach (which was mentioned by several people in the discussion): the actual safety of a function like this is usually down to the combination of the lines that (in your example) are both inside and outside the trusted block. What that means is that it's important that an external user of the function doesn't just see it as safe, but recognizes that the function -- as a whole -- is one whose promise of safety is conditional on its internals being correct. And that's essentially what trusted is for. So, a better approach would be for the function to be marked up like this: trusted fun () // alerts the outside user { // lines that on their own are provably safe go here system { // these lines are allowed to use system code } // only provably safe lines here again } ... and the compiler's behaviour would be to explicitly verify standard safe rules for all the lines inside the trusted function _except_ the ones inside a system { ... } block. Cf. Steven Schveighoffer's remarks here: https://forum.dlang.org/post/qv7t8b$2h2t$1 digitalmars.com This way the function signature gives a clear indicator to the user which functions are provably safe, and which are safe only on the assumption that the developer has done their job properly.
Jan 15
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 15.01.20 17:54, Joseph Rushton Wakeling wrote:
 On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
[...]
  safe fun() {
     //safe code here
      trusted {
         //those few lines that require manual checking
     }
     //we are safe again
 }
[...]
 So here's the problem with this approach (which was mentioned by several 
 people in the discussion): the actual safety of a function like this is 
 usually down to the combination of the lines that (in your example) are 
 both inside and outside the  trusted block.
Yup. But a proposal could specify that that's the intended meaning for an safe function that contains trusted blocks. Whereas it's more of a cheat when we use trusted nested functions like that. [...]
 So, a better approach would be for the function to be marked up like this:
 
  trusted fun ()    // alerts the outside user
 {
      // lines that on their own are provably safe go here
       system {
          // these lines are allowed to use  system code
      }
      // only provably safe lines here again
 }
 
 ... and the compiler's behaviour would be to explicitly verify standard 
  safe rules for all the lines inside the  trusted function _except_ the 
 ones inside a  system { ... } block.
 
 Cf. Steven Schveighoffer's remarks here: 
 https://forum.dlang.org/post/qv7t8b$2h2t$1 digitalmars.com
 
 This way the function signature gives a clear indicator to the user 
 which functions are provably  safe, and which are safe only on the 
 assumption that the developer has done their job properly.
I don't think that's what Steven had in mind. In that world, safe would be very, very limited, because it couldn't be allowed to call trusted functions. That means safe would only apply to trivial functions, and trusted would assume the role that safe has today. But you'd have to wrap every call from an trusted to another trusted function in an system block. It wouldn't be practical. The real purpose of trusted in that example is to allow the system block in the body, and to signal to reviewers and maintainers that the whole function is unsafe despite the mechanical checks that are done on most of the lines. To a user, trusted functions would still be the same as safe ones. Unfortunately, adding the mechanical checks of safe to trusted would mean breaking all trusted code that exists. So implementing that scheme seems unrealistic. But as Steven says, it can be done when we use trusted blocks instead of system blocks and safe instead of trusted on the function. I.e.: safe fun () { // lines that the compiler accepts as safe go here trusted { // these lines are allowed to use system code } // only safe lines here again } It weakens the meaning of safe somewhat, but it's often treated that way already. There's clearly a need.
Jan 15
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 15 Jan 2020 19:06:11 +0100 schrieb ag0aep6g:

 
 The real purpose of  trusted in that example is to allow the  system
 block in the body, and to signal to reviewers and maintainers that the
 whole function is unsafe despite the mechanical checks that are done on
 most of the lines. To a user,  trusted functions would still be the same
 as  safe ones.
 
 Unfortunately, adding the mechanical checks of  safe to  trusted would
 mean breaking all  trusted code that exists. So implementing that scheme
 seems unrealistic.
 
I think it shouldn't be much of a problem, as there is a very nice transition path: * Add system block support * Add -transition=systemBlocks which enforces system blocks in trusted functions * Users gradually add system blocks to their trusted functions, until everything compiles with -transition=systemBlocks. If you did not add all blocks yet, your code will still compile fine without - transisition=systemBlocks * -transition=systemBlocks becomes default
 But as Steven says, it can be done when we use  trusted blocks instead
 of  system blocks and  safe instead of  trusted on the function. I.e.:
 
       safe fun ()
      {
          // lines that the compiler accepts as  safe go here  trusted {
              // these lines are allowed to use  system code
          }
          // only  safe lines here again
      }
 
 It weakens the meaning of  safe somewhat, but it's often treated that
 way already. There's clearly a need.
I don't really like this. It makes trusted functions completely useless legacy cruft. And there's no longer any way to annotate a function as 'this is 100% safe code', so then you'll have to check every safe function as thoroughly as trusted functions. -- Johannes
Jan 15
next sibling parent ag0aep6g <anonymous example.com> writes:
On 15.01.20 19:38, Johannes Pfau wrote:
 Am Wed, 15 Jan 2020 19:06:11 +0100 schrieb ag0aep6g:
[...]
 I think it shouldn't be much of a problem, as there is a very nice
 transition path:
 
 * Add  system block support
 * Add -transition=systemBlocks which enforces  system blocks in trusted
 functions
 * Users gradually add  system blocks to their trusted functions, until
 everything compiles with -transition=systemBlocks. If you did not add all
 blocks yet, your code will still compile fine without -
 transisition=systemBlocks
 * -transition=systemBlocks becomes default
If that's deemed acceptable, I'm on board. But the alternative requires zero work from users (if we just keep trusted functions around as legacy cruft).
 But as Steven says, it can be done when we use  trusted blocks instead
 of  system blocks and  safe instead of  trusted on the function. I.e.:
[...]
 I don't really like this. It makes  trusted functions completely useless
 legacy cruft. And there's no longer any way to annotate a function as
 'this is 100% safe code', so then you'll have to check every safe
 function as thoroughly as trusted functions.
You already have to check for trusted nested functions if you want to make sure an safe function is really 100% safe. And those already routinely leak their unsafety into the surrounding safe function. I don't see what would change in that regard.
Jan 15
prev sibling parent Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Wednesday, 15 January 2020 at 18:38:25 UTC, Johannes Pfau 
wrote:
 Am Wed, 15 Jan 2020 19:06:11 +0100 schrieb ag0aep6g:

 
 The real purpose of  trusted in that example is to allow the 
  system block in the body, and to signal to reviewers and 
 maintainers that the whole function is unsafe despite the 
 mechanical checks that are done on most of the lines. To a 
 user,  trusted functions would still be the same as  safe ones.
 
 Unfortunately, adding the mechanical checks of  safe to 
  trusted would mean breaking all  trusted code that exists. So 
 implementing that scheme seems unrealistic.
 
I think it shouldn't be much of a problem, as there is a very nice transition path: * Add system block support * Add -transition=systemBlocks which enforces system blocks in trusted functions * Users gradually add system blocks to their trusted functions, until everything compiles with -transition=systemBlocks. If you did not add all blocks yet, your code will still compile fine without - transisition=systemBlocks * -transition=systemBlocks becomes default
 But as Steven says, it can be done when we use  trusted blocks 
 instead of  system blocks and  safe instead of  trusted on the 
 function. I.e.:
 
       safe fun ()
      {
          // lines that the compiler accepts as  safe go here 
  trusted {
              // these lines are allowed to use  system code
          }
          // only  safe lines here again
      }
 
 It weakens the meaning of  safe somewhat, but it's often 
 treated that way already. There's clearly a need.
I don't really like this. It makes trusted functions completely useless legacy cruft. And there's no longer any way to annotate a function as 'this is 100% safe code', so then you'll have to check every safe function as thoroughly as trusted functions.
Oh, I hate that discussion. trusted functions were useless from the start. They are just the same as safe from a caller perspective. And trusted blocks are nothing new either, only the syntax is more ugly because anonymous lambdas are used to simulate them: () trusted { // system stuff here }() And the argument that trusted functions could better be found and that that would increase the safety is invalid. trusted blocks can be searched just as good, and any function containing them still need to be treated with caution.
Jan 16
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 15 January 2020 at 18:06:11 UTC, ag0aep6g wrote:
 I don't think that's what Steven had in mind. In that world, 
  safe would be very, very limited, because it couldn't be 
 allowed to call  trusted functions.
Well, apologies to Steven if I've misinterpreted his proposal. But what I had in mind was that safe would be able to call trusted just as it does now. So, put that together with what I wrote above, and you have something that allows better validation of the internals of trusted functions, and still gives the user clarity about which functions are safe in their own terms, and which are safe based on some programmer provided guarantees.
Jan 15
parent reply ag0aep6g <anonymous example.com> writes:
On 15.01.20 19:41, Joseph Rushton Wakeling wrote:
 Well, apologies to Steven if I've misinterpreted his proposal. But what 
 I had in mind was that  safe would be able to call  trusted just as it 
 does now.
 
 So, put that together with what I wrote above, and you have something 
 that allows better validation of the internals of  trusted functions, 
 and still gives the user clarity about which functions are safe in their 
 own terms, and which are safe based on some programmer provided guarantees.
You're saying that an safe function `f` is somehow more guaranteed to be safe than an trusted function `g`, even though `f` may be calling `g`. I don't see how that makes sense. ---- R f(P params) safe { return g(params); } R g(P params) trusted { /* ... whatever ... */ } ---- Any assumptions you have about `f` better also be true about `g`.
Jan 15
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 15 Jan 2020 20:14:13 +0100 schrieb ag0aep6g:

 On 15.01.20 19:41, Joseph Rushton Wakeling wrote:
 Well, apologies to Steven if I've misinterpreted his proposal. But what
 I had in mind was that  safe would be able to call  trusted just as it
 does now.
 
 So, put that together with what I wrote above, and you have something
 that allows better validation of the internals of  trusted functions,
 and still gives the user clarity about which functions are safe in
 their own terms, and which are safe based on some programmer provided
 guarantees.
You're saying that an safe function `f` is somehow more guaranteed to be safe than an trusted function `g`, even though `f` may be calling `g`. I don't see how that makes sense. ---- R f(P params) safe { return g(params); } R g(P params) trusted { /* ... whatever ... */ } ---- Any assumptions you have about `f` better also be true about `g`.
A trusted function in D is supposed to have a safe interface: For each possible combination of argument values, a trusted function must not cause invalid memory accesses. This is the only reason you can safely call trusted functions from safe functions. It's the programmers responsibility to design trusted function APIs in a way so they can't be misused. -- Johannes
Jan 15
parent ag0aep6g <anonymous example.com> writes:
On 15.01.20 22:08, Johannes Pfau wrote:
 Am Wed, 15 Jan 2020 20:14:13 +0100 schrieb ag0aep6g:
 
 On 15.01.20 19:41, Joseph Rushton Wakeling wrote:
[...]
 So, put that together with what I wrote above, and you have something
 that allows better validation of the internals of  trusted functions,
 and still gives the user clarity about which functions are safe in
 their own terms, and which are safe based on some programmer provided
 guarantees.
You're saying that an safe function `f` is somehow more guaranteed to be safe than an trusted function `g`, even though `f` may be calling `g`. I don't see how that makes sense.
[...]
 A trusted function in D is supposed to have a safe interface: For each
 possible combination of argument values, a trusted function must not
 cause invalid memory accesses. This is the only reason you can safely
 call trusted functions from safe functions. It's the programmers
 responsibility to design trusted function APIs in a way so they can't be
 misused.
I completely agree, but I'm not sure how it's related to my rejection of Joseph's statement that trusted "gives the user [= caller] clarity about which functions [...] are safe based on some programmer provided guarantees". The distinction between safe and trusted matters when looking at the implementation of a function. It doesn't matter when calling an safe/ trusted function, precisely because trusted function must have safe interfaces just like safe functions.
Jan 15
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 15 January 2020 at 19:14:13 UTC, ag0aep6g wrote:
 You're saying that an  safe function `f` is somehow more 
 guaranteed to be safe than an  trusted function `g`, even 
 though `f` may be calling `g`. I don't see how that makes sense.
No, I'm saying that it is useful to have a clear distinction, visible in the API declaration, between functions whose promise of memory safety can be validated by the compiler, versus functions whose promise of memory safety cannot be validated by the compiler. It's a practical distinction, because it gives the developer clarity about which parts of a codebase need to be examined in order to rule out the possibility of memory safety violations.
Jan 15
parent reply ag0aep6g <anonymous example.com> writes:
On Wednesday, 15 January 2020 at 23:16:21 UTC, Joseph Rushton 
Wakeling wrote:
 No, I'm saying that it is useful to have a clear distinction, 
 visible in the API declaration, between functions whose promise 
 of memory safety can be validated by the compiler, versus 
 functions whose promise of memory safety cannot be validated by 
 the compiler.
You're saying that it is useful for me as a user of Phobos when I see that, for example, std.stdio.File.open is trusted [1], right? I disagree. It doesn't mean anything other than safe in that context. Knowing that the function is trusted instead of safe is of zero use to a user of Phobos. It cannot help them make any decision whatsoever.
 It's a practical distinction, because it gives the developer 
 clarity about which parts of a codebase need to be examined in 
 order to rule out the possibility of memory safety violations.
I.e., it's a tool for library authors, not for library users. You clearly spoke of an "external" or "outside" user of the function before. [1] https://dlang.org/phobos/std_stdio.html#.File.open
Jan 15
next sibling parent reply IGotD- <nise nise.com> writes:
On Wednesday, 15 January 2020 at 23:46:11 UTC, ag0aep6g wrote:
 You're saying that it is useful for me as a user of Phobos when 
 I see that, for example, std.stdio.File.open is  trusted [1], 
 right?
Speaking of libraries, as you can see many calls will go the C library which is of course is unsafe code. The only way I can see it is that safe code should be able to call unsafe code by default, for convenience. Now there is a possibility that the programmer wants that the code shall not call any unsafe code what so ever. Then I think that a function attribute should be added that tells that the code is not allowed to call any unsafe code down the line. This is better because. 1. It is a restriction set by the programmer and not some arbitrary promise from a third party. 2. It is absolute. The compiler can check this, either it hits unsafe or it doesn't. It is either true or false and it cannot be questioned.
Jan 15
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 16.01.20 01:09, IGotD- wrote:
 On Wednesday, 15 January 2020 at 23:46:11 UTC, ag0aep6g wrote:
 You're saying that it is useful for me as a user of Phobos when I see 
 that, for example, std.stdio.File.open is  trusted [1], right?
Speaking of libraries, as you can see many calls will go the C library which is of course is unsafe code. The only way I can see it is that safe code should be able to call unsafe code by default, for convenience. ...
That's nonsense, it completely undermines safe, because many sensible system functions do not have a safe interface. safe code shouldn't be able to directly call `free` from the C standard library on arbitrary pointers.
 Now there is a possibility that the programmer wants that the code shall 
 not call any unsafe code what so ever.
trusted functions are not unsafe, it's just not checked by the compiler.
 Then I think that a function 
 attribute should be added that tells that the code is not allowed to 
 call any unsafe code down the line. This is better because.
 
 1. It is a restriction set by the programmer and not some arbitrary 
 promise from a third party.
It's an arbitrary promise from the compiler implementation...
 2. It is absolute. The compiler can check this, either it hits unsafe or 
 it doesn't. It is either true or false and it cannot be questioned.
 
This is not better. Why do you trust compilers and standard library behave according to the specification but are unwilling to do that for any other code that cannot be verified by the compiler? You are making an arbitrary distinction where there is none. The point of safe is that if you write safe code (and no trusted code), memory corruption is not supposed to occur; any memory corruption that occurs anyway is not your own fault. (Except maybe for not doing due diligence when evaluating a potential dependency before using it.)
Jan 15
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 15 January 2020 at 23:46:11 UTC, ag0aep6g wrote:
 You're saying that it is useful for me as a user of Phobos when 
 I see that, for example, std.stdio.File.open is  trusted [1], 
 right?
Yes, I am saying exactly that.
 I disagree. It doesn't mean anything other than  safe in that 
 context. Knowing that the function is  trusted instead of  safe 
 is of zero use to a user of Phobos. It cannot help them make 
 any decision whatsoever.
I see what you're getting at here -- you mean that if we're treating this function as a black box that I have no influence over, then both safe and trusted mean the same thing in terms of how that black box ought to behave. But that doesn't mean the distinction isn't useful. It gives me a clear indication that the code of this function (note, _the code of this function_, not the code of other functions it might call) could contain bugs that would allow memory safety violations. Contrast this with, say, Rust, where any function can do something like this: fn foo(x: &[int]) { unsafe { // any old horrible stuff can happen here } } ... where any number of memory safety bugs could exist in that `unsafe { ... }` block, and yet there is no possible clue from the API that this could be going on. The fact that in a similar situation D forces you to annotate the function with ` trusted`, and alert users to the _possibility_ that memory safety bugs could exist within the code of this function, is useful information even if you can't access the source code.
 I.e., it's a tool for library authors, not for library users. 
 You clearly spoke of an "external" or "outside" user of the 
 function before.
It's clearly much less useful to anyone who doesn't have access to the source code (which doesn't mean it's not useful at all). But in general, given the ability to read and search the source code (which users as well as authors can do), it's very useful to be able to ask the question: "Of the code in this project that claims to be memory safe, which bits could actually contain memory safety bugs?"
Jan 15
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On Thursday, 16 January 2020 at 00:21:21 UTC, Joseph Rushton 
Wakeling wrote:
 I see what you're getting at here -- you mean that if we're 
 treating this function as a black box that I have no influence 
 over, then both  safe and  trusted mean the same thing in terms 
 of how that black box ought to behave.
Exactly.
 But that doesn't mean the distinction isn't useful.  It gives 
 me a clear indication that the code of this function (note, 
 _the code of this function_, not the code of other functions it 
 might call) could contain bugs that would allow memory safety 
 violations.
And that isn't useful to a user in any way. [...]
 The fact that in a similar situation D forces you to annotate 
 the function with ` trusted`, and alert users to the 
 _possibility_ that memory safety bugs could exist within the 
 code of this function, is useful information even if you can't 
 access the source code.
I don't agree. trusted doesn't alert you any more of the possibility of a memory safety bug than safe. You can't assume that an safe function won't corrupt your memory any more than you can assume the same about an trusted function. [...]
 It's clearly much less useful to anyone who doesn't have access 
 to the source code (which doesn't mean it's not useful at all).
(It is useless, though.)
  But in general, given the ability to read and search the 
 source code (which users as well as authors can do), it's very 
 useful to be able to ask the question: "Of the code in this 
 project that claims to be memory safe, which bits could 
 actually contain memory safety bugs?"
Yes, but you find those interesting bits by grepping over the source code, not by looking at the attributes of public functions. Many safe functions have trusted innards that don't show up in API documentation.
Jan 15
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 16 January 2020 at 00:40:12 UTC, ag0aep6g wrote:
 And that isn't useful to a user in any way.
The fact that I personally find it useful -- and going by this discussion thread, so do some others -- means that this claim can't possibly be true.
 I don't agree.  trusted doesn't alert you any more of the 
 possibility of a memory safety bug than  safe. You can't assume 
 that an  safe function won't corrupt your memory any more than 
 you can assume the same about an  trusted function.
safe on a function tells you no more than it promises: that the compiler will alert you to any memory safety violations it is able to detect in that function's implementation. If it does not detect any, that could be because none are present, or it could be that at some point nested inside the code it calls there is something trusted that the compiler does not attempt to validate. trusted on a function, on the other hand, explicitly tells you that its safety is contingent on something other than compiler validation, and that you should not expect the compiler to validate _any_ part of its internals. Do you seriously not see any difference between an annotation that tells you: "this function should be memory-safe, and the compiler will attempt to validate this to the best of its ability" versus, "this function should be memory-safe, but the compiler will not make any attempt to validate that"? And do you seriously not see any value in having that distinction clear? Obviously the allegedly safe function _could_ just be a thin wrapper around a trusted lambda containing all sorts of horrible crack. But on balance of probabilities, in any library written by vaguely competent people, that should probably not be your _first_ assumption.
 (It is useless, though.)
Don't mistake "I don't find it useful" for "No one finds it useful".
 Yes, but you find those interesting bits by grepping over the 
 source code, not by looking at the attributes of public 
 functions. Many  safe functions have  trusted innards that 
 don't show up in API documentation.
FWIW I do think it may be a bit of a code smell or anti-pattern to have nested trusted functions inside safe functions. Of course, there's still the possibility that a safe function can call some external but private trusted function, but it reduces the attack space if the internal code of a safe function can be fully validated _in its own terms_ (i.e. if it can be validated that there is nothing _inside the function_ that could cause a memory safety bug). BTW, that's part of the motivation for my suggestions at the beginning of this thread about how trusted could be improved: https://forum.dlang.org/post/heujvxcsppiyagxfvliv forum.dlang.org If the compiler would validate all the contents of trusted functions except for lines contained in ` system { ... }` blocks (see the original example if it's not clear what I mean), then that would probably be a preferred way to achieve what developers currently do using safe functions with some nested trusted lambdas. And that would probably mean that APIs would be annotated in a more informative way about the _true_ risk of memory safety bugs.
Jan 15
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.01.20 02:43, Joseph Rushton Wakeling wrote:
 
 Do you seriously not see any difference between an annotation that tells 
 you: "this function should be memory-safe, and the compiler will attempt 
 to validate this to the best of its ability" versus, "this function 
 should be memory-safe, but the compiler will not make any attempt to 
 validate that"?  And do you seriously not see any value in having that 
 distinction clear?
It's an implementation detail. If you care about the distinction, you should check out the function's implementation, not its signature.
Jan 15
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 16 January 2020 at 01:53:18 UTC, Timon Gehr wrote:
 It's an implementation detail. If you care about the 
 distinction, you should check out the function's 
 implementation, not its signature.
Sure. But on a practical day-to-day basis, safe vs trusted signatures help to prioritize one's allocation of care somewhat. I'm coming to the conclusion that much of the differences of opinion in this thread are between folks who want to see things as absolutes, and folks who recognize that these features are tools for mitigating risk, not eliminating it.
Jan 15
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.01.20 03:06, Joseph Rushton Wakeling wrote:
 On Thursday, 16 January 2020 at 01:53:18 UTC, Timon Gehr wrote:
 It's an implementation detail. If you care about the distinction, you 
 should check out the function's implementation, not its signature.
Sure. But on a practical day-to-day basis, safe vs trusted signatures help to prioritize one's allocation of care somewhat. ...
You have to be careful when writing a trusted function, not when calling it. If you do not trust a given library, there is no reason to be more careful around a trusted API than around a safe API, as they do not mean different things. safe does not fully eliminate risk of memory corruption in practice, but that does not mean there is anything non-absolute about the specifications of the attributes. As I am sure you understand, if you see a safe function signature, you don't know that its implementation is not a single trusted function call, so the difference in signature is meaningless unless you adhere to specific conventions (which the library you will be considering to use as a dependency most likely will not do).
 I'm coming to the conclusion that much of the differences of opinion in 
 this thread are between folks who want to see things as absolutes, and 
 folks who recognize that these features are tools for mitigating risk, 
 not eliminating it.
 
I was not able to figure out a response to this sentence that is both polite and honest.
Jan 15
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 16 January 2020 at 03:34:26 UTC, Timon Gehr wrote:
 You have to be careful when writing a  trusted function, not 
 when calling it. If you do not trust a given library, there is 
 no reason to be more careful around a  trusted API than around 
 a  safe API, as they do not mean different things.
Thanks for being patient enough to clarify the issues here. This helps to make much clearer the thinking behind the point of view that's been articulated elsewhere in this discussion, that the trusted/ safe distinction is there for library maintainers rather than users. I agree that it's _primarily_ there for maintainers, but I don't agree that it has _no_ value for users (more on that below). But your observations do make clear how unreasonable my rather judgemental "I'm coming to the conclusion ..." remarks were, so apologies to everyone for that. (It's no excuse, but I was quite tired when I wrote that, and so probably not exercising as much good judgement as I ought.)
  safe does not fully eliminate risk of memory corruption in 
 practice, but that does not mean there is anything non-absolute 
 about the specifications of the attributes.
Would we be able to agree that the absolute part of the spec of both amounts to, "The emergence of a memory safety problem inside this function points to a bug either in the function itself or in the initialization of the data that is passed to it" ... ? (In the latter case I'm thinking that e.g. one can have a perfectly, provably correct safe function taking a slice as input, and its behaviour can still get messed up because the user initializes a slice in some crazy unsafe way and passes that in.)
 As I am sure you understand, if you see a  safe function 
 signature, you don't know that its implementation is not a 
 single  trusted function call
Yes, on this we agree. (I even mentioned this case in one of my posts above.)
 so the difference in signature is meaningless unless you adhere 
 to specific conventions
Here's where I think we start having a disagreement. I think it is meaningful to be able to distinguish between "The compiler will attempt to validate the memory safety of this function to the extent possible given the trusted assumptions injected by the developer" (which _might_ be the entirety of the function), versus "The safety of this function will definitely not be validated in any way by the compiler". Obviously that's _more_ helpful to the library authors than users, but it's still informative to the user: it's saying that while the _worst case_ assumptions are the same (100% unvalidated), the best case are not.
 (which the library you will be considering to use as a 
 dependency most likely will not do).
Obviously in general one should not assume virtue on the part of library developers. But OTOH in a day-to-day practical working scenario, where one has to prioritize how often one wants to deep-dive into implementation details -- versus just taking a function's signature and docs at face value and only enquiring more deeply if something breaks -- it's always useful to have a small hint about the best vs. worst case scenarios. It's not that safe provides a stronger guarantee than trusted, it's that trusted makes clear that you are definitely in worst-case territory. It's not a magic bullet, it's just another data point that helps inform the question of whether one might want to deep-dive up front or not (a decision which might be influenced by plenty of other factors besides memory safety concerns). The distinction only becomes meaningless if one is unable to deep-dive and explore the library code.
 I'm coming to the conclusion that much of the differences of 
 opinion in this thread are between folks who want to see 
 things as absolutes, and folks who recognize that these 
 features are tools for mitigating risk, not eliminating it.
 
I was not able to figure out a response to this sentence that is both polite and honest.
Well FWIW, don't feel the need to be polite to me if you don't think I deserve it :-) But in any case, apologies again to everyone for those remarks. They were unfair and based on a misinterpretation of the positions being articulated.
Jan 16
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/16/20 6:50 AM, Joseph Rushton Wakeling wrote:

 Obviously in general one should not assume virtue on the part of library 
 developers.  But OTOH in a day-to-day practical working scenario, where 
 one has to prioritize how often one wants to deep-dive into 
 implementation details -- versus just taking a function's signature and 
 docs at face value and only enquiring more deeply if something breaks -- 
 it's always useful to have a small hint about the best vs. worst case 
 scenarios.
 
 It's not that  safe provides a stronger guarantee than  trusted, it's 
 that  trusted makes clear that you are definitely in worst-case 
 territory.  It's not a magic bullet, it's just another data point that 
 helps inform the question of whether one might want to deep-dive up 
 front or not (a decision which might be influenced by plenty of other 
 factors besides memory safety concerns).
 
In fact, because of how the system works, safe code is LESS likely to mean what you think. If you see a safe function, it just means "some of this is mechanically checked". It doesn't mean that the function is so much more solid than a trusted function that you can skip the review. It can have trusted escapes that force the whole function into the realm of needing review. I would say that today, safe and trusted are indistinguishable from each other as a user of the function. If we moved to a scheme more like I was writing about in the post you quoted, then they actually do start to take on a more solid meaning. It's still not fool-proof -- safe functions can call trusted functions, which can call system functions. BUT if everyone does the job they should be doing, then you shouldn't be able to call trusted functions and corrupt memory, and you should not have to necessarily review safe functions. There are still cases where you still have to review functions that are safe which do not have inner functions that are trusted. These are cases where data that is usually accessible to safe functions can cause memory problems in conjunction with trusted functions. When you need to break the rules, it's very hard to contain where the rule breaking stops. -Steve
Jan 16
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 16 January 2020 at 15:45:46 UTC, Steven 
Schveighoffer wrote:
 In fact, because of how the system works,  safe code is LESS 
 likely to mean what you think.
I'm not quite sure I follow what you mean here, can you clarify/explain?
 If you see a  safe function, it just means "some of this is 
 mechanically checked". It doesn't mean that the function is so 
 much more solid than a  trusted function that you can skip the 
 review. It can have trusted escapes that force the whole 
 function into the realm of needing review.
Yes, agreed. And hence why your proposal for how to improve trusted really makes sense.
 If we moved to a scheme more like I was writing about in the 
 post you quoted, then they actually do start to take on a more 
 solid meaning. It's still not fool-proof --  safe functions can 
 call  trusted functions, which can call  system functions. BUT 
 if everyone does the job they should be doing, then you 
 shouldn't be able to call  trusted functions and corrupt 
 memory, and you should not have to necessarily review  safe 
 functions.
Yes, this was exactly how I interpreted your proposal.
 There are still cases where you still have to review functions 
 that are  safe which do not have inner functions that are 
 trusted. These are cases where data that is usually accessible 
 to safe functions can cause memory problems in conjunction with 
 trusted functions. When you need to break the rules, it's very 
 hard to contain where the rule breaking stops.
For example, in a class or struct implementation where a private variable can be accessed by both safe and trusted methods ... ?
Jan 16
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/16/20 1:08 PM, Joseph Rushton Wakeling wrote:
 On Thursday, 16 January 2020 at 15:45:46 UTC, Steven Schveighoffer wrote:
 In fact, because of how the system works,  safe code is LESS likely to 
 mean what you think.
I'm not quite sure I follow what you mean here, can you clarify/explain?
For example, I want a safe function that uses malloc to allocate, and free to deallocate. Perhaps that is just scratch space and it's just an implementation detail. I want everything *else* in the function to be safe. So I have to mark the function safe, not trusted. Otherwise I don't get the compiler checks. In this way, safe cannot really be used as a marker of "don't need to manually verify" because it's the only way to turn on the mechanical checking. So there is more incentive to mark code safe than trusted at the function level. I guess I should have worded it as, you are probably going to see safe prototypes that more often then not still need checking. Same goes for template functions. How do you even know whether it can be safe or not? You can try it, but that doesn't mean there's no trusted blocks inside. I just don't see the practicality or validity of worrying about trusted functions more than safe ones from a user perspective. That being said, code out there is almost always too trusting when marking functions trusted. They should be small and easily reviewable. The longer the function, the more chances for assumptions to sneak in.
 There are still cases where you still have to review functions that 
 are  safe which do not have inner functions that are trusted. These 
 are cases where data that is usually accessible to safe functions can 
 cause memory problems in conjunction with trusted functions. When you 
 need to break the rules, it's very hard to contain where the rule 
 breaking stops.
For example, in a class or struct implementation where a private variable can be accessed by both safe and trusted methods ... ?
A recent example was a tagged union [1]. The tag is just an integer or boolean indicating which member of the union is valid. As long as the tag matches which actual element of the union is valid, you can use trusted functions to access the union member. However, safe code is able to twiddle the tag without the compiler complaining. The trusted code is expecting the link between the union member that is valid and the tag. In other words, you can muck with the tag all day long in safe land, even in a completely safe function. But it may violates the assumptions that the trusted functions make, making the other parts unsafe. Therefore, you have to review the whole type, even the safe calls, to make sure none of them violates the invariant. And this is why some folks (ag0aep6g) disagree that trusted functions can be valid in this situation -- they have to be valid for ALL inputs in ALL contexts, because the alternative is that you have to manually check safe code. I can live with the idea that safe code needs checking within context, as long as it helps me ensure that *most* of the stuff is right. The other option is to somehow use the compiler to enforce the semantic, like marking the *data* system. In other words you are telling the compiler "I know that it's normally safe to change this tag, but in this case, you can't, because it will mess things up elsewhere". -Steve [1] https://github.com/dlang/phobos/pull/7347
Jan 16
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 16, 2020 at 02:18:09PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
 On 1/16/20 1:08 PM, Joseph Rushton Wakeling wrote:
[...]
 For example, in a class or struct implementation where a private
 variable can be accessed by both  safe and  trusted methods ... ?
A recent example was a tagged union [1]. The tag is just an integer or boolean indicating which member of the union is valid. As long as the tag matches which actual element of the union is valid, you can use trusted functions to access the union member. However, safe code is able to twiddle the tag without the compiler complaining. The trusted code is expecting the link between the union member that is valid and the tag. In other words, you can muck with the tag all day long in safe land, even in a completely safe function. But it may violates the assumptions that the trusted functions make, making the other parts unsafe.
Good example! So in this case, the trust really is between the tag and the union, not so much in the trusted function itself. The trusted function is really just *assuming* the validity of the correspondence between the tag and the union. Without encoding this context somehow, the compiler cannot guarantee that some outside code ( safe code, no less!) won't break the invariant and thereby invalidate the trusted function.
 Therefore, you have to review the whole type, even the safe calls, to
 make sure none of them violates the invariant.
:-( And I guess this extends to any type that has trusted methods that make assumptions about the data stored in the type. Which logically leads to the idea that the data itself should be tagged somehow, and therefore your idea of tagging the *data*. [...]
 The other option is to somehow use the compiler to enforce the
 semantic, like marking the *data*  system. In other words you are
 telling the compiler "I know that it's normally safe to change this
 tag, but in this case, you can't, because it will mess things up
 elsewhere".
[...] So it's basically a way of tainting any code that touches the data, such that you're not allowed to touch the data unless you are system or trusted. This actually makes a lot of sense, the more I think about it. Take a pointer T*, for example. Why is it illegal to modify the pointer (i.e. do pointer arithmetic with T*) in safe code? The act of changing the pointer doesn't in itself corrupt memory. What corrupts memory is when the pointer is changed in a way that *breaks assumptions* laid upon it by safe code, such that when we subsequently dereference it, we may end up in UB land. We may say that pointer dereference is trusted, in the same sense as the tagged union access you described -- it's assuming that the pointer points to something valid -- and our pointer arithmetic has just broken that assumption. Similarly, it's illegal to manipulate the .ptr field of an int[] in safe code: not because that in itself corrupts memory, but because that breaks the assumption that an expression like int[i] will access valid data (provided it's within the bounds of .length). Again, the manipulation of .ptr is system, and array dereference with [i] is trusted in the same sense as tagged union access: arr[i] *assumes* that there's a valid correspondence between .ptr, .length, and whatever .ptr points to. If therefore we prohibit manipulating .ptr in safe code but allow arr[i] (which makes assumptions about .ptr), then it makes sense to prohibit manipulation of the tagged union's tag field and allow the trusted member to look up union fields. It could even be argued that union field lookup ought to be safe in the same way arr[i] is safe: it won't corrupt memory or read out-of-bounds, contingent upon the assumptions laid on an int[]'s .ptr and .length fields not to have been broken. IOW, we're talking about "sensitive data" here, i.e., data that must not be modified in the wrong ways because it will break assumptions that other code have laid upon it. Manipulating pointers is system because pointers are sensitive data. Manipulating ints is safe because ints are not sensitive data. In the same vein, the tag field of a tagged union is sensitive data, and therefore manipulating it must be system, i.e., only a trusted function ought to be allowed to do that. By default, safe comes with its own set of what constitutes sensitive data, and operations on such data are rightfully restricted. Allowing the user to tag data as sensitive seems to be a logical extension of safe. T -- It said to install Windows 2000 or better, so I installed Linux instead.
Jan 16
parent ag0aep6g <anonymous example.com> writes:
On Thursday, 16 January 2020 at 20:47:54 UTC, H. S. Teoh wrote:
 On Thu, Jan 16, 2020 at 02:18:09PM -0500, Steven Schveighoffer
[...]
 The other option is to somehow use the compiler to enforce the 
 semantic, like marking the *data*  system. In other words you 
 are telling the compiler "I know that it's normally safe to 
 change this tag, but in this case, you can't, because it will 
 mess things up elsewhere".
[...] So it's basically a way of tainting any code that touches the data, such that you're not allowed to touch the data unless you are system or trusted.
[...]
 By default,  safe comes with its own set of what constitutes 
 sensitive data, and operations on such data are rightfully 
 restricted.  Allowing the user to tag data as sensitive seems 
 to be a logical extension of  safe.
For reference, here's the upcoming DIP: https://github.com/dlang/DIPs/pull/179
Jan 16
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 16, 2020 at 10:45:46AM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]
 In fact, because of how the system works,  safe code is LESS likely to
 mean what you think.
 
 If you see a  safe function, it just means "some of this is
 mechanically checked". It doesn't mean that the function is so much
 more solid than a  trusted function that you can skip the review. It
 can have trusted escapes that force the whole function into the realm
 of needing review.
[...] Yeah, that's the part that makes me uncomfortable every time I see a trusted lambda inside a function that *clearly* does not sport a safe interface, as in, its safety is dependent on the surrounding code. I think it would be better to completely outlaw trusted blocks inside a safe function, and to require calling an external trusted function. And inside a trusted function, most of the body will still be subject to safe checks, except for explicitly-marked out system scopes. This way, the meaning of safe becomes "this function has been thoroughly mechanically checked, and it will not corrupt memory provided all trusted functions that it calls operate correctly". And trusted would mean "this function has been mechanically checked except for those blocks explicitly marked safe, which must be reviewed manually together with the rest of the function body". The latter is useful as a preventative measure: if you allow unrestricted use of system code inside a trusted function, then every single code change made to that function requires the manual re-evaluation of the entire function, because you don't know if you've inadvertently introduced a safety hole. Not allowing system code by default means if you accidentally slip up outside of the isolated system blocks, the compiler will complain and you will fix it. This way, you minimize the surface area of potential problems to a smaller scope inside the trusted function, and leverage the compiler's automatic checks to catch your mistakes, as opposed to having zero safeguards as soon as you slap trusted on your function. T -- Try to keep an open mind, but not so open your brain falls out. -- theboz
Jan 16
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 16.01.20 12:50, Joseph Rushton Wakeling wrote:
 On Thursday, 16 January 2020 at 03:34:26 UTC, Timon Gehr wrote:
 ...
  safe does not fully eliminate risk of memory corruption in practice, 
 but that does not mean there is anything non-absolute about the 
 specifications of the attributes.
Would we be able to agree that the absolute part of the spec of both amounts to, "The emergence of a memory safety problem inside this function points to a bug either in the function itself or in the initialization of the data that is passed to it" ... ? ...
More or less. Two points: - The _only_ precondition a trusted/ safe function can assume for guaranteeing no memory corruption is that there is no preexisting memory corruption. - For callers that threat the library as a black-box, this definition is essentially sufficient. (This is why there is not really a reason to treat the signatures differently to the point where changing from one to the other is a breaking API change.) White-box callers get the additional language guarantee that if the function corrupts memory, that happens when it executed some bad trusted code and this is the motivation behind having both safe and trusted. system exists because in low-level code, sometimes you want to write or use functions that have highly non-trivial preconditions for ensuring no memory corruption happens.
 (In the latter case I'm thinking that e.g. one can have a perfectly, 
 provably correct  safe function taking a slice as input, and its 
 behaviour can still get messed up because the user initializes a slice 
 in some crazy unsafe way and passes that in.)
 ...
That is preexisting memory corruption. If you use trusted/ system code to destroy an invariant that the safe part of the language assumes to hold for a given type, you have corrupted memory.
 As I am sure you understand, if you see a  safe function signature, 
 you don't know that its implementation is not a single  trusted 
 function call
Yes, on this we agree.  (I even mentioned this case in one of my posts above.)
 so the difference in signature is meaningless unless you adhere to 
 specific conventions
Here's where I think we start having a disagreement.  I think it is meaningful to be able to distinguish between "The compiler will attempt to validate the memory safety of this function to the extent possible given the trusted assumptions injected by the developer" (which _might_ be the entirety of the function), versus "The safety of this function will definitely not be validated in any way by the compiler". Obviously that's _more_ helpful to the library authors than users, but it's still informative to the user: it's saying that while the _worst case_ assumptions are the same (100% unvalidated), the best case are not. ...
It is possible to write a trusted function that consists of a single call to a safe function, so you are assuming a convention where people do not call safe code from trusted code in certain ways. Anyway, my central point was that it is an implementation detail. That does not mean it is necessarily useless to a user in all circumstances, but that someone who writes a library will likely choose to hide it.
 (which the library you will be considering to use as a dependency most 
 likely will not do).
Obviously in general one should not assume virtue on the part of library developers.  But OTOH in a day-to-day practical working scenario, where one has to prioritize how often one wants to deep-dive into implementation details -- versus just taking a function's signature and docs at face value and only enquiring more deeply if something breaks -- it's always useful to have a small hint about the best vs. worst case scenarios. ...
Right now, the library developer has a valid incentive to actively avoid trusted functions in their API. This is because it is always possible, trusted is an implementation detail and changing this detail can in principle break dependent code. (E.g., a template instantiated with a safe delegate will give you a different instantiation from the same template instantiated with a trusted delegate, and if e.g., you have some static cache in your template function, a change from safe to trusted in some API can silently slow down the downstream application by a factor of two, change iteration orders through hash tables, etc.)
 It's not that  safe provides a stronger guarantee than  trusted, it's 
 that  trusted makes clear that you are definitely in worst-case 
 territory.  It's not a magic bullet, it's just another data point that 
 helps inform the question of whether one might want to deep-dive up 
 front or not (a decision which might be influenced by plenty of other 
 factors besides memory safety concerns).
 
 The distinction only becomes meaningless if one is unable to deep-dive 
 and explore the library code.
 ...
I just think that if you are willing to do that, you should use e.g. grep, not the function signature where a competent library author will likely choose to hide trusted as an implementation detail.
Jan 16
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Thu, 16 Jan 2020 04:34:26 +0100 schrieb Timon Gehr:

 On 16.01.20 03:06, Joseph Rushton Wakeling wrote:
 On Thursday, 16 January 2020 at 01:53:18 UTC, Timon Gehr wrote:
 It's an implementation detail. If you care about the distinction, you
 should check out the function's implementation, not its signature.
Sure. But on a practical day-to-day basis, safe vs trusted signatures help to prioritize one's allocation of care somewhat. ...
You have to be careful when writing a trusted function, not when calling it. If you do not trust a given library, there is no reason to be more careful around a trusted API than around a safe API, as they do not mean different things. safe does not fully eliminate risk of memory corruption in practice, but that does not mean there is anything non-absolute about the specifications of the attributes. As I am sure you understand, if you see a safe function signature, you don't know that its implementation is not a single trusted function call, so the difference in signature is meaningless unless you adhere to specific conventions (which the library you will be considering to use as a dependency most likely will not do).
 I'm coming to the conclusion that much of the differences of opinion in
 this thread are between folks who want to see things as absolutes, and
 folks who recognize that these features are tools for mitigating risk,
 not eliminating it.
 
 
I was not able to figure out a response to this sentence that is both polite and honest.
I'm curious, what do you think would be the ideal scheme if we could redesign it from scratch? Only safe/ system as function attributes and trusted (or system) blocks which can be used in safe functions? -- Johannes
Jan 17
next sibling parent reply Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Friday, 17 January 2020 at 08:10:48 UTC, Johannes Pfau wrote:

 I'm curious, what do you think would be the ideal scheme if we 
 could redesign it from scratch? Only  safe/ system as function 
 attributes and  trusted (or  system) blocks which can be used 
 in  safe functions?
Yes, pretty much the same as in Rust. And we can get there: just need to introduce trusted blocks (which would not a change what is already possible but only a simplification of syntax) and changing all trusted functions to safe (with one big trusted block around the function body). These are pretty small changes, but would yield a much better structure.
Jan 17
next sibling parent reply Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Friday, 17 January 2020 at 09:21:30 UTC, Dominikus Dittes 
Scherkl wrote:
 On Friday, 17 January 2020 at 08:10:48 UTC, Johannes Pfau wrote:

 I'm curious, what do you think would be the ideal scheme if we 
 could redesign it from scratch? Only  safe/ system as function 
 attributes and  trusted (or  system) blocks which can be used 
 in  safe functions?
Yes, pretty much the same as in Rust. And we can get there: just need to introduce trusted blocks (which would not a change what is already possible but only a simplification of syntax) and changing all trusted functions to safe (with one big trusted block around the function body). These are pretty small changes, but would yield a much better structure.
And by the way: I would not call the blocks system, they should be distinguishable from system functions to make them better searchable! I still think the trusted blocks should be rare and small and only necessary at the lowest level deep within libraries.
Jan 17
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/17/20 4:39 AM, Dominikus Dittes Scherkl wrote:
 On Friday, 17 January 2020 at 09:21:30 UTC, Dominikus Dittes Scherkl wrote:
 On Friday, 17 January 2020 at 08:10:48 UTC, Johannes Pfau wrote:

 I'm curious, what do you think would be the ideal scheme if we could 
 redesign it from scratch? Only  safe/ system as function attributes 
 and  trusted (or  system) blocks which can be used in  safe functions?
Yes, pretty much the same as in Rust. And we can get there: just need to introduce trusted blocks (which would not a change what is already possible but only a simplification of syntax) and changing all trusted functions to safe (with one big trusted block around the function body). These are pretty small changes, but would yield a much better structure.
And by the way: I would not call the blocks system, they should be distinguishable from system functions to make them better searchable! I still think the trusted blocks should be rare and small and only necessary at the lowest level deep within libraries.
system blocks would only live inside trusted functions. So you can search for trusted, and then look for the system blocks to see which parts need attention. -Steve
Jan 17
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 17 January 2020 at 09:21:30 UTC, Dominikus Dittes 
Scherkl wrote:
 [snip]

 Yes, pretty much the same as in Rust.
Somewhat OT, but HN is blowing up with an article [1] about the Rust community complaining unsafe being overused, resulting in the maintainer of the HTTP framework quitting. [1] https://words.steveklabnik.com/a-sad-day-for-rust
Jan 17
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/17/20 11:33 AM, jmh530 wrote:
 On Friday, 17 January 2020 at 09:21:30 UTC, Dominikus Dittes Scherkl wrote:
 [snip]

 Yes, pretty much the same as in Rust.
Somewhat OT, but HN is blowing up with an article [1] about the Rust community complaining unsafe being overused, resulting in the maintainer of the HTTP framework quitting. [1] https://words.steveklabnik.com/a-sad-day-for-rust
Well, those people for sure won't be running to D any time soon. We have had our share of quitters as well. It happens, people get very emotional sometimes. It's good to remember that the Internet is not a nice place, it's good to have somewhat of a thick skin for that kind of stuff. -Steve
Jan 17
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 17 January 2020 at 16:57:04 UTC, Steven Schveighoffer 
wrote:
 [snip]

 Well, those people for sure won't be running to D any time soon.

 We have had our share of quitters as well. It happens, people 
 get very emotional sometimes. It's good to remember that the 
 Internet is not a nice place, it's good to have somewhat of a 
 thick skin for that kind of stuff.

 -Steve
I don't disagree with your sentiments. My interest in the piece was also that Rust doesn't have the trusted/system dichotomy. The problem the community had with the code was that there were unsafe blocks all over the place, which made it difficult to verify if it was safe or not. I think that ties in with some of the discussion here.
Jan 17
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 17.01.20 18:16, jmh530 wrote:
 My interest in the piece was also that Rust doesn't have the 
 trusted/system dichotomy.
They really do, and the fact that many people do not understand this is a pretty solid argument against Rust's `unsafe` keyword. In Rust, the `unsafe` keyword is overloaded to have dual meanings. Functions with `unsafe` blocks are trusted functions. Functions annotated `unsafe` are system functions.
Jan 17
parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 17 January 2020 at 18:12:21 UTC, Timon Gehr wrote:
 [snip]

 They really do, and the fact that many people do not understand 
 this is a pretty solid argument against Rust's `unsafe` 
 keyword. In Rust, the `unsafe` keyword is overloaded to have 
 dual meanings. Functions with `unsafe` blocks are trusted 
 functions. Functions annotated `unsafe` are system functions.
Ah, that is correct. I had not even realized it. Some of the discussion on this thread and some others recently who are skeptical of trusted would thus find this useful. Rust has the equivalent of system and system blocks, but no equivalent keyword to express trusted. At the same time, they are also having problems with people marking too much code as their equivalent of system, contributing to problems in their community. This would suggest removing trusted should have a very high bar.
Jan 17
prev sibling parent reply IGotD- <nise nise.com> writes:
On Friday, 17 January 2020 at 16:33:14 UTC, jmh530 wrote:
 On Friday, 17 January 2020 at 09:21:30 UTC, Dominikus Dittes 
 Scherkl wrote:
 [snip]

 Yes, pretty much the same as in Rust.
Somewhat OT, but HN is blowing up with an article [1] about the Rust community complaining unsafe being overused, resulting in the maintainer of the HTTP framework quitting. [1] https://words.steveklabnik.com/a-sad-day-for-rust
I think this is pretty funny and also predictable. There is an overuse of unsafe in Rust because the programmers want/must escape their Gulag in order to do the things they want, it's human behaviour. I have for long predicted that Rust is not the way forward for programming languages even if the language have some good things and now they are trying to escape from it. BTW, here is another proof of that, the blog "Learning Rust the dangerous way", http://cliffle.com/p/dangerust/. Which was appreciated by many users and beginners. This is one of the reasons I'm a bit skeptical against DIP 1028, https://forum.dlang.org/thread/ejaxvwklkyfnksjkldux forum.dlang.org. That people will not value it that much. I have nothing against a safe subset, but I'm not sure making it default is the right way to go. A bit OT, but there will be another round of that DIP at least.
Jan 17
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jan 17, 2020 at 05:28:09PM +0000, IGotD- via Digitalmars-d wrote:
 On Friday, 17 January 2020 at 16:33:14 UTC, jmh530 wrote:
[...]
 [1] https://words.steveklabnik.com/a-sad-day-for-rust
I think this is pretty funny and also predictable. There is an overuse of unsafe in Rust because the programmers want/must escape their Gulag in order to do the things they want, it's human behaviour.
The other human behaviour is that people form habits and then resist changing said habits. See, the thing is that there's a lot to be said about defaults that incentivize people to do things the Right Way(tm). You provide the option to do things differently, there's an escape hatch for when you need it, but you also nudge them in the right direction, so that if they're undecided or not paying attention, they automatically default to doing it the right way. One thing that D did quite well IMO is that the default way to do things often coincides with the best way. As opposed to say C++, where the most obvious way to write a piece of code is almost certainly the wrong way, due to any number of potential problems (built-in arrays are unsafe, avoid raw pointers, avoid new, avoid writing loops, avoid mutation, the list goes on). But you have to start out with the right defaults, because once people form habits around those defaults, they will resist change. Inertia is a powerful force. One of the areas D didn't incentivize in the right way is being system by default. DIP 1028 is trying to change that, but you see the consequences of not starting out that way in the first place: people are resisting it because they have become accustomed to system by default, and dislike changing their habits. [...]
 This is one of the reasons I'm a bit skeptical against DIP 1028,
 https://forum.dlang.org/thread/ejaxvwklkyfnksjkldux forum.dlang.org.
 That people will not value it that much. I have nothing against a safe
 subset, but I'm not sure making it default is the right way to go.
[...] IMO it would have worked had it been the default from the very beginning. This is why language decisions are so hard to make, because you don't really know what's the best design except in retrospect, but wrong decisions are hard to change after the fact because of inertia. By the time you accumulate enough experience to know what would have worked better, you may already be stuck with the previous decision. T -- Heuristics are bug-ridden by definition. If they didn't have bugs, they'd be algorithms.
Jan 17
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 17 January 2020 at 19:21:14 UTC, H. S. Teoh wrote:
 See, the thing is that there's a lot to be said about defaults 
 that incentivize people to do things the Right Way(tm).
This is overblown. Rather than go by secondary sources go straight to the author: «Actix always will be “shit full of UB” and “benchmark cheater”. (Btw, with tfb benchmark I just wanted to push rust to the limits, I wanted it to be on the top, I didn’t want to push other rust frameworks down.) » https://github.com/actix/actix-web He was clearly exploring a terrain where it makes sense to go low level, that is, he tried to get ahead in benchmarks. You get what you pay for...
Jan 17
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 17.01.20 09:10, Johannes Pfau wrote:
 Am Thu, 16 Jan 2020 04:34:26 +0100 schrieb Timon Gehr:
 ...
 I'm curious, what do you think would be the ideal scheme if we could
 redesign it from scratch?
Different approaches have different trade-offs and I am not sure how to weight the benefits and drawbacks. However, whatever the final solution is, the type system should not make a difference between safe and trusted function signatures.
 Only  safe/ system as function attributes and
  trusted (or  system) blocks which can be used in  safe functions?
I do not particularly like this option. trusted blocks are better than system blocks, but an implicit assumption many people are still going to make is that only the code inside the block is trusted, and this will typically be incorrect.
Jan 17
prev sibling parent David Nadlinger <code klickverbot.at> writes:
On Friday, 17 January 2020 at 08:10:48 UTC, Johannes Pfau wrote:
 I'm curious, what do you think would be the ideal scheme if we 
 could redesign it from scratch? Only  safe/ system as function 
 attributes and  trusted (or  system) blocks which can be used 
 in  safe functions?
For the record, this is also exactly what I argued for in the thread linked in the original post, from more than 7 years ago. No redesigning from scratch is needed for this. Just add trusted blocks and discourage (and perhaps slowly deprecate, over years) the use of trusted as a function attribute. — David
Jan 18
prev sibling parent reply David Nadlinger <code klickverbot.at> writes:
On Thursday, 16 January 2020 at 00:21:21 UTC, Joseph Rushton 
Wakeling wrote:
 The fact that in a similar situation D forces you to annotate 
 the function with ` trusted`, and alert users to the 
 _possibility_ that memory safety bugs could exist within the 
 code of this function, is useful information even if you can't 
 access the source code.
Detail the scenario where this would be useful, please. If you want to audit a program to make sure there are no uses of potentially memory-unsafe code, you need access to all the source code: Even safe functions can contain arbitrary amounts of potentially unsafe code, as they can call into trusted functions. You make this point yourself in the quoted post; any information conveyed by trusted is necessarily incomplete, by virtue of it being intransitive. In other words, this "alert", as you put it, has next to no information content on account of its arbitrarily high false-negative rate ( safe functions calling into unsafe code), and is thus worse than useless. If you don't have access to the source code, you don't know anything about what is used to implement a safe function. If you do, you don't care where exactly the keyword you are grepping for in an audit is, as long as it is proximate to the potentially-unsafe code. trusted has no place on the API level. — David
Jan 18
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 18 January 2020 at 21:44:17 UTC, David Nadlinger 
wrote:
 On Thursday, 16 January 2020 at 00:21:21 UTC, Joseph Rushton 
 Wakeling wrote:
 The fact that in a similar situation D forces you to annotate 
 the function with ` trusted`, and alert users to the 
 _possibility_ that memory safety bugs could exist within the 
 code of this function, is useful information even if you can't 
 access the source code.
Detail the scenario where this would be useful, please.
I'm honestly not sure what is useful to add to the existing discussion. But I think the problem is that most folks are looking for certainties, whereas I'm prepared to entertain a certain amount of probability in certain contexts. (Please read on to understand what I mean by that, rather than assuming that I'm prepared to allow memory safety to be a roll of the dice:-) Put it like this: for any given safe function you see, odds are that in practice it uses no trusted code outside the standard library/runtime. Is that a guarantee? No. But it's a reasonable heuristic to use on a day-to-day "How concerned do I have to be that this function might do something scary?" basis. (Depending on what the function does, I might be able to make an educated guess of the likelihood there's something trusted closer to home.) OTOH if I see a function marked as trusted I have a cast-iron guarantee that the compiler did not do anything to verify the memory safety of _this particular function_. Which gives me a nudge that on the balance of probability, I might want to give a bit more up-front scrutiny to exactly what it's doing -- either by reading the source code if I can, or by playing with it a bit to see if I can trip it up with some unexpected input. Is that an _audit_? No. But it doesn't seem vastly different to the kind of day to day trust-versus-verify judgement calls that we all have to make, on a day to day basis, about all sorts of matters of code correctness in the APIs that we use.
 If you want to audit a program to make sure there are no uses 
 of potentially memory-unsafe code, you need access to all the 
 source code
Yes, agreed. But that's the difference between doing a full safety audit versus the typical day-to-day "Does it seem reasonable to use this function for my use-case?" judgement calls that we all make when writing code.
Jan 23
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/15/20 1:06 PM, ag0aep6g wrote:
 On 15.01.20 17:54, Joseph Rushton Wakeling wrote:
 On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
[...]
  safe fun() {
     //safe code here
      trusted {
         //those few lines that require manual checking
     }
     //we are safe again
 }
[...]
 So here's the problem with this approach (which was mentioned by 
 several people in the discussion): the actual safety of a function 
 like this is usually down to the combination of the lines that (in 
 your example) are both inside and outside the  trusted block.
Yup. But a proposal could specify that that's the intended meaning for an safe function that contains trusted blocks. Whereas it's more of a cheat when we use trusted nested functions like that. [...]
 So, a better approach would be for the function to be marked up like 
 this:

  trusted fun ()    // alerts the outside user
 {
      // lines that on their own are provably safe go here
       system {
          // these lines are allowed to use  system code
      }
      // only provably safe lines here again
 }

 ... and the compiler's behaviour would be to explicitly verify 
 standard  safe rules for all the lines inside the  trusted function 
 _except_ the ones inside a  system { ... } block.

 Cf. Steven Schveighoffer's remarks here: 
 https://forum.dlang.org/post/qv7t8b$2h2t$1 digitalmars.com

 This way the function signature gives a clear indicator to the user 
 which functions are provably  safe, and which are safe only on the 
 assumption that the developer has done their job properly.
I don't think that's what Steven had in mind. In that world, safe would be very, very limited, because it couldn't be allowed to call trusted functions. That means safe would only apply to trivial functions, and trusted would assume the role that safe has today. But you'd have to wrap every call from an trusted to another trusted function in an system block. It wouldn't be practical.
 The real purpose of  trusted in that example is to allow the  system 
 block in the body, and to signal to reviewers and maintainers that the 
 whole function is unsafe despite the mechanical checks that are done on 
 most of the lines. To a user,  trusted functions would still be the same 
 as  safe ones.
I'll interject here. I was thinking actually exactly along the lines of Joseph's code, and exactly what you are saying as well. And I think Joe's ideas are along the same lines, just they were misinterpreted here. There are two things to look at for safety. One is that a function is safe or not safe (that is, it has a safe implementation, even if there are calls to system functions, so therefore is callable from mechanically checked safe code). This is the part where the compiler uses function attributes to determine what is callable and what is not. The second is how much manual review is needed for the code. This is a signal to the reviewer/reader. In the current regime, the two reasons for marking are muddled -- we don't have a good way to say "this needs manual checking, but I also want the benefits of mechanical checking". This is why I proposed a change to trusted code STILL being mechanically checked, unless you want an escape. This would allow you to mark all code that needs manual review trusted, even if it's mechanically checked (it still needs review if the system-calling parts can muck with the data). There may be a case as well to make data only accessible from system escapes, because the semantics of the data affect the memory safety of an aggregate.
 
 Unfortunately, adding the mechanical checks of  safe to  trusted would 
 mean breaking all  trusted code that exists. So implementing that scheme 
 seems unrealistic.
Yeah, most likely. We would possibly need a fourth attribute for this purpose, or we can continue to rely on safe/ trusted meaning what they mean today (it's doable, though confusing).
 
 But as Steven says, it can be done when we use  trusted blocks instead 
 of  system blocks and  safe instead of  trusted on the function. I.e.:
 
       safe fun ()
      {
          // lines that the compiler accepts as  safe go here
           trusted {
              // these lines are allowed to use  system code
          }
          // only  safe lines here again
      }
 
 It weakens the meaning of  safe somewhat, but it's often treated that 
 way already. There's clearly a need.
safe code is tremendously hampered without trusted. You could only do things like simple math. As soon as you start needing things like memory allocation, or i/o, you need escapes. That is the reality we have. -Steve
Jan 16
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 16 January 2020 at 15:30:45 UTC, Steven 
Schveighoffer wrote:
 The second is how much manual review is needed for the code. 
 This is a signal to the reviewer/reader. In the current regime, 
 the two reasons for marking are muddled -- we don't have a good 
 way to say "this needs manual checking, but I also want the 
 benefits of mechanical checking". This is why I proposed a 
 change to trusted code STILL being mechanically checked, unless 
 you want an escape. This would allow you to mark all code that 
 needs manual review trusted, even if it's mechanically checked 
 (it still needs review if the system-calling parts can muck 
 with the data).
As was pointed out trusted does not achieve much more than a comment, so why not just have a statement- / operator-level escape using a distinguishable and greppable marker like . Then you can just prepend that to all function calls or operations that are unsafe: safe_function(…){ ptr = … //TRUSTED: this is safe because x, y, z free(ptr); } Then leave on any existing mechanical checks and keep adding until it passes.
Jan 16
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/16/20 10:46 AM, Ola Fosheim Grøstad wrote:
 On Thursday, 16 January 2020 at 15:30:45 UTC, Steven Schveighoffer wrote:
 The second is how much manual review is needed for the code. This is a 
 signal to the reviewer/reader. In the current regime, the two reasons 
 for marking are muddled -- we don't have a good way to say "this needs 
 manual checking, but I also want the benefits of mechanical checking". 
 This is why I proposed a change to trusted code STILL being 
 mechanically checked, unless you want an escape. This would allow you 
 to mark all code that needs manual review trusted, even if it's 
 mechanically checked (it still needs review if the system-calling 
 parts can muck with the data).
As was pointed out trusted does not achieve much more than a comment, so why not just have a statement- / operator-level escape using a distinguishable and greppable marker like . Then you can just prepend that to all function calls or operations that are unsafe: safe_function(…){      ptr = …      //TRUSTED: this is safe because x, y, z      free(ptr); } Then leave on any existing mechanical checks and keep adding until it passes.
Enforcing the name of the comment helps the greppability. And putting trusted on the entire function helps avoid grep finding blocks whereby you now have to dig back up to find the context. safe functions with no trusted escapes have a much different level of review required. If you can assume all the inputs are valid, then the function shouldn't need review -- even if it calls trusted functions. I was looking for a distinction from safe functions like this, and safe ones with escapes. I do like the syntax, though system free(ptr); isn't much worse. -Steve
Jan 16
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 16 January 2020 at 15:46:01 UTC, Ola Fosheim Grøstad 
wrote:
 Then leave on any existing mechanical checks and keep adding    
 until it passes.
Note: for this to work in the general case (with more advanced verification) you most likely need to add an assume construct to tell the compiler what invariants have been upheld by the escaped/unknown code that was called.
Jan 16
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On Thursday, 16 January 2020 at 15:30:45 UTC, Steven 
Schveighoffer wrote:
[...]
 But as Steven says, it can be done when we use  trusted blocks 
 instead of  system blocks and  safe instead of  trusted on the 
 function. I.e.:
 
       safe fun ()
      {
          // lines that the compiler accepts as  safe go here
           trusted {
              // these lines are allowed to use  system code
          }
          // only  safe lines here again
      }
 
 It weakens the meaning of  safe somewhat, but it's often 
 treated that way already. There's clearly a need.
safe code is tremendously hampered without trusted. You could only do things like simple math. As soon as you start needing things like memory allocation, or i/o, you need escapes. That is the reality we have.
Three ways to design safe/ trusted: 1) safe cannot call trusted. I agree that this would be virtually useless. 2) safe can call trusted. trusted functions must be self-contained safe. This is how trusted is currently meant to work. 3) safe code can contain trusted parts (blocks/lambdas). Those trusted parts may rely on the surrounding safe code for safety. The safe parts are effectively in a third state: Mechanical checks are performed, but manual checks are still needed to verify the interaction with the trusted parts. table, as far as I'm concerned. doing that is strictly speaking valid today. But if we do get trusted blocks, that's an opportunity to make it valid. I.e., "[weakening] the meaning of safe". In contrast, with system blocks (in trusted functions) we could To reiterate, I'm ok with both system blocks and trusted getting any kind of block syntax. But that would be harder to formalize, and there's no point in doing it if we want to transition to block syntax anyway. that one way or the other. And if block syntax gets introduced, then that's the time to strike.
Jan 16
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 16, 2020 at 10:30:45AM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]
 There are two things to look at for safety. One is that a function is
 safe or not safe (that is, it has a safe implementation, even if there
 are calls to system functions, so therefore is callable from
 mechanically checked safe code). This is the part where the compiler
 uses function attributes to determine what is callable and what is
 not.
 
 The second is how much manual review is needed for the code. This is a
 signal to the reviewer/reader. In the current regime, the two reasons
 for marking are muddled -- we don't have a good way to say "this needs
 manual checking, but I also want the benefits of mechanical checking".
 This is why I proposed a change to trusted code STILL being
 mechanically checked, unless you want an escape. This would allow you
 to mark all code that needs manual review trusted, even if it's
 mechanically checked (it still needs review if the system-calling
 parts can muck with the data).
[...] This is why I proposed that trusted functions should *still* be subject to safe checks, only with the exception that they're allowed to have embedded system blocks where such checks are relaxed (and these system blocks are only allowed inside trusted functions). So the trusted is a visual marker that it needs to be manually verified, but you still have the benefit of the compiler automatically verifying most of its body except for the (hopefully small) system block where such checks are temporarily suspended. T -- It is impossible to make anything foolproof because fools are so ingenious. -- Sammy
Jan 16
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 16 January 2020 at 18:05:25 UTC, H. S. Teoh wrote:
 This is why I proposed that  trusted functions should *still* 
 be subject to  safe checks, only with the exception that 
 they're allowed to have embedded  system blocks where such 
 checks are relaxed (and these  system blocks are only allowed 
 inside  trusted functions).  So the  trusted is a visual marker 
 that it needs to be manually verified, but you still have the 
 benefit of the compiler automatically verifying most of its 
 body except for the (hopefully small)  system block where such 
 checks are temporarily suspended.
Hang on, have 3 of us all made the same proposal? (OK, I just reiterated what I understood to be Steven's proposal, but ...:-) I'll leave it to others to decide if we're great minds or fools or anything in between ;-)
Jan 16
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 16, 2020 at 06:10:16PM +0000, Joseph Rushton Wakeling via
Digitalmars-d wrote:
 On Thursday, 16 January 2020 at 18:05:25 UTC, H. S. Teoh wrote:
 This is why I proposed that  trusted functions should *still* be
 subject to  safe checks, only with the exception that they're
 allowed to have embedded  system blocks where such checks are
 relaxed (and these  system blocks are only allowed inside  trusted
 functions).  So the  trusted is a visual marker that it needs to be
 manually verified, but you still have the benefit of the compiler
 automatically verifying most of its body except for the (hopefully
 small)  system block where such checks are temporarily suspended.
Hang on, have 3 of us all made the same proposal? (OK, I just reiterated what I understood to be Steven's proposal, but ...:-) I'll leave it to others to decide if we're great minds or fools or anything in between ;-)
Fools or not, the important thing is whether we can convince Walter to agree with this... This is far from the first time such an idea came up. I remember back when Mihail Strashuns was actively contributing to Phobos, we had this discussion on Github where we agreed that we'd like to reduce the scope of trusted as much as possible, meaning, the unsafe parts of trusted should be as small as possible in order to minimize the surface area of potential problems. This was before people came up with the idea of a nested trusted lambda. We both felt very uncomfortable that there were some functions Phobos that were marked trusted, but were so large that it was impractical to review the entire function body for correctness. Furthermore, since Phobos at the time was undergoing a rapid rate of change, we were uncomfortable with the idea that any random PR might touch some seemingly-innocuous part of a trusted function and break its safety, yet there would be no warning whatsoever from the autotester because the compiler simply turned off all checks inside a trusted function. IIRC it was that discussion, which later led to other, further discussions, that eventually resulted in the idea of using nested trusted lambdas inside functions. Of course, in the interim, we also learned from Walter what his stance was: trusted functions should sport a safe API, i.e., even though by necessity it has to do uncheckable things inside, its outward-facing API should be such that it's impossible to break its safety without also breaking your own safety. I.e., taking `int[]` is OK because, presumably, safe code will not allow you to construct an `int[]` that has an illegal pointer or wrong length; but taking `int*, size_t` is not OK, because the caller can just pass the wrong length and you're screwed. Eventually, this restriction was relaxed for nested trusted lambdas, due to the API restriction being too onerous and impractical in some cases. Regardless of what the story was, the basic idea is the same: to shrink the scope of unchecked code as much as possible, and to leverage the compiler's safe-ty checks as much as possible. Ideally, most of a trusted function's body should actually be safe, and only a small part system -- where the compiler is unable to mechanically verify its correctness. That way, if you make a mistake while editing a trusted function, most of the time the compiler will catch it. Only inside the system block (or whatever we decide to call the unchecked block), the checks are suspended and you have to be extra careful when changing it. Basically, we want all the help we can get from the compiler to minimize human error, and we want to reduce the scope of human error to as narrow a scope as possible (while acknowledging that we can never fully eliminate it -- which is why we need trusted in the first place). T -- Once bitten, twice cry...
Jan 16
prev sibling parent Ogi <ogion.art gmail.com> writes:
On Wednesday, 15 January 2020 at 16:54:58 UTC, Joseph Rushton 
Wakeling wrote:
 So here's the problem with this approach (which was mentioned 
 by several people in the discussion): the actual safety of a 
 function like this is usually down to the combination of the 
 lines that (in your example) are both inside and outside the 
  trusted block.
The are two scenarios of memory corruption in a function with a trusted block. Either trusted section is incorrect, or the trusted section is given incorrect input. Safe code that has no deal with trusted code cannot compromise safety. So while the culprit can be outside of trusted block, it will lower the number of suspects. It will give you a good idea of the required safety checks.
Jan 16
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
 Without  trusted the safety design will be much simpler to 
 grasp. Currently the are three vague keywords, complex rules on 
 which functions can call which and best practices on which 
 attribute you should choose.
I also don't understand what's the point with trusted. Should it only be used as a trampoline for safe code into the unknown or "some holds allowed" like something in between system and safe. I find this highly confusing. It's like x86 protection rings (ring 0 - 3) where rings 1 - 2 are seldom used. If you think about it trusted is not necessary as you say that trusted "must be manually verified", aka unsafe. I think you should be allowed to call unsafe code from safe code and it is up to the programmer to check the code that they call and make an assessment on stability. trusted call can call unsafe code further down the line so trusted isn't that useful. I might have misunderstood the point of trusted totally and you can try to make me understand what's the point of it. I didn't find the documentation very helpful.
Jan 15
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 15 January 2020 at 19:27:42 UTC, IGotD- wrote:
 On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
 Without  trusted the safety design will be much simpler to 
 grasp. Currently the are three vague keywords, complex rules 
 on which functions can call which and best practices on which 
 attribute you should choose.
I also don't understand what's the point with trusted. Should it only be used as a trampoline for safe code into the unknown or "some holds allowed" like something in between system and safe. I find this highly confusing. It's like x86 protection rings (ring 0 - 3) where rings 1 - 2 are seldom used. If you think about it trusted is not necessary as you say that trusted "must be manually verified", aka unsafe.
No. trusted is about saying "This function should be safe to use, but that safety has been designed by the developer rather than being automatically verifiable by the compiler." Contrast with system which can be applied to functions that are inherently not guaranteed to be safe -- e.g. where the safety (or not) depends on what input the user provides.
Jan 15
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 15 January 2020 at 19:49:53 UTC, Joseph Rushton 
Wakeling wrote:
 No.   trusted is about saying "This function should be safe to 
 use, but that safety has been designed by the developer rather 
 than being automatically verifiable by the compiler."
Could make sense if you only could call trusted from trusted... If you can call trusted from safe then it makes no sense whatsoever. If you view attributes as something for external APIs then it makes no sense to distinguish between trusted and safe. It is just noise that conveys nothing of substance.
Jan 15
prev sibling parent reply IGotD- <nise nise.com> writes:
On Wednesday, 15 January 2020 at 19:49:53 UTC, Joseph Rushton 
Wakeling wrote:
 No.   trusted is about saying "This function should be safe to 
 use, but that safety has been designed by the developer rather 
 than being automatically verifiable by the compiler."

 Contrast with  system which can be applied to functions that 
 are inherently not guaranteed to be safe -- e.g. where the 
 safety (or not) depends on what input the user provides.
This is why I think it should be removed. In my world there is no "trust the human". Also trusted in kind of backwards. It should be the caller that designate a call or operation trusted, not the the function that you call. Otherwise it is like asking car sales man if I can trust him.
Jan 15
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 15 January 2020 at 21:17:38 UTC, IGotD- wrote:
 This is why I think it should be removed. In my world there is 
 no "trust the human".
Presumably your programs are therefore self-crafted binary, since you couldn't possibly trust the humans who wrote the standard library to write valid code, or the compiler writers to translate it correctly into machine instructions? :-)
 Also  trusted in kind of backwards. It should be the caller 
 that designate a call or operation trusted, not the the 
 function that you call. Otherwise it is like asking car sales 
 man if I can trust him.
I think you're getting caught up on the choice of terminology. It's just a hierarchy of guarantees: safe -- this function should behave in a memory-safe way for all possible inputs you can provide, and this can be validated by the compiler trusted -- this function should behave in a memory-safe way for all possible inputs you can provide, but this has been validated by the developer, and cannot be automatically validated by the compiler system -- some of the possible inputs to this function will cause it to behave in a memory-unsafe way You don't have to like the choice of keywords, but you should recognize that they describe valuable distinctions. There are some nice examples of how these distinctions are useful in the article linked to above.
Jan 15
next sibling parent reply IGotD- <nise nise.com> writes:
On Wednesday, 15 January 2020 at 23:01:57 UTC, Joseph Rushton 
Wakeling wrote:
 Presumably your programs are therefore self-crafted binary, 
 since you couldn't possibly trust the humans who wrote the 
 standard library to write valid code, or the compiler writers 
 to translate it correctly into machine instructions? :-)
safe is a subset of D that guarantees no memory corruption. The only way to ensure this is if the compiler have all the source code (will object code also work?) and can check that all the calls are also safe. If this condition is not met, it is not safe by definition. trusted code has reduced memory guarantees and can also call unsafe code further along the line and therefore unsafe. trusted is therefore impossible and the criteria cannot be met. It's just a badge meaning nothing.
 I think you're getting caught up on the choice of terminology.  
 It's just a hierarchy of guarantees:
No, I'm caught up in the semantics. I see that a condition cannot be met and therefore unnecessary. trusted is an oxymoron.
Jan 15
next sibling parent ag0aep6g <anonymous example.com> writes:
On Wednesday, 15 January 2020 at 23:24:38 UTC, IGotD- wrote:
  safe is a subset of D that guarantees no memory corruption.
Not quite. safe is a subset of D that guarantees no memory corruption, as long as people write only correct trusted code (and there might be more restrictions). This weaker definition is being used because the strong one is considered impractical. There might be text in the documentation that suggests the stronger definition, but that is not going to take precedence. The weaker one is widely accepted as more useful. If anything, a conflict would surely be resolved by changing the documentation.
Jan 15
prev sibling next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 16.01.20 00:24, IGotD- wrote:
 On Wednesday, 15 January 2020 at 23:01:57 UTC, Joseph Rushton Wakeling 
 wrote:
 Presumably your programs are therefore self-crafted binary, since you 
 couldn't possibly trust the humans who wrote the standard library to 
 write valid code, or the compiler writers to translate it correctly 
 into machine instructions? :-)
safe is a subset of D that guarantees no memory corruption.
safe does not mean "this unconditionally can not corrupt memory", it says "if its trusted dependencies are written right, safe code can not introduce memory corruption". If you don't write trusted code yourself and only use the standard library, safe gives you the guarantee you mention per the language and standard library specification, but in practice there can be bugs in the language implementation or its dependencies. Assuming a correct compiler implementation of the safety checks, the locations where memory corruption can be caused will however all be within ` trusted` function calls. Similar concerns apply to other safe languages such as Java. (Why do you think users need to update their JVM regularly? It's not just about performance upgrades.) What do you think "security updates" for your OS do? How is safe supposed to be _useful_ if it cannot assume the OS, system C libraries, and hardware behave as specified?
 The only 
 way to ensure this is if the compiler have all the source code (will 
 object code also work?) and can check that all the calls are also  safe. 
 If this condition is not met, it is not safe by definition.  trusted 
 code has reduced memory guarantees and can also call unsafe code further 
 along the line and therefore unsafe.
 
  trusted is therefore impossible and the criteria cannot be met. It's 
 just a badge meaning nothing.
 ...
Nonsense. It's the place where you inject assumptions into your conditional verifier. It reduces the amount of code you have to audit to convince yourself that a given dependency will not corrupt the internal state of your program.
 I think you're getting caught up on the choice of terminology. It's 
 just a hierarchy of guarantees:
No, I'm caught up in the semantics. I see that a condition cannot be met and therefore unnecessary.
You are operating on bad assumptions or bad logic. trusted is clearly useful, because it makes no sense to require every piece of trusted code to be part of the compiler implementation.
  trusted is an oxymoron.
 
trusted void foo(int[] a,int b){ if(0<=b&&b<a.length){ a.ptr[b]=2; } } system void bar(int[] a,int b){ a.ptr[b]=2; } How do you not see the difference, that it is useful to annotate that difference, and that it is okay to call foo from safe code, but not bar?
Jan 15
prev sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Wednesday, 15 January 2020 at 23:24:38 UTC, IGotD- wrote:
 On Wednesday, 15 January 2020 at 23:01:57 UTC, Joseph Rushton 
 Wakeling wrote:
 Presumably your programs are therefore self-crafted binary, 
 since you couldn't possibly trust the humans who wrote the 
 standard library to write valid code, or the compiler writers 
 to translate it correctly into machine instructions? :-)
safe is a subset of D that guarantees no memory corruption. The only way to ensure this is if the compiler have all the source code (will object code also work?) and can check that all the calls are also safe. If this condition is not met, it is not safe by definition. trusted code has reduced memory guarantees and can also call unsafe code further along the line and therefore unsafe.
No, that's where you're wrong. trusted gives the same guarantees than safe. The only difference is that safe can automatically be checked and trusted cannot. ANY memory violation in a trusted code is a BUG and the responsibility of the programmer. That's why trusted is important and should be only applied to the parts that cannot be checked by the compiler. Why limit then trusted block to functions and not to scopes? Imo the call interface has a semantic that does not allow (or at least not encourage) to interact directly with data of the callers scope. All interactions have to be done via parameters which scopes and lifetimes are known. This is not the case with simple scopes. So the difference between the two is the ABI which adds some guarantees that a simple scope cannot (see Steven Schveighofer's example).
Jan 16
parent reply IGotD- <nise nise.com> writes:
On Thursday, 16 January 2020 at 10:44:56 UTC, Patrick Schluter 
wrote:
 No, that's where you're wrong.  trusted gives the same 
 guarantees than  safe. The only difference is that  safe can 
 automatically be checked and  trusted cannot. ANY memory 
 violation in a trusted code is a BUG and the responsibility of 
 the programmer.
Then we can remove safe all together and trust the programmer to only use the safe subset of D.
 That's why  trusted is important and should be only applied to 
 the parts that cannot be checked by the compiler.
Then trusted is as good as putting a comment in the code.
 All interactions have to be done via parameters which scopes 
 and lifetimes are known. This is not the case with simple 
 scopes. So the difference between the two is the ABI which adds 
 some guarantees that a simple scope cannot (see Steven 
 Schveighofer's example).
Yes, so if safe code can only call function with the safe attribute but can have system blocks in it. This is the same thing as trusted. This is very similar to another language and I think it at least got that part right.
Jan 16
parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 16 January 2020 at 10:58:33 UTC, IGotD- wrote:
 On Thursday, 16 January 2020 at 10:44:56 UTC, Patrick Schluter 
 wrote:
 No, that's where you're wrong.  trusted gives the same 
 guarantees than  safe. The only difference is that  safe can 
 automatically be checked and  trusted cannot. ANY memory 
 violation in a trusted code is a BUG and the responsibility of 
 the programmer.
Then we can remove safe all together and trust the programmer to only use the safe subset of D.
The difference between "a small subset of the program must be manually verified in order to guarantee memory safety" and "the entire program must be manually verified in order to guarantee memory safety" is not insignificant.
Jan 16
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 16 January 2020 at 14:46:15 UTC, Paul Backus wrote:
 On Thursday, 16 January 2020 at 10:58:33 UTC, IGotD- wrote:
 Then we can remove  safe all together and trust the programmer 
 to only use the safe subset of D.
The difference between "a small subset of the program must be manually verified in order to guarantee memory safety" and "the entire program must be manually verified in order to guarantee memory safety" is not insignificant.
If memory-safety is the default then you only need to mark functions and code blocks as unsafe. No need for trusted or safe.
Jan 16
parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 16 January 2020 at 15:02:10 UTC, Ola Fosheim Grøstad 
wrote:
 On Thursday, 16 January 2020 at 14:46:15 UTC, Paul Backus wrote:
 On Thursday, 16 January 2020 at 10:58:33 UTC, IGotD- wrote:
 Then we can remove  safe all together and trust the 
 programmer to only use the safe subset of D.
The difference between "a small subset of the program must be manually verified in order to guarantee memory safety" and "the entire program must be manually verified in order to guarantee memory safety" is not insignificant.
If memory-safety is the default then you only need to mark functions and code blocks as unsafe. No need for trusted or safe.
This is a non-sequitur. Did you mean to respond to a different message?
Jan 16
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 16 January 2020 at 15:12:07 UTC, Paul Backus wrote:
 On Thursday, 16 January 2020 at 15:02:10 UTC, Ola Fosheim 
 Grøstad wrote:
 If memory-safety is the default then you only need to mark 
 functions and code blocks as unsafe. No need for  trusted or 
  safe.
This is a non-sequitur. Did you mean to respond to a different message?
It wasn't really clear what "IGotD-" meant. Although I suspect he was ironic, but if taken literally it would be fair to say that what D has is pretty much the same as this: assume all code is written as memory safe code and add an escape that allows for writing unsafe constructs adorned with a comment that says that code is trusted... except you also need to mark functions as unsafe. Not really a big shift from Rust, except Rust provides dedicated typing-constructs for doing unsafe operations like dealing with uninitialized variables.
Jan 16
parent IGotD- <nise nise.com> writes:
On Thursday, 16 January 2020 at 15:29:58 UTC, Ola Fosheim Grøstad 
wrote:
 On Thursday, 16 January 2020 at 15:12:07 UTC, Paul Backus wrote:

 It wasn't really clear what "IGotD-" meant. Although I suspect 
 he was ironic, but if taken literally it would be fair to say 
 that what D has is pretty much the same as this: assume all 
 code is written as memory safe code and add an escape that 
 allows for writing unsafe constructs adorned with a comment 
 that says that code is trusted... except you also need to mark 
 functions as unsafe. Not really a big shift from Rust, except 
 Rust provides dedicated typing-constructs for doing unsafe 
 operations like dealing with uninitialized variables.
Yes, kind of. It similar what ag0aep6g described in alternative 3. On Thursday, 16 January 2020 at 16:56:17 UTC, ag0aep6g wrote:
 3)  safe code can contain  trusted parts (blocks/lambdas). 
 Those  trusted parts may rely on the surrounding  safe code for 
 safety. The  safe parts are effectively in a third state: 
 Mechanical checks are performed, but manual checks are still 
 needed to verify the interaction with the  trusted parts.
but the difference is that we don't need trusted. We can let system rely on the safety of the surrounding safe block. I can't see how this really different from having that extra trusted attribute. I might have forgotten to tell that we need this extra system {} block.
Jan 16
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 15, 2020 at 11:01:57PM +0000, Joseph Rushton Wakeling via
Digitalmars-d wrote:
 On Wednesday, 15 January 2020 at 21:17:38 UTC, IGotD- wrote:
 This is why I think it should be removed. In my world there is no
 "trust the human".
Presumably your programs are therefore self-crafted binary, since you couldn't possibly trust the humans who wrote the standard library to write valid code, or the compiler writers to translate it correctly into machine instructions? :-)
[...] And you also built your own hardware, because you cannot trust the hardware manufacturers that they didn't have a bug in silicon that causes memory corruption under rare circumstances, in spite of your correctly-coded instructions. Or they didn't put a trapdoor in silicon that allows unknown 3rd parties from some remote server to gain ring 0 access to your CPU. T -- My program has no bugs! Only undocumented features...
Jan 15
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 15 January 2020 at 23:26:01 UTC, H. S. Teoh wrote:
 And you also built your own hardware, because you cannot trust 
 the hardware manufacturers that they didn't have a bug in 
 silicon that causes memory corruption under rare circumstances, 
 in spite of your correctly-coded instructions. Or they didn't 
 put a trapdoor in silicon that allows unknown 3rd parties from 
 some remote server to gain ring 0 access to your CPU.
To be honest, I just use butterflies :-)
Jan 15
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 15 January 2020 at 19:27:42 UTC, IGotD- wrote:
 I might have misunderstood the point of  trusted totally and 
 you can try to make me understand what's the point of it. I 
 didn't find the documentation very helpful.
"How to Write trusted Code in D", on the D blog, is a good introduction: https://dlang.org/blog/2016/09/28/how-to-write-trusted-code-in-d/
Jan 15
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
 If you think of it, it makes no sense for  trusted to be a 
 function attribute. It doesn’t describe its behavior but its 
 implementation. If you call a function, it should not be your 
 concern;  safe and  trusted are the same thing for you. But 
 these two attributes result in two different signatures. If a 
 function expects a  safe callback, you can’t pass a  trusted 
 function to it without  safe wrapping. If some library makes a 
 function that used to be  safe  trusted, that’s a breaking 
 change. Etc, etc. Why should we introduce complexity out of 
 nowhere?
To me, this is the only part of the argument against trusted that's really convincing. Having the safe/ trusted distinction be part of the type system and the ABI is almost 100% downside, and changing that would eliminate some pain points at essentially no cost to the language. However, both of these issues can be fixed without a huge overhaul of trusted. The type-system issue can be fixed by introducing implicit conversions between safe and trusted functions, and the ABI issue can be fixed by changing how trusted affects name mangling. Once these issues are fixed, the only remaining benefits of the proposed overhaul are cosmetic, and IMO not worth the disruption they would cause.
Jan 15
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 16, 2020 at 01:07:21AM +0000, Paul Backus via Digitalmars-d wrote:
 On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
[...]
 If a function expects a  safe callback, you can’t pass a  trusted
 function to it without  safe wrapping.
[...] Hogwash. Did you even test this before making statements like that? void fun(void function() safe dg) { } void trustme() trusted { } void main() { fun(&trustme); } Compiles fine. [...]
 However, both of these issues can be fixed without a huge overhaul of
  trusted. The type-system issue can be fixed by introducing implicit
 conversions between  safe and  trusted functions, and the ABI issue
 can be fixed by changing how  trusted affects name mangling.
trusted *already* implicitly converts to safe. We're arguing over nothing here. T -- Two wrongs don't make a right; but three rights do make a left...
Jan 15
parent Ogi <ogion.art gmail.com> writes:
On Thursday, 16 January 2020 at 01:32:23 UTC, H. S. Teoh wrote:
 Hogwash. Did you even test this before making statements like 
 that?

 	void fun(void function()  safe dg) { }

 	void trustme()  trusted { }

 	void main() {
 		fun(&trustme);
 	}

 Compiles fine.
You’re right. I feel like a moron now. If I am not mistaken, this is the only example of covariant function attributes.
Jan 16
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 16.01.20 02:07, Paul Backus wrote:
 On Wednesday, 15 January 2020 at 14:30:02 UTC, Ogi wrote:
 If you think of it, it makes no sense for  trusted to be a function 
 attribute. It doesn’t describe its behavior but its implementation. If 
 you call a function, it should not be your concern;  safe and  trusted 
 are the same thing for you. But these two attributes result in two 
 different signatures. If a function expects a  safe callback, you 
 can’t pass a  trusted function to it without  safe wrapping. If some 
 library makes a function that used to be  safe  trusted, that’s a 
 breaking change. Etc, etc. Why should we introduce complexity out of 
 nowhere?
To me, this is the only part of the argument against trusted that's really convincing. Having the safe/ trusted distinction be part of the type system and the ABI is almost 100% downside, and changing that would eliminate some pain points at essentially no cost to the language. However, both of these issues can be fixed without a huge overhaul of trusted. The type-system issue can be fixed by introducing implicit conversions between safe and trusted functions,
That's already how it works. The OP just didn't bother to check whether the claim is true: void main(){ void delegate() safe dg0=() trusted{}; // ok void delegate() trusted dg1=() safe{}; // ok }
 and the ABI issue can 
 be fixed by changing how  trusted affects name mangling.
 ...
It's not just the ABI, it's also static introspection. The way to fix that is to treat ` trusted` functions as ` safe` functions from the outside. (E.g.., `typeof(() trusted{})` would be `void delegate() safe`.)
 Once these issues are fixed, the only remaining benefits of the proposed 
 overhaul are cosmetic, and IMO not worth the disruption they would cause.
 
Jan 15