www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - how much "real-life" code can be marked safe ?

reply someone <someone somewhere.com> writes:
... just wondering:

I am writing pretty trivial code, nothing out of the ordinary, 
and attempted to check how much of it could be marked safe ...

- Lots of tiny common library functions are pretty easy

- Getter/Setter properties are easy too

- almost all this() constructors are a no-go providing you do 
something in-between with the parameters until you assign them 
back to the target variables; eg: you have a char parameter (that 
you need to do something with it) that needs to be assigned to a 
class/structure string member variable and then it needs a cast 
... but no castings are allowed with  safe

But when you start attempting to declare  safe chunks of code 
that actually DO things ... well, it seems end-of-the-story.

Declaring  safe void() main() {...} as I was advised in some 
previous post (to avoid declaring  safe everywhere) is almost 
impossible unless you are doing the hello world app.

I would love to hear how you, I mean the community, approach code 
safeness ?
Jul 01 2021
next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Friday, 2 July 2021 at 00:26:52 UTC, someone wrote:

 ... just wondering:
 ...


Imho, if you want all of the app to be safe, and you cannot avoid 
unsafe code, then there are two choices:
1. Mark the method doing unsafe stuff as  trusted, or pieces of 
code which are unsafe with trusted lambda hack.
2. Mark the top caller of your  system methods as  trusted.

In both cases, unsafe code should be manually checked as best as 
it can be done. Best is to think twice whether you can redesign 
your code to avoid unsafe operations.

On PR review step if there is such thing for your app, reviewers 
should carefully review those  trusted blocks of code.

Oh well, and tests, tests and more tests, i.e. It should be 
thoroughly tested.

Best regards,
Alexandru.
Jul 02 2021
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/1/21 8:26 PM, someone wrote:

 ... just wondering:
 
 I am writing pretty trivial code, nothing out of the ordinary, and 
 attempted to check how much of it could be marked safe ...


It should be quite a bit.


 
 - Lots of tiny common library functions are pretty easy
 
 - Getter/Setter properties are easy too
 
 - almost all this() constructors are a no-go providing you do something 
 in-between with the parameters until you assign them back to the target 
 variables; eg: you have a char parameter (that you need to do something 
 with it) that needs to be assigned to a class/structure string member 
 variable and then it needs a cast ... but no castings are allowed with 
  safe


This doesn't make a lot of sense, lots of constructors are safe.

And casting is OK as long as it's safe, including class downcasts.


 
 But when you start attempting to declare  safe chunks of code that 
 actually DO things ... well, it seems end-of-the-story.


This is why you encapsulate the unsafe parts in  trusted functions.

The classic example is the posix `read` and `write` functions. The 
function `int read(int fd, void *data, int len)` is not safe, nor can it 
be trusted. But a trusted wrapper is possible: ` trusted int 
readSafe(int fd, ubyte[] data)`. Now all of a sudden, safe functions 
have more utility.


 
 Declaring  safe void() main() {...} as I was advised in some previous 
 post (to avoid declaring  safe everywhere) is almost impossible unless 
 you are doing the hello world app.


The point is to mark main  safe, and then fix whatever things that 
aren't safe that it calls to provide a safe interface.


 
 I would love to hear how you, I mean the community, approach code 
 safeness ?


Any function I want to make safe, I mark it as  safe. Then I fix the 
compiler complaints until it compiles (including functions that it 
calls), using as little  trusted code as possible.

-Steve
Jul 02 2021
prev sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
(Responding out of order:)

On Friday, 2 July 2021 at 00:26:52 UTC, someone wrote:

 But when you start attempting to declare  safe chunks of code 
 that actually DO things ... well, it seems end-of-the-story.


If you find yourself unable to get real work done in ` safe` 
code, this is almost certainly a sign of one of the following 
problems:

0) You don't fully understand the purpose and valid use of any or 
all of the ` trusted`, `inout`, `scope`, and `return` annotations.

1) Your code is avoiding use of the garbage collector, and/or 
does not have `-dip1000` enabled. (` safe` is still quite useful 
without the garbage collector, but even with `-dip1000` you'll 
still need a lot of ` trusted` code.)

2) You have at least one dependency that isn't correctly designed 
for use with ` safe`.

As long as you're willing to use the garbage collector, almost 
all algorithms can be expressed in an efficient ` safe` way, 
**but** this sometimes requires knowledge of several advanced 
features of D, and how and when to combine them.


 but no castings are allowed with  safe


That is simply not true. Many explicit casts are ` safe`, as are 
nearly all implicit casts.

Casting away `const` or `immutable` is ` system`, but should 
hardly ever be necessary if you understand how to write 
constructors correctly (see below), and use `inout` appropriately.

Many reinterpret casts are also illegal, but `union` and `class` 
provide ` safe` ways of achieving the same goals for the common 
cases.


 - almost all this() constructors are a no-go providing you do 
 something in-between with the parameters until you assign them 
 back to the target variables;


Constructors can be ` safe`, but you have to understand how they 
work in D:

The key difficulty is that what appears to be the first 
"assignment" to each field in a constructor actually constructs 
that field, instead. Subsequent assignments really are 
assignments.

So, when constructing a `const` object each field can only be 
"assigned" once, because the fields are also `const`, even in the 
object's constructor. If you need to do complex calculations to 
determine field values, use temporary variables and `static` 
helper functions until you get the final value, and then 
unconditionally assign that value to a field *once*.

**TLDR**; You probably don't understand how to use ` safe` 
correctly. (Most people don't; the rules are complicated and 
non-obvious.)

Post some example code that you think can't be ` safe`, and I can 
probably show you how to fix it, unless it involves manual memory 
management or an incompatible dependency. Even then, the 
non-` safe` code can often be isolated behind an ` trusted` API.
Jul 02 2021
parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Friday, 2 July 2021 at 22:08:31 UTC, tsbockman wrote:

 (Responding out of order:)

 On Friday, 2 July 2021 at 00:26:52 UTC, someone wrote:

 But when you start attempting to declare  safe chunks of code 
 that actually DO things ... well, it seems end-of-the-story.


 If you find yourself unable to get real work done in ` safe` 
 code, this is almost certainly a sign of one of the following 
 problems:

 0) You don't fully understand the purpose and valid use of any 
 or all of the ` trusted`, `inout`, `scope`, and `return` 
 annotations.

 1) Your code is avoiding use of the garbage collector, and/or 
 does not have `-dip1000` enabled. (` safe` is still quite 
 useful without the garbage collector, but even with `-dip1000` 
 you'll still need a lot of ` trusted` code.)

 2) You have at least one dependency that isn't correctly 
 designed for use with ` safe`.


I'd add:
3. An edge case. Ex: You need to mutate some data and then assume 
it is immutable in a constructor.

4. Functionality that doesn't account for  safe/immutable or any 
other features when it can in standard library. Take for example 
array.dup, there is no inout alternative for it, and you're 
pretty much stuck with trusted code, when you'd like to dup an 
array that is inout.
manual.

Although these two should be on the lowest place in this list by 
priority.
Jul 03 2021
parent reply tsbockman <thomas.bockman gmail.com> writes:
On Saturday, 3 July 2021 at 16:06:33 UTC, Alexandru Ermicioi 
wrote:

 3. An edge case. Ex: You need to mutate some data and then 
 assume it is immutable in a constructor.


Can you give a valid example where that is necessary? The main 
examples that I can think of either can be ` safe` with the right 
API, or are motivated by a desire to avoid the GC and/or 
druntime, thus falling under (1).


 4. Functionality that doesn't account for  safe/immutable or 
 any other features when it can in standard library.


True, although it's just another example of my point (2). The 
standard library and druntime are dependencies, too...


 Take for example array.dup, there is no inout alternative for 
 it,
 and you're pretty much stuck with trusted code, when you'd like 
 to dup an array that is inout.


`inout` is usually just a convenient way to use one 
implementation to handle mutable, `const` and `immutable` cases. 
In those rare cases where `inout` itself won't work, it is almost 
always possible to accomplish the same thing using `template 
this` or separate overloads:

```D
import std.traits : Unqual, CopyConstness;

struct A {
     int*[] arr;
     this(this This, Arr)(Arr arr) scope pure  safe nothrow
         if(is(Arr : E[], E) && is(E : CopyConstness!(This, 
Unqual!E)))
     {
         this.arr = arr.dup;
     }
}

void main()
{
     A ma = new int*[5];
     const(A) ca = new const(int*[3]);
     const(A) ia = new immutable(int*[4]);
}
```

Again, it's certainly not obvious how to do this, or why it is 
necessary, but it is *possible*.

The one exception here is when the array is already typed `inout` 
before it is passed to the constructor. But, that's an example of 
(2) since this logic applies transitively throughout the call 
stack: if you need to call `dup` anywhere, don't erase the 
constness with `inout`.
Jul 03 2021
parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 3 July 2021 at 20:09:56 UTC, tsbockman wrote:

 On Saturday, 3 July 2021 at 16:06:33 UTC, Alexandru Ermicioi 
 wrote:

 3. An edge case. Ex: You need to mutate some data and then 
 assume it is immutable in a constructor.


 Can you give a valid example where that is necessary? The main 
 examples that I can think of either can be ` safe` with the 
 right API, or are motivated by a desire to avoid the GC and/or 
 druntime, thus falling under (1).


Can't remember any specific code now, but suppose you have a 
mutable object as input to a function or constructor. You need to 
return or assign an immutable copy of that struct and you can do 
that with right copy constructor on that object, but before that 
you need to do a couple of mutations on that object. In this use 
case you can't avoid cast(immutable) easily. Note: it is desired 
to not mutate the original object.

The summary is this: the construction of immutable instances can 
be done just through initialization statement. You can't mutate 
the instance and then assign/assume it as an immutable.


 4. Functionality that doesn't account for  safe/immutable or 
 any other features when it can in standard library.


 True, although it's just another example of my point (2). The 
 standard library and druntime are dependencies, too...


Right, didn't take it that way.


 The one exception here is when the array is already typed 
 `inout` before it is passed to the constructor. But, that's an 
 example of (2) since this logic applies transitively throughout 
 the call stack: if you need to call `dup` anywhere, don't erase 
 the constness with `inout`.


Yeah this is a working workaround, for this specific use case. I 
still would prefer inout version, even if I need to do some 
unsafe casts due to clear intentions it gives the user, and 
better errors.
Anyway this is just a proof of your 2nd point.
Jul 04 2021
parent Paul Backus <snarwin gmail.com> writes:
On Sunday, 4 July 2021 at 08:43:11 UTC, Alexandru Ermicioi wrote:

 On Saturday, 3 July 2021 at 20:09:56 UTC, tsbockman wrote:

 On Saturday, 3 July 2021 at 16:06:33 UTC, Alexandru Ermicioi 
 wrote:

 3. An edge case. Ex: You need to mutate some data and then 
 assume it is immutable in a constructor.


 Can you give a valid example where that is necessary? The main 
 examples that I can think of either can be ` safe` with the 
 right API, or are motivated by a desire to avoid the GC and/or 
 druntime, thus falling under (1).


 Can't remember any specific code now, but suppose you have a 
 mutable object as input to a function or constructor. You need 
 to return or assign an immutable copy of that struct and you 
 can do that with right copy constructor on that object, but 
 before that you need to do a couple of mutations on that 
 object. In this use case you can't avoid cast(immutable) 
 easily. Note: it is desired to not mutate the original object.


```d
immutable(Foo) example(ref Foo input)
{
     Foo mutableCopy = input;
     mutableCopy.mutate();
     return immutable(Foo)(mutableCopy); // call copy ctor
}
```

I guess if your object is very expensive to copy you might want 
to use `cast(immutable)` here, but IMO the real solution is to 
refactor your code so that the call to `.mutate()` is not 
necessary.

For example, let's say that what `mutate` does is change the 
member variable `bar`. You could rewrite the above as:

```d
immutable(Foo) example(ref Foo input)
{
     auto copy = immutable(Foo)(
         // update this field
         computeNewValue(input.bar),
         // copy the rest
         input.baz,
         input.quux,
         /* ... */
     );
     return copy;
}
```
Jul 04 2021