www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What exactly does " safe" mean?

reply "monarch_dodra" <monarchdodra gmail.com> writes:
The way I understood it,  safe defines a list of things that are 
or aren't legal inside the implementation of a function. It also 
changes the scheme of bounds checking, in release code.

What bothers me though, is that from an interface point of view, 
it doesn't really mean anything (or at least, I haven't really 
understood anything). AFAIK: if I call something " safe", chances 
of a core dump are relatively "lower", but they can still happen:
* A function that accepts a pointer as an argument can be marked 
safe, so all bets are off there, no, since the pointer can be 
dereferenced?
* Member functions for structs that have pointers, too, can be 
marked safe...

Or does it only mean "if you give me valid pointers, I can't core 
dump*"?
(*ignoring current flaws, such as escaping slices from static 
arrays)

The main reason about this question is that now I'm confused 
about  trusted: what are the conditions a developer needs to take 
into account before marking a function " trusted" ?

Ditto for member functions, when they operate on pointer members. 
Can those be  safe?

Yeah, overall, I'm confused as to what " safe" means from an 
interface point of view :(
Jun 01 2013
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sat, 01 Jun 2013 21:59:18 +0200
"monarch_dodra" <monarchdodra gmail.com> wrote:


 The way I understood it,  safe defines a list of things that are 
 or aren't legal inside the implementation of a function. It also 
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of view, 
 it doesn't really mean anything (or at least, I haven't really 
 understood anything). AFAIK: if I call something " safe", chances 
 of a core dump are relatively "lower", but they can still happen:
 * A function that accepts a pointer as an argument can be marked 
 safe, so all bets are off there, no, since the pointer can be 
 dereferenced?
 * Member functions for structs that have pointers, too, can be 
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't core 
 dump*"?
 (*ignoring current flaws, such as escaping slices from static 
 arrays)
 
 The main reason about this question is that now I'm confused 
 about  trusted: what are the conditions a developer needs to take 
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer members. 
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an 
 interface point of view :(


Core dumps aren't the big problem  safe tries to avoid. The big problem
is memory corruption, ie trampling memory you didn't expect to (or
shouldn't be allowed to).
Jun 01 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 20:22:01 UTC, Nick Sabalausky wrote:

 On Sat, 01 Jun 2013 21:59:18 +0200
 "monarch_dodra" <monarchdodra gmail.com> wrote:


 The way I understood it,  safe defines a list of things that 
 are or aren't legal inside the implementation of a function. 
 It also changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of 
 view, it doesn't really mean anything (or at least, I haven't 
 really understood anything). AFAIK: if I call something 
 " safe", chances of a core dump are relatively "lower", but 
 they can still happen:
 * A function that accepts a pointer as an argument can be 
 marked safe, so all bets are off there, no, since the pointer 
 can be dereferenced?
 * Member functions for structs that have pointers, too, can be 
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't 
 core dump*"?
 (*ignoring current flaws, such as escaping slices from static 
 arrays)
 
 The main reason about this question is that now I'm confused 
 about  trusted: what are the conditions a developer needs to 
 take into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer 
 members. Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an 
 interface point of view :(


 Core dumps aren't the big problem  safe tries to avoid. The big 
 problem
 is memory corruption, ie trampling memory you didn't expect to 
 (or
 shouldn't be allowed to).


So, let's say I have:

--------
void foo(int* p)  safe
{
     *p = 0;
}
--------

I suppose that this give foo the liberty of saying "p points to 
someplace valid" ... "and if not, it's not my fault"?

I suppose something that is trusted then means "I will not 
trample your memory under any circumstance, even if I'm doing 
unsafe things under the hood (unless you give a pointer that is 
already bad)"?
Jun 01 2013
parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 1 June 2013 at 20:31:45 UTC, monarch_dodra wrote:

 On Saturday, 1 June 2013 at 20:22:01 UTC, Nick Sabalausky wrote:

 Core dumps aren't the big problem  safe tries to avoid. The 
 big problem
 is memory corruption, ie trampling memory you didn't expect to 
 (or
 shouldn't be allowed to).


 So, let's say I have:

 --------
 void foo(int* p)  safe
 {
     *p = 0;
 }
 --------


I don't see how this is related to  safe. Remove the attribute, 
and you are in same situation.


 I suppose that this give foo the liberty of saying "p points to 
 someplace valid" ... "and if not, it's not my fault"?


I think in D any reference or pointer may refer to anywhere.

Actually, example above is not a big deal. Example of bigger 
problem:

extern(C) int printf(const char*,...)  safe;

alias int T;

auto foo(lazy T i)  safe
{
    return { return i; } ;
}

auto bar()  safe
{
    T i = 4;
    return foo(i);
}

void baz()  safe
{
    double[2] i = 3.14;
}

void main()  safe
{
    auto x = bar();
    baz();
    printf("%d\n", x());
}

This may print garbage or not, it also can print constant garbage 
or changing, depending on compiler switches. Assume that foo, bar 
and baz are splitted in several modules and instead of int there 
is more complex data structure. This is example of memory error 
not prevented by  safe.
Jun 01 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 1 June 2013 at 19:59:19 UTC, monarch_dodra wrote:

 The way I understood it,  safe defines a list of things that 
 are or aren't legal inside the implementation of a function. It 
 also changes the scheme of bounds checking, in release code.


You completely get the point. Safe attribute does not guarantee 
safety, but blocks some practices which are supposed to be 
unsafe. Such characteristic raises several questions:

1) How strong are  safe commitments? I value them quite weak, 
because  safe function can call trusted and systems function 
indirectly.

2) How (un)safe are those practices which are still allowed? They 
are also unsafe and issue here is not trivial cases like 
dereferencing null.

Current type system has some holes, so any practice blocked in 
 safe can be achieved by employing type system holes (delegates, 
ref and lazy parameters). On the other hand, there is no point in 
using these holes - just do not make a function  safe if it 
cannot be. However there is still a problem of doing 
unintentional (especially memory) errors.


 What bothers me though, is that from an interface point of 
 view, it doesn't really mean anything (or at least, I haven't 
 really understood anything). AFAIK: if I call something 
 " safe", chances of a core dump are relatively "lower", but 
 they can still happen:
 * A function that accepts a pointer as an argument can be 
 marked safe, so all bets are off there, no, since the pointer 
 can be dereferenced?
 * Member functions for structs that have pointers, too, can be 
 marked safe...


Yes,  safe is not an interface feature (in a sense of delivering 
some guarantee). As it stands, it is a commitment to not to do 
some potentially unsafe actions. It is useful only in cases when 
user doesn't want do unsafe things and asks compiler for help in 
detecting some of them.


 Or does it only mean "if you give me valid pointers, I can't 
 core dump*"?
 (*ignoring current flaws, such as escaping slices from static 
 arrays)


Of course not.


 The main reason about this question is that now I'm confused 
 about  trusted: what are the conditions a developer needs to 
 take into account before marking a function " trusted" ?


I think it is the same except what is listed in the spec. Nothing 
conceptually different.


 Ditto for member functions, when they operate on pointer 
 members. Can those be  safe?


What do you mean?


 Yeah, overall, I'm confused as to what " safe" means from an 
 interface point of view :(
Jun 01 2013
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 21:59:18 monarch_dodra wrote:

 The way I understood it,  safe defines a list of things that are
 or aren't legal inside the implementation of a function. It also
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of view,
 it doesn't really mean anything (or at least, I haven't really
 understood anything). AFAIK: if I call something " safe", chances
 of a core dump are relatively "lower", but they can still happen:
 * A function that accepts a pointer as an argument can be marked
 safe, so all bets are off there, no, since the pointer can be
 dereferenced?
 * Member functions for structs that have pointers, too, can be
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't core
 dump*"?
 (*ignoring current flaws, such as escaping slices from static
 arrays)
 
 The main reason about this question is that now I'm confused
 about  trusted: what are the conditions a developer needs to take
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer members.
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an
 interface point of view :(


 safe is for memory safety, meaning that  safe code cannot corrupt memory. You 
can get segfaults due to null pointers and the like, but you can't have code 
which writes passed the end of a buffer, or which uses a freed memory, or does 
anything else which involves writing or reading from memory which variables 
aren't supposed to have access to.

Assuming that there are no bugs in  safe, the one thing that can invalidate it 
is  trusted. With  trusted code, it is the _programmer_ who is then 
guaranteeing that the code is actually  safe. The code is doing something 
which is potentially not safe (and therefore is considered  system by the 
compiler) but which _could_ be safe if the code is correct, and if the 
programmer is marking the code as  trusted, they are then telling the compiler 
that they've verified that the code isn't doing anything which could corrupt 
memory. As long as the programmer doesn't screw that up, then any  safe code 
calling that  trusted function is indeed  safe, but if the programmer screwed 
it up, then you could still get memory corruption. However, here's really no 
way to get around that problem with a systems language, since most code needs 
to eventually call something that's  system (e.g. all I/O needs  system stuff 
internally). But by limiting how much code is  system or  trusted, most code 
is  safe with a minimal amount of code having to have been verified by an 
appropriately competent programmer as being  trusted.

- Jonathan M Davis
Jun 01 2013
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:

  safe is for memory safety, meaning that  safe code cannot 
 corrupt memory. You
 can get segfaults due to null pointers and the like, but you 
 can't have code
 which writes passed the end of a buffer, or which uses a freed 
 memory, or does
 anything else which involves writing or reading from memory 
 which variables
 aren't supposed to have access to.


Not true.

void foo(int* p)  safe
{
	*p = 0;
}

void main()
{
	int[3] buf1 = [1, 2, 3];
	int[1] buf2;
	int* p = buf2.ptr;
	--p;
	foo(p);
	import std.stdio;
	writeln(buf1);
}

For me, this prints [1, 2, 0]. You could easily come up with an 
example which writes to freed memory.

You can argue that foo didn't "cause" this problem (the undefined 
behaviour from the pointer arithmetic in main did), but that's 
irrelevant: what guarantees do I have when I call a  safe 
function that I don't have with any non- safe function?

Do  safe functions only provide guarantees when the inputs are 
valid, or is it the case the  safe functions are guaranteed to 
not *introduce* any new undefined behaviour?
Jun 01 2013
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 23:34:09 Peter Alexander wrote:

 On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:

  safe is for memory safety, meaning that  safe code cannot
 corrupt memory. You
 can get segfaults due to null pointers and the like, but you
 can't have code
 which writes passed the end of a buffer, or which uses a freed
 memory, or does
 anything else which involves writing or reading from memory
 which variables
 aren't supposed to have access to.


 
 Not true.
 
 void foo(int* p)  safe
 {
 	*p = 0;
 }
 
 void main()
 {
 	int[3] buf1 = [1, 2, 3];
 	int[1] buf2;
 	int* p = buf2.ptr;
 	--p;
 	foo(p);
 	import std.stdio;
 	writeln(buf1);
 }
 
 For me, this prints [1, 2, 0]. You could easily come up with an
 example which writes to freed memory.
 
 You can argue that foo didn't "cause" this problem (the undefined
 behaviour from the pointer arithmetic in main did), but that's
 irrelevant: what guarantees do I have when I call a  safe
 function that I don't have with any non- safe function?
 
 Do  safe functions only provide guarantees when the inputs are
 valid, or is it the case the  safe functions are guaranteed to
 not *introduce* any new undefined behaviour?


They're guaranteed to not introduce any such behavior. They can't possibly 
make any guarantees if the caller did  system operations and passed a bad 
pointer to the  safe function. But if all of the functions in the call stack 
are  safe, and you call an  safe function, then you can't get any memory 
corruption unless it (or a function that it calls) calls an  trusted function 
which was incorrectly verified by the programmer who marked it as  trusted.

- Jonathan M Davis
Jun 01 2013
parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 1 June 2013 at 21:41:40 UTC, Jonathan M Davis wrote:

 They're guaranteed to not introduce any such behavior. They 
 can't possibly
 make any guarantees if the caller did  system operations and 
 passed a bad
 pointer to the  safe function. But if all of the functions in 
 the call stack
 are  safe, and you call an  safe function, then you can't get 
 any memory
 corruption unless it (or a function that it calls) calls an 
  trusted function
 which was incorrectly verified by the programmer who marked it 
 as  trusted.

 - Jonathan M Davis


Updated example from above to show how  safe can introduce UB.

import std.stdio;

class A
{
    int[] data;
    ~this()
    {
       writeln(data);
    }
}

void foo(int[] a)  safe
{
    A x = new A;
    x.data = a;
}

void main()  safe
{
    int[4] y;
    foo(y);
}
Jun 01 2013
next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 1 June 2013 at 21:46:01 UTC, Maxim Fomin wrote:

 Updated example from above to show how  safe can introduce UB.

 void main()  safe
 {
    int[4] y;
    foo(y);
 }


I believe that's a compiler bug.

 safe requires:
- No taking the address of a local variable or function parameter.

A slice of a local static array should count as this, but it 
currently doesn't.
Jun 01 2013
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 23:45:59 Maxim Fomin wrote:

 On Saturday, 1 June 2013 at 21:41:40 UTC, Jonathan M Davis wrote:

 They're guaranteed to not introduce any such behavior. They
 can't possibly
 make any guarantees if the caller did  system operations and
 passed a bad
 pointer to the  safe function. But if all of the functions in
 the call stack
 are  safe, and you call an  safe function, then you can't get
 any memory
 corruption unless it (or a function that it calls) calls an
  trusted function
 which was incorrectly verified by the programmer who marked it
 as  trusted.
 
 - Jonathan M Davis


 
 Updated example from above to show how  safe can introduce UB.
 
 import std.stdio;
 
 class A
 {
     int[] data;
     ~this()
     {
        writeln(data);
     }
 }
 
 void foo(int[] a)  safe
 {
     A x = new A;
     x.data = a;
 }
 
 void main()  safe
 {
     int[4] y;
     foo(y);
 }


That's a known bug in  safe. Slicing a static array should be considered 
 system just like taking the address of a local variable is considered 
 system:

http://d.puremagic.com/issues/show_bug.cgi?id=8838

The guarantees of  safe hold only so long as there are no holes in it, but any 
and all holes we find get fixed. Making ref be truly  safe has been a large
part 
of the recent ref discussions, as you can currently get away with doing 
something like

ref int id(ref int i) { return i; }

ref int foo()
{
    int j;
    return id(j);
}

What it looks like we're going to do in this case is detect when this 
situation might happen and insert a runtime check which throws an Error if a 
reference to a local variable tries to escape, but regardless of the solution, 
it's an example of something that is currently considered  safe by the 
compiler when it really isn't. All such holes need to be plugged, or  safe 
isn't doing its job.

So, if you find any more holes in  safe, please report them in bugzilla:

http://d.puremagic.com/issues

- Jonathan M Davis
Jun 01 2013
parent reply Piotr Szturmaj <bncrbme jadamspam.pl> writes:
W dniu 01.06.2013 23:55, Jonathan M Davis pisze:

 The guarantees of  safe hold only so long as there are no holes in it, but any
 and all holes we find get fixed. Making ref be truly  safe has been a large
part
 of the recent ref discussions, as you can currently get away with doing
 something like

 ref int id(ref int i) { return i; }

 ref int foo()
 {
      int j;
      return id(j);
 }


I know that introducing another type qualifier may complicate things
but this would be a compile time solution.

I mean _scope_ type qualifier, so your example could be rewritten as:

ref int id(ref int i) { return i; }

ref int foo()
{
     int j;
     return id(j); // error: could not pass scope(int) as ref int parameter
}

Taking an address of local variable would always yield a scope
qualified type, scope(int) in this example.

Obviously, scope qualified type could be passed to functions taking
scope parameters:

void bar(scope ref int i) { i = 10; }

void foo()
{
     int j = 0;
     bar(j); // ok
     assert(i == 10);
}

I think this could fill the  safety holes.
Jun 01 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 01:12:53 Piotr Szturmaj wrote:

 W dniu 01.06.2013 23:55, Jonathan M Davis pisze:

 The guarantees of  safe hold only so long as there are no holes in it, but
 any and all holes we find get fixed. Making ref be truly  safe has been a
 large part of the recent ref discussions, as you can currently get away
 with doing something like
 
 ref int id(ref int i) { return i; }
 
 ref int foo()
 {
 
      int j;
      return id(j);
 
 }


 
 I know that introducing another type qualifier may complicate things
 but this would be a compile time solution.
 
 I mean _scope_ type qualifier, so your example could be rewritten as:
 
 ref int id(ref int i) { return i; }
 
 ref int foo()
 {
      int j;
      return id(j); // error: could not pass scope(int) as ref int parameter
 }
 
 Taking an address of local variable would always yield a scope
 qualified type, scope(int) in this example.
 
 Obviously, scope qualified type could be passed to functions taking
 scope parameters:
 
 void bar(scope ref int i) { i = 10; }
 
 void foo()
 {
      int j = 0;
      bar(j); // ok
      assert(i == 10);
 }
 
 I think this could fill the  safety holes.


Except that that makes it so that you can't return a ref argument, which is 
completely unacceptable. You need to be able to pass local variables to 
functions which accept ref in  safe code, and you need functions which ref 
arguments to be able to return those arguments by ref. The only thing that we 
want to prevent is a local variable from escaping its original scope. It's 
perfectly valid that the id function accept a local variable by ref and 
returns it by ref. What's invalid is that the function that the local variable 
was declared in then returns it by ref.

Manu suggested something similar to what you're suggesting with the addition 
of having making it so that you can then return variables as scope ref, in 
which case, the caller would see that the function was accepting by scope ref 
and returning by scope ref and that none of the variables that it accepted 
were scope ref in the caller. But this requires having having yet another 
annotation - scope ref - which Andrei was completely against (and Walter too 
IIRC), and it actually would end up making something like this illegal as 
well, went it shouldn't:

scope ref int foo(scope ref int i, scope ref int j)
{
    return j;
}

scope ref bar(scope ref int q)
{
    int i;
    return foo(i, q);
}

The compiler can't know whether it's i or q that's being returned from foo, so 
it would have to given a compilation error, which is more restrictive than the 
runtime solution that has been proposed. So, you can do less, and you have to 
have mark up your functions with even more attributes, and it's yet another 
attribute for those learning the language to have to learn.

Contrast this with simply inserting a very cheap runtime check in the rare 
cases where the compiler detects that a local variable might escape. No 
additional attributes are needed. So, the code is simpler, and there's less 
for people to learn. There's almost no performance hit (and if you want it to 
be zero, then use -noboundscheck). And we lose _zero_ functionality. None of 
that is the case with the scope ref proposal.

Walter and Andrei do not like the idea of introducing even more attributes to 
solve this problem and were very excited to have this solution proposed 
(unfortunately, I'm not sure who proposed it though, since I missed that part 
of the conversation). And I'm inclined to agree with them. It's very simple 
and cheap. The only real downside is that it's caught at runtime rather than 
compile time, but it's quickly and easily caught at runtime, and the 
simplicity of it makes it seem like a great solution.

- Jonathan M Davis
Jun 01 2013
prev sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 01.06.2013 23:34, schrieb Peter Alexander:

 On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:

  safe is for memory safety, meaning that  safe code cannot corrupt
 memory. You
 can get segfaults due to null pointers and the like, but you can't
 have code
 which writes passed the end of a buffer, or which uses a freed memory,
 or does
 anything else which involves writing or reading from memory which
 variables
 aren't supposed to have access to.


 Not true.

 void foo(int* p)  safe
 {
      *p = 0;
 }

 void main()
 {
      int[3] buf1 = [1, 2, 3];
      int[1] buf2;
      int* p = buf2.ptr;
      --p;
      foo(p);
      import std.stdio;
      writeln(buf1);
 }

 For me, this prints [1, 2, 0]. You could easily come up with an example
 which writes to freed memory.

 You can argue that foo didn't "cause" this problem (the undefined
 behaviour from the pointer arithmetic in main did), but that's
 irrelevant: what guarantees do I have when I call a  safe function that
 I don't have with any non- safe function?

 Do  safe functions only provide guarantees when the inputs are valid, or
 is it the case the  safe functions are guaranteed to not *introduce* any
 new undefined behaviour?


I always assumed that the role of  safe is to behave like safe code in 


No C like tricks are allowed and in certain scenarios one could even 
disallow the linkage of modules not considered safe.

For example in .NET, IIS only allows assemblies with unsafe code if 
configured by the administrator. Unsafe code is also forbidden for Go
packages on App Engine.

--
Paulo
Jun 01 2013
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:

 On Saturday, June 01, 2013 21:59:18 monarch_dodra wrote:

 The way I understood it,  safe defines a list of things that 
 are
 or aren't legal inside the implementation of a function. It 
 also
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of 
 view,
 it doesn't really mean anything (or at least, I haven't really
 understood anything). AFAIK: if I call something " safe", 
 chances
 of a core dump are relatively "lower", but they can still 
 happen:
 * A function that accepts a pointer as an argument can be 
 marked
 safe, so all bets are off there, no, since the pointer can be
 dereferenced?
 * Member functions for structs that have pointers, too, can be
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't 
 core
 dump*"?
 (*ignoring current flaws, such as escaping slices from static
 arrays)
 
 The main reason about this question is that now I'm confused
 about  trusted: what are the conditions a developer needs to 
 take
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer 
 members.
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an
 interface point of view :(


  safe is for memory safety, meaning that  safe code cannot 
 corrupt memory. You
 can get segfaults due to null pointers and the like, but you 
 can't have code
 which writes passed the end of a buffer, or which uses a freed 
 memory, or does
 anything else which involves writing or reading from memory 
 which variables
 aren't supposed to have access to.

 Assuming that there are no bugs in  safe, the one thing that 
 can invalidate it
 is  trusted. With  trusted code, it is the _programmer_ who is 
 then
 guaranteeing that the code is actually  safe. The code is doing 
 something
 which is potentially not safe (and therefore is considered 
  system by the
 compiler) but which _could_ be safe if the code is correct, and 
 if the
 programmer is marking the code as  trusted, they are then 
 telling the compiler
 that they've verified that the code isn't doing anything which 
 could corrupt
 memory. As long as the programmer doesn't screw that up, then 
 any  safe code
 calling that  trusted function is indeed  safe, but if the 
 programmer screwed
 it up, then you could still get memory corruption. However, 
 here's really no
 way to get around that problem with a systems language, since 
 most code needs
 to eventually call something that's  system (e.g. all I/O needs 
  system stuff
 internally). But by limiting how much code is  system or 
  trusted, most code
 is  safe with a minimal amount of code having to have been 
 verified by an
 appropriately competent programmer as being  trusted.

 - Jonathan M Davis


OK. But by that standard, can't (mostly) anything be trusted? 
What about something that writes garbage, to a memory location it 
was *asked* to write to? Or if wrong usage of the function can 
lead to an inconsistence memory state, but without "out of bounds 
accesses"?

For instance: "emplace!T(T* p)": This function takes the address 
of a T, and writes T.init over it. It does a memcopy, so it can't 
be  safe, but I can 100% guarantee I'm not doing anything wrong, 
so I'm marking it as  trusted. This should be fine, right? Or is 
raw memory copying alway unsafe?

Now, technically, emplace can't be called in  safe code, since it 
requires a pointer to begin with.

But still, if I were to give emplace an "already constructed 
object", it will happily clobber that object for me, leaking the 
destructor, and possibly putting the program in an invalid memory 
state.

Now, it was *my* fault for calling emplace with an already built 
object, but it was the ( trusted) emplace that clobbered-it.

--------
Long story short, I'm having trouble drawing the line between 
system and trusted functions, especially in regards to these low 
level operations. The same question also works for, say "move" or 
"uninitializedArray": both 100% guarantee bounded memory access, 
but both can leave you with garbage in your memory...
Jun 01 2013
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:

 On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:

 On Saturday, June 01, 2013 21:59:18 monarch_dodra wrote:

 The way I understood it,  safe defines a list of things that
 are
 or aren't legal inside the implementation of a function. It
 also
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of
 view,
 it doesn't really mean anything (or at least, I haven't really
 understood anything). AFAIK: if I call something " safe",
 chances
 of a core dump are relatively "lower", but they can still
 happen:
 * A function that accepts a pointer as an argument can be
 marked
 safe, so all bets are off there, no, since the pointer can be
 dereferenced?
 * Member functions for structs that have pointers, too, can be
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't
 core
 dump*"?
 (*ignoring current flaws, such as escaping slices from static
 arrays)
 
 The main reason about this question is that now I'm confused
 about  trusted: what are the conditions a developer needs to
 take
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer
 members.
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an
 interface point of view :(


 
  safe is for memory safety, meaning that  safe code cannot
 corrupt memory. You
 can get segfaults due to null pointers and the like, but you
 can't have code
 which writes passed the end of a buffer, or which uses a freed
 memory, or does
 anything else which involves writing or reading from memory
 which variables
 aren't supposed to have access to.
 
 Assuming that there are no bugs in  safe, the one thing that
 can invalidate it
 is  trusted. With  trusted code, it is the _programmer_ who is
 then
 guaranteeing that the code is actually  safe. The code is doing
 something
 which is potentially not safe (and therefore is considered
  system by the
 compiler) but which _could_ be safe if the code is correct, and
 if the
 programmer is marking the code as  trusted, they are then
 telling the compiler
 that they've verified that the code isn't doing anything which
 could corrupt
 memory. As long as the programmer doesn't screw that up, then
 any  safe code
 calling that  trusted function is indeed  safe, but if the
 programmer screwed
 it up, then you could still get memory corruption. However,
 here's really no
 way to get around that problem with a systems language, since
 most code needs
 to eventually call something that's  system (e.g. all I/O needs
  system stuff
 internally). But by limiting how much code is  system or
  trusted, most code
 is  safe with a minimal amount of code having to have been
 verified by an
 appropriately competent programmer as being  trusted.
 
 - Jonathan M Davis


 
 OK. But by that standard, can't (mostly) anything be trusted?
 What about something that writes garbage, to a memory location it
 was *asked* to write to? Or if wrong usage of the function can
 lead to an inconsistence memory state, but without "out of bounds
 accesses"?


When a programmer marks a function as  trusted, they are saying that they 
guarantee that the function will not do anything to corrupt memory. So, yes, a 
programmer could mark absolutely anything as  trusted - including stuff that is 
blatantly unsafe and will do all kinds of nasty stuff - but that's the 
programmer's fault. The compiler told them that it was  system and therefore 
could not be verified to be memory safe, and the programmer insisted that it 
was memory safe.

Any code which cannot be guaranteed to be actually safe should not be marked 
with  trusted. It's up to the programmer figure out whether what they're doing 
is valid or not.


 For instance: "emplace!T(T* p)": This function takes the address
 of a T, and writes T.init over it. It does a memcopy, so it can't
 be  safe, but I can 100% guarantee I'm not doing anything wrong,
 so I'm marking it as  trusted. This should be fine, right? Or is
 raw memory copying alway unsafe?


Raw memory copying should be fine as long as the memory being copied from and 
the memory being copied to is valid.


 Now, technically, emplace can't be called in  safe code, since it
 requires a pointer to begin with.


Pointers can be in  safe code. They're perfectly safe in and of themselves. 
It's certain operations on pointers which are unsafe (such as pointer 
arithmetic).


 But still, if I were to give emplace an "already constructed
 object", it will happily clobber that object for me, leaking the
 destructor, and possibly putting the program in an invalid memory
 state.
 
 Now, it was *my* fault for calling emplace with an already built
 object, but it was the ( trusted) emplace that clobbered-it.


Well, given that the safety of the operation relies on what's being passed in, 
the operation itself can't reasonably be marked as  safe, because you can't 
guarantee that the operation isn't going to corrupt memory.

- Jonathan M Davis
Jun 01 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 22:15:00 UTC, Jonathan M Davis wrote:

 On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:

 OK. But by that standard, can't (mostly) anything be trusted?
 What about something that writes garbage, to a memory location 
 it
 was *asked* to write to? Or if wrong usage of the function can
 lead to an inconsistence memory state, but without "out of 
 bounds
 accesses"?


 When a programmer marks a function as  trusted, they are saying 
 that they
 guarantee that the function will not do anything to corrupt 
 memory. So, yes, a
 programmer could mark absolutely anything as  trusted - 
 including stuff that is
 blatantly unsafe and will do all kinds of nasty stuff - but 
 that's the
 programmer's fault.


Well, by "mostly", I did mean stuff that's not blatantly wrong. I 
don't usually write stuff with the express objective of 
clobbering memory.

But given the previous answers, I think I see why anything that 
should work can't be marked  safe.


 But still, if I were to give emplace an "already constructed
 object", it will happily clobber that object for me, leaking 
 the
 destructor, and possibly putting the program in an invalid 
 memory
 state.
 
 Now, it was *my* fault for calling emplace with an already 
 built
 object, but it was the ( trusted) emplace that clobbered-it.


 Well, given that the safety of the operation relies on what's 
 being passed in,
 the operation itself can't reasonably be marked as  safe, 
 because you can't
 guarantee that the operation isn't going to corrupt memory.


But isn't that exactly the same as my "void foo(int* p)  safe{*p 
= 0}" example ? That relies on what is being passed in to 
guarantee safety :/

 confused
Jun 02 2013
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 09:59:08 monarch_dodra wrote:

 On Saturday, 1 June 2013 at 22:15:00 UTC, Jonathan M Davis wrote:

 On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:

 Now, it was *my* fault for calling emplace with an already
 built
 object, but it was the ( trusted) emplace that clobbered-it.


 
 Well, given that the safety of the operation relies on what's
 being passed in,
 the operation itself can't reasonably be marked as  safe,
 because you can't
 guarantee that the operation isn't going to corrupt memory.


 
 But isn't that exactly the same as my "void foo(int* p)  safe{*p
 = 0}" example ? That relies on what is being passed in to
 guarantee safety :/
 
  confused


I don't know. The function isn't doing anything unsafe in and of itself. foo 
_can_ be marked as  safe, because none of the operations that it's doing are 
unsafe. The problem is when the caller does something unsafe. So, maybe that's 
the answer. It's the caller which can't be marked as  safe, because it's doing 
something stupid. foo isn't doing anything unsafe, so it _can_ be  safe.

I think that part of the problem is the fact that  safety is generally viewed 
with the idea that unsafe stuff is called by safe stuff and not that unsafe
stuff 
is called by safe stuff. And these cases under discussion are cases where the 
safe stuff ends up corrupting stuff if called from unsafe stuff.

So, maybe emplace should be marked as  trusted (assuming that it can't corrupt 
anything if called from an  safe function and that its safety does not depend 
on its arguments). It _is_ a bit of a tough topic though.

- Jonathan M Davis
Jun 02 2013
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 02 Jun 2013 03:59:08 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:


 On Saturday, 1 June 2013 at 22:15:00 UTC, Jonathan M Davis wrote:



 Well, given that the safety of the operation relies on what's being  
 passed in,
 the operation itself can't reasonably be marked as  safe, because you  
 can't
 guarantee that the operation isn't going to corrupt memory.


 But isn't that exactly the same as my "void foo(int* p)  safe{*p = 0}"  
 example ? That relies on what is being passed in to guarantee safety :/

  confused


provable  safe depends on the precondition that its parameters are valid  
and  safe.

The easiest way to do this is to mark main as  safe.  Then you can't go  
unsafe.

As people have pointed out, there are bugs/holes.  They need to be fixed.

 trusted should be used VERY cautiously.  It basically says "I know this  
is  safe, but the compiler can't prove it".  These situations should be  
very very rare.

Think of  safe functions as bricks.  By themselves, they are solid and  
will hold up a building well.  But if you put them on top of garbage, they  
will be as useless as cardboard.

-Steve
Jun 03 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 12:03:51 Steven Schveighoffer wrote:

 Think of  safe functions as bricks.  By themselves, they are solid and
 will hold up a building well.  But if you put them on top of garbage, they
 will be as useless as cardboard.


Nice analogy.

- Jonathan M Davis
Jun 03 2013
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/1/13 3:59 PM, monarch_dodra wrote:

 Yeah, overall, I'm confused as to what " safe" means from an interface
 point of view :(


If you call the function from a program with memory integrity and it 
returns, it hasn't compromised the memory integrity of that program.

Homework: define memory integrity :o).


Andrei
Jun 01 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 21:45:18 UTC, Andrei Alexandrescu 
wrote:

 On 6/1/13 3:59 PM, monarch_dodra wrote:

 Yeah, overall, I'm confused as to what " safe" means from an 
 interface
 point of view :(


 If you call the function from a program with memory integrity 
 and it returns, it hasn't compromised the memory integrity of 
 that program.

 Homework: define memory integrity :o).


 Andrei


OK. In a word, I guess that makes sense. I'll stick to that 
standard.

But there is still the "emplace" question: When I call "emplace" 
on a pointer to a built object, is does the program still have 
memory integrity? At the end of the call, was it emplace that 
compromised it? Was it the exact instance the "call was 
initialized"?
Jun 01 2013
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 00:04:18 monarch_dodra wrote:

 On Saturday, 1 June 2013 at 21:45:18 UTC, Andrei Alexandrescu
 
 wrote:

 On 6/1/13 3:59 PM, monarch_dodra wrote:

 Yeah, overall, I'm confused as to what " safe" means from an
 interface
 point of view :(


 
 If you call the function from a program with memory integrity
 and it returns, it hasn't compromised the memory integrity of
 that program.
 
 Homework: define memory integrity :o).
 
 
 Andrei


 
 OK. In a word, I guess that makes sense. I'll stick to that
 standard.
 
 But there is still the "emplace" question: When I call "emplace"
 on a pointer to a built object, is does the program still have
 memory integrity? At the end of the call, was it emplace that
 compromised it? Was it the exact instance the "call was
 initialized"? 


Because the safety of the function depends on the caller, I'd say that it 
should be  system.

- Jonathan M Davis
Jun 01 2013