www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What exactly does " safe" mean?

reply "monarch_dodra" <monarchdodra gmail.com> writes:
The way I understood it,  safe defines a list of things that are 
or aren't legal inside the implementation of a function. It also 
changes the scheme of bounds checking, in release code.

What bothers me though, is that from an interface point of view, 
it doesn't really mean anything (or at least, I haven't really 
understood anything). AFAIK: if I call something " safe", chances 
of a core dump are relatively "lower", but they can still happen:
* A function that accepts a pointer as an argument can be marked 
safe, so all bets are off there, no, since the pointer can be 
dereferenced?
* Member functions for structs that have pointers, too, can be 
marked safe...

Or does it only mean "if you give me valid pointers, I can't core 
dump*"?
(*ignoring current flaws, such as escaping slices from static 
arrays)

The main reason about this question is that now I'm confused 
about  trusted: what are the conditions a developer needs to take 
into account before marking a function " trusted" ?

Ditto for member functions, when they operate on pointer members. 
Can those be  safe?

Yeah, overall, I'm confused as to what " safe" means from an 
interface point of view :(
Jun 01 2013
next sibling parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Sat, 01 Jun 2013 21:59:18 +0200
"monarch_dodra" <monarchdodra gmail.com> wrote:

 The way I understood it,  safe defines a list of things that are 
 or aren't legal inside the implementation of a function. It also 
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of view, 
 it doesn't really mean anything (or at least, I haven't really 
 understood anything). AFAIK: if I call something " safe", chances 
 of a core dump are relatively "lower", but they can still happen:
 * A function that accepts a pointer as an argument can be marked 
 safe, so all bets are off there, no, since the pointer can be 
 dereferenced?
 * Member functions for structs that have pointers, too, can be 
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't core 
 dump*"?
 (*ignoring current flaws, such as escaping slices from static 
 arrays)
 
 The main reason about this question is that now I'm confused 
 about  trusted: what are the conditions a developer needs to take 
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer members. 
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an 
 interface point of view :(

Core dumps aren't the big problem safe tries to avoid. The big problem is memory corruption, ie trampling memory you didn't expect to (or shouldn't be allowed to).
Jun 01 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 20:22:01 UTC, Nick Sabalausky wrote:
 On Sat, 01 Jun 2013 21:59:18 +0200
 "monarch_dodra" <monarchdodra gmail.com> wrote:

 The way I understood it,  safe defines a list of things that 
 are or aren't legal inside the implementation of a function. 
 It also changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of 
 view, it doesn't really mean anything (or at least, I haven't 
 really understood anything). AFAIK: if I call something 
 " safe", chances of a core dump are relatively "lower", but 
 they can still happen:
 * A function that accepts a pointer as an argument can be 
 marked safe, so all bets are off there, no, since the pointer 
 can be dereferenced?
 * Member functions for structs that have pointers, too, can be 
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't 
 core dump*"?
 (*ignoring current flaws, such as escaping slices from static 
 arrays)
 
 The main reason about this question is that now I'm confused 
 about  trusted: what are the conditions a developer needs to 
 take into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer 
 members. Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an 
 interface point of view :(

Core dumps aren't the big problem safe tries to avoid. The big problem is memory corruption, ie trampling memory you didn't expect to (or shouldn't be allowed to).

So, let's say I have: -------- void foo(int* p) safe { *p = 0; } -------- I suppose that this give foo the liberty of saying "p points to someplace valid" ... "and if not, it's not my fault"? I suppose something that is trusted then means "I will not trample your memory under any circumstance, even if I'm doing unsafe things under the hood (unless you give a pointer that is already bad)"?
Jun 01 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 1 June 2013 at 19:59:19 UTC, monarch_dodra wrote:
 The way I understood it,  safe defines a list of things that 
 are or aren't legal inside the implementation of a function. It 
 also changes the scheme of bounds checking, in release code.

You completely get the point. Safe attribute does not guarantee safety, but blocks some practices which are supposed to be unsafe. Such characteristic raises several questions: 1) How strong are safe commitments? I value them quite weak, because safe function can call trusted and systems function indirectly. 2) How (un)safe are those practices which are still allowed? They are also unsafe and issue here is not trivial cases like dereferencing null. Current type system has some holes, so any practice blocked in safe can be achieved by employing type system holes (delegates, ref and lazy parameters). On the other hand, there is no point in using these holes - just do not make a function safe if it cannot be. However there is still a problem of doing unintentional (especially memory) errors.
 What bothers me though, is that from an interface point of 
 view, it doesn't really mean anything (or at least, I haven't 
 really understood anything). AFAIK: if I call something 
 " safe", chances of a core dump are relatively "lower", but 
 they can still happen:
 * A function that accepts a pointer as an argument can be 
 marked safe, so all bets are off there, no, since the pointer 
 can be dereferenced?
 * Member functions for structs that have pointers, too, can be 
 marked safe...

Yes, safe is not an interface feature (in a sense of delivering some guarantee). As it stands, it is a commitment to not to do some potentially unsafe actions. It is useful only in cases when user doesn't want do unsafe things and asks compiler for help in detecting some of them.
 Or does it only mean "if you give me valid pointers, I can't 
 core dump*"?
 (*ignoring current flaws, such as escaping slices from static 
 arrays)

Of course not.
 The main reason about this question is that now I'm confused 
 about  trusted: what are the conditions a developer needs to 
 take into account before marking a function " trusted" ?

I think it is the same except what is listed in the spec. Nothing conceptually different.
 Ditto for member functions, when they operate on pointer 
 members. Can those be  safe?

What do you mean?
 Yeah, overall, I'm confused as to what " safe" means from an 
 interface point of view :(

Jun 01 2013
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 21:59:18 monarch_dodra wrote:
 The way I understood it,  safe defines a list of things that are
 or aren't legal inside the implementation of a function. It also
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of view,
 it doesn't really mean anything (or at least, I haven't really
 understood anything). AFAIK: if I call something " safe", chances
 of a core dump are relatively "lower", but they can still happen:
 * A function that accepts a pointer as an argument can be marked
 safe, so all bets are off there, no, since the pointer can be
 dereferenced?
 * Member functions for structs that have pointers, too, can be
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't core
 dump*"?
 (*ignoring current flaws, such as escaping slices from static
 arrays)
 
 The main reason about this question is that now I'm confused
 about  trusted: what are the conditions a developer needs to take
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer members.
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an
 interface point of view :(

safe is for memory safety, meaning that safe code cannot corrupt memory. You can get segfaults due to null pointers and the like, but you can't have code which writes passed the end of a buffer, or which uses a freed memory, or does anything else which involves writing or reading from memory which variables aren't supposed to have access to. Assuming that there are no bugs in safe, the one thing that can invalidate it is trusted. With trusted code, it is the _programmer_ who is then guaranteeing that the code is actually safe. The code is doing something which is potentially not safe (and therefore is considered system by the compiler) but which _could_ be safe if the code is correct, and if the programmer is marking the code as trusted, they are then telling the compiler that they've verified that the code isn't doing anything which could corrupt memory. As long as the programmer doesn't screw that up, then any safe code calling that trusted function is indeed safe, but if the programmer screwed it up, then you could still get memory corruption. However, here's really no way to get around that problem with a systems language, since most code needs to eventually call something that's system (e.g. all I/O needs system stuff internally). But by limiting how much code is system or trusted, most code is safe with a minimal amount of code having to have been verified by an appropriately competent programmer as being trusted. - Jonathan M Davis
Jun 01 2013
parent Paulo Pinto <pjmlp progtools.org> writes:
Am 01.06.2013 23:34, schrieb Peter Alexander:
 On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:
  safe is for memory safety, meaning that  safe code cannot corrupt
 memory. You
 can get segfaults due to null pointers and the like, but you can't
 have code
 which writes passed the end of a buffer, or which uses a freed memory,
 or does
 anything else which involves writing or reading from memory which
 variables
 aren't supposed to have access to.

Not true. void foo(int* p) safe { *p = 0; } void main() { int[3] buf1 = [1, 2, 3]; int[1] buf2; int* p = buf2.ptr; --p; foo(p); import std.stdio; writeln(buf1); } For me, this prints [1, 2, 0]. You could easily come up with an example which writes to freed memory. You can argue that foo didn't "cause" this problem (the undefined behaviour from the pointer arithmetic in main did), but that's irrelevant: what guarantees do I have when I call a safe function that I don't have with any non- safe function? Do safe functions only provide guarantees when the inputs are valid, or is it the case the safe functions are guaranteed to not *introduce* any new undefined behaviour?

I always assumed that the role of safe is to behave like safe code in Ada, Modula-3, C#, Oberon family and so on. No C like tricks are allowed and in certain scenarios one could even disallow the linkage of modules not considered safe. For example in .NET, IIS only allows assemblies with unsafe code if configured by the administrator. Unsafe code is also forbidden for Go packages on App Engine. -- Paulo
Jun 01 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 1 June 2013 at 20:31:45 UTC, monarch_dodra wrote:
 On Saturday, 1 June 2013 at 20:22:01 UTC, Nick Sabalausky wrote:
 Core dumps aren't the big problem  safe tries to avoid. The 
 big problem
 is memory corruption, ie trampling memory you didn't expect to 
 (or
 shouldn't be allowed to).

So, let's say I have: -------- void foo(int* p) safe { *p = 0; } --------

I don't see how this is related to safe. Remove the attribute, and you are in same situation.
 I suppose that this give foo the liberty of saying "p points to 
 someplace valid" ... "and if not, it's not my fault"?

I think in D any reference or pointer may refer to anywhere. Actually, example above is not a big deal. Example of bigger problem: extern(C) int printf(const char*,...) safe; alias int T; auto foo(lazy T i) safe { return { return i; } ; } auto bar() safe { T i = 4; return foo(i); } void baz() safe { double[2] i = 3.14; } void main() safe { auto x = bar(); baz(); printf("%d\n", x()); } This may print garbage or not, it also can print constant garbage or changing, depending on compiler switches. Assume that foo, bar and baz are splitted in several modules and instead of int there is more complex data structure. This is example of memory error not prevented by safe.
Jun 01 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:
  safe is for memory safety, meaning that  safe code cannot 
 corrupt memory. You
 can get segfaults due to null pointers and the like, but you 
 can't have code
 which writes passed the end of a buffer, or which uses a freed 
 memory, or does
 anything else which involves writing or reading from memory 
 which variables
 aren't supposed to have access to.

Not true. void foo(int* p) safe { *p = 0; } void main() { int[3] buf1 = [1, 2, 3]; int[1] buf2; int* p = buf2.ptr; --p; foo(p); import std.stdio; writeln(buf1); } For me, this prints [1, 2, 0]. You could easily come up with an example which writes to freed memory. You can argue that foo didn't "cause" this problem (the undefined behaviour from the pointer arithmetic in main did), but that's irrelevant: what guarantees do I have when I call a safe function that I don't have with any non- safe function? Do safe functions only provide guarantees when the inputs are valid, or is it the case the safe functions are guaranteed to not *introduce* any new undefined behaviour?
Jun 01 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:
 On Saturday, June 01, 2013 21:59:18 monarch_dodra wrote:
 The way I understood it,  safe defines a list of things that 
 are
 or aren't legal inside the implementation of a function. It 
 also
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of 
 view,
 it doesn't really mean anything (or at least, I haven't really
 understood anything). AFAIK: if I call something " safe", 
 chances
 of a core dump are relatively "lower", but they can still 
 happen:
 * A function that accepts a pointer as an argument can be 
 marked
 safe, so all bets are off there, no, since the pointer can be
 dereferenced?
 * Member functions for structs that have pointers, too, can be
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't 
 core
 dump*"?
 (*ignoring current flaws, such as escaping slices from static
 arrays)
 
 The main reason about this question is that now I'm confused
 about  trusted: what are the conditions a developer needs to 
 take
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer 
 members.
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an
 interface point of view :(

safe is for memory safety, meaning that safe code cannot corrupt memory. You can get segfaults due to null pointers and the like, but you can't have code which writes passed the end of a buffer, or which uses a freed memory, or does anything else which involves writing or reading from memory which variables aren't supposed to have access to. Assuming that there are no bugs in safe, the one thing that can invalidate it is trusted. With trusted code, it is the _programmer_ who is then guaranteeing that the code is actually safe. The code is doing something which is potentially not safe (and therefore is considered system by the compiler) but which _could_ be safe if the code is correct, and if the programmer is marking the code as trusted, they are then telling the compiler that they've verified that the code isn't doing anything which could corrupt memory. As long as the programmer doesn't screw that up, then any safe code calling that trusted function is indeed safe, but if the programmer screwed it up, then you could still get memory corruption. However, here's really no way to get around that problem with a systems language, since most code needs to eventually call something that's system (e.g. all I/O needs system stuff internally). But by limiting how much code is system or trusted, most code is safe with a minimal amount of code having to have been verified by an appropriately competent programmer as being trusted. - Jonathan M Davis

OK. But by that standard, can't (mostly) anything be trusted? What about something that writes garbage, to a memory location it was *asked* to write to? Or if wrong usage of the function can lead to an inconsistence memory state, but without "out of bounds accesses"? For instance: "emplace!T(T* p)": This function takes the address of a T, and writes T.init over it. It does a memcopy, so it can't be safe, but I can 100% guarantee I'm not doing anything wrong, so I'm marking it as trusted. This should be fine, right? Or is raw memory copying alway unsafe? Now, technically, emplace can't be called in safe code, since it requires a pointer to begin with. But still, if I were to give emplace an "already constructed object", it will happily clobber that object for me, leaking the destructor, and possibly putting the program in an invalid memory state. Now, it was *my* fault for calling emplace with an already built object, but it was the ( trusted) emplace that clobbered-it. -------- Long story short, I'm having trouble drawing the line between system and trusted functions, especially in regards to these low level operations. The same question also works for, say "move" or "uninitializedArray": both 100% guarantee bounded memory access, but both can leave you with garbage in your memory...
Jun 01 2013
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 23:34:09 Peter Alexander wrote:
 On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:
  safe is for memory safety, meaning that  safe code cannot
 corrupt memory. You
 can get segfaults due to null pointers and the like, but you
 can't have code
 which writes passed the end of a buffer, or which uses a freed
 memory, or does
 anything else which involves writing or reading from memory
 which variables
 aren't supposed to have access to.

Not true. void foo(int* p) safe { *p = 0; } void main() { int[3] buf1 = [1, 2, 3]; int[1] buf2; int* p = buf2.ptr; --p; foo(p); import std.stdio; writeln(buf1); } For me, this prints [1, 2, 0]. You could easily come up with an example which writes to freed memory. You can argue that foo didn't "cause" this problem (the undefined behaviour from the pointer arithmetic in main did), but that's irrelevant: what guarantees do I have when I call a safe function that I don't have with any non- safe function? Do safe functions only provide guarantees when the inputs are valid, or is it the case the safe functions are guaranteed to not *introduce* any new undefined behaviour?

They're guaranteed to not introduce any such behavior. They can't possibly make any guarantees if the caller did system operations and passed a bad pointer to the safe function. But if all of the functions in the call stack are safe, and you call an safe function, then you can't get any memory corruption unless it (or a function that it calls) calls an trusted function which was incorrectly verified by the programmer who marked it as trusted. - Jonathan M Davis
Jun 01 2013
parent Piotr Szturmaj <bncrbme jadamspam.pl> writes:
W dniu 01.06.2013 23:55, Jonathan M Davis pisze:
 The guarantees of  safe hold only so long as there are no holes in it, but any
 and all holes we find get fixed. Making ref be truly  safe has been a large
part
 of the recent ref discussions, as you can currently get away with doing
 something like

 ref int id(ref int i) { return i; }

 ref int foo()
 {
      int j;
      return id(j);
 }

I know that introducing another type qualifier may complicate things but this would be a compile time solution. I mean _scope_ type qualifier, so your example could be rewritten as: ref int id(ref int i) { return i; } ref int foo() { int j; return id(j); // error: could not pass scope(int) as ref int parameter } Taking an address of local variable would always yield a scope qualified type, scope(int) in this example. Obviously, scope qualified type could be passed to functions taking scope parameters: void bar(scope ref int i) { i = 10; } void foo() { int j = 0; bar(j); // ok assert(i == 10); } I think this could fill the safety holes.
Jun 01 2013
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/1/13 3:59 PM, monarch_dodra wrote:
 Yeah, overall, I'm confused as to what " safe" means from an interface
 point of view :(

If you call the function from a program with memory integrity and it returns, it hasn't compromised the memory integrity of that program. Homework: define memory integrity :o). Andrei
Jun 01 2013
prev sibling next sibling parent "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 1 June 2013 at 21:41:40 UTC, Jonathan M Davis wrote:
 They're guaranteed to not introduce any such behavior. They 
 can't possibly
 make any guarantees if the caller did  system operations and 
 passed a bad
 pointer to the  safe function. But if all of the functions in 
 the call stack
 are  safe, and you call an  safe function, then you can't get 
 any memory
 corruption unless it (or a function that it calls) calls an 
  trusted function
 which was incorrectly verified by the programmer who marked it 
 as  trusted.

 - Jonathan M Davis

Updated example from above to show how safe can introduce UB. import std.stdio; class A { int[] data; ~this() { writeln(data); } } void foo(int[] a) safe { A x = new A; x.data = a; } void main() safe { int[4] y; foo(y); }
Jun 01 2013
prev sibling next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Saturday, 1 June 2013 at 21:46:01 UTC, Maxim Fomin wrote:
 Updated example from above to show how  safe can introduce UB.

 void main()  safe
 {
    int[4] y;
    foo(y);
 }

I believe that's a compiler bug. safe requires: - No taking the address of a local variable or function parameter. A slice of a local static array should count as this, but it currently doesn't.
Jun 01 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 23:45:59 Maxim Fomin wrote:
 On Saturday, 1 June 2013 at 21:41:40 UTC, Jonathan M Davis wrote:
 They're guaranteed to not introduce any such behavior. They
 can't possibly
 make any guarantees if the caller did  system operations and
 passed a bad
 pointer to the  safe function. But if all of the functions in
 the call stack
 are  safe, and you call an  safe function, then you can't get
 any memory
 corruption unless it (or a function that it calls) calls an
  trusted function
 which was incorrectly verified by the programmer who marked it
 as  trusted.
 
 - Jonathan M Davis

Updated example from above to show how safe can introduce UB. import std.stdio; class A { int[] data; ~this() { writeln(data); } } void foo(int[] a) safe { A x = new A; x.data = a; } void main() safe { int[4] y; foo(y); }

That's a known bug in safe. Slicing a static array should be considered system just like taking the address of a local variable is considered system: http://d.puremagic.com/issues/show_bug.cgi?id=8838 The guarantees of safe hold only so long as there are no holes in it, but any and all holes we find get fixed. Making ref be truly safe has been a large part of the recent ref discussions, as you can currently get away with doing something like ref int id(ref int i) { return i; } ref int foo() { int j; return id(j); } What it looks like we're going to do in this case is detect when this situation might happen and insert a runtime check which throws an Error if a reference to a local variable tries to escape, but regardless of the solution, it's an example of something that is currently considered safe by the compiler when it really isn't. All such holes need to be plugged, or safe isn't doing its job. So, if you find any more holes in safe, please report them in bugzilla: http://d.puremagic.com/issues - Jonathan M Davis
Jun 01 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 21:45:18 UTC, Andrei Alexandrescu 
wrote:
 On 6/1/13 3:59 PM, monarch_dodra wrote:
 Yeah, overall, I'm confused as to what " safe" means from an 
 interface
 point of view :(

If you call the function from a program with memory integrity and it returns, it hasn't compromised the memory integrity of that program. Homework: define memory integrity :o). Andrei

OK. In a word, I guess that makes sense. I'll stick to that standard. But there is still the "emplace" question: When I call "emplace" on a pointer to a built object, is does the program still have memory integrity? At the end of the call, was it emplace that compromised it? Was it the exact instance the "call was initialized"?
Jun 01 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 00:04:18 monarch_dodra wrote:
 On Saturday, 1 June 2013 at 21:45:18 UTC, Andrei Alexandrescu
 
 wrote:
 On 6/1/13 3:59 PM, monarch_dodra wrote:
 Yeah, overall, I'm confused as to what " safe" means from an
 interface
 point of view :(

If you call the function from a program with memory integrity and it returns, it hasn't compromised the memory integrity of that program. Homework: define memory integrity :o). Andrei

OK. In a word, I guess that makes sense. I'll stick to that standard. But there is still the "emplace" question: When I call "emplace" on a pointer to a built object, is does the program still have memory integrity? At the end of the call, was it emplace that compromised it? Was it the exact instance the "call was initialized"?

Because the safety of the function depends on the caller, I'd say that it should be system. - Jonathan M Davis
Jun 01 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:
 On Saturday, 1 June 2013 at 21:02:44 UTC, Jonathan M Davis wrote:
 On Saturday, June 01, 2013 21:59:18 monarch_dodra wrote:
 The way I understood it,  safe defines a list of things that
 are
 or aren't legal inside the implementation of a function. It
 also
 changes the scheme of bounds checking, in release code.
 
 What bothers me though, is that from an interface point of
 view,
 it doesn't really mean anything (or at least, I haven't really
 understood anything). AFAIK: if I call something " safe",
 chances
 of a core dump are relatively "lower", but they can still
 happen:
 * A function that accepts a pointer as an argument can be
 marked
 safe, so all bets are off there, no, since the pointer can be
 dereferenced?
 * Member functions for structs that have pointers, too, can be
 marked safe...
 
 Or does it only mean "if you give me valid pointers, I can't
 core
 dump*"?
 (*ignoring current flaws, such as escaping slices from static
 arrays)
 
 The main reason about this question is that now I'm confused
 about  trusted: what are the conditions a developer needs to
 take
 into account before marking a function " trusted" ?
 
 Ditto for member functions, when they operate on pointer
 members.
 Can those be  safe?
 
 Yeah, overall, I'm confused as to what " safe" means from an
 interface point of view :(

safe is for memory safety, meaning that safe code cannot corrupt memory. You can get segfaults due to null pointers and the like, but you can't have code which writes passed the end of a buffer, or which uses a freed memory, or does anything else which involves writing or reading from memory which variables aren't supposed to have access to. Assuming that there are no bugs in safe, the one thing that can invalidate it is trusted. With trusted code, it is the _programmer_ who is then guaranteeing that the code is actually safe. The code is doing something which is potentially not safe (and therefore is considered system by the compiler) but which _could_ be safe if the code is correct, and if the programmer is marking the code as trusted, they are then telling the compiler that they've verified that the code isn't doing anything which could corrupt memory. As long as the programmer doesn't screw that up, then any safe code calling that trusted function is indeed safe, but if the programmer screwed it up, then you could still get memory corruption. However, here's really no way to get around that problem with a systems language, since most code needs to eventually call something that's system (e.g. all I/O needs system stuff internally). But by limiting how much code is system or trusted, most code is safe with a minimal amount of code having to have been verified by an appropriately competent programmer as being trusted. - Jonathan M Davis

OK. But by that standard, can't (mostly) anything be trusted? What about something that writes garbage, to a memory location it was *asked* to write to? Or if wrong usage of the function can lead to an inconsistence memory state, but without "out of bounds accesses"?

When a programmer marks a function as trusted, they are saying that they guarantee that the function will not do anything to corrupt memory. So, yes, a programmer could mark absolutely anything as trusted - including stuff that is blatantly unsafe and will do all kinds of nasty stuff - but that's the programmer's fault. The compiler told them that it was system and therefore could not be verified to be memory safe, and the programmer insisted that it was memory safe. Any code which cannot be guaranteed to be actually safe should not be marked with trusted. It's up to the programmer figure out whether what they're doing is valid or not.
 For instance: "emplace!T(T* p)": This function takes the address
 of a T, and writes T.init over it. It does a memcopy, so it can't
 be  safe, but I can 100% guarantee I'm not doing anything wrong,
 so I'm marking it as  trusted. This should be fine, right? Or is
 raw memory copying alway unsafe?

Raw memory copying should be fine as long as the memory being copied from and the memory being copied to is valid.
 Now, technically, emplace can't be called in  safe code, since it
 requires a pointer to begin with.

Pointers can be in safe code. They're perfectly safe in and of themselves. It's certain operations on pointers which are unsafe (such as pointer arithmetic).
 But still, if I were to give emplace an "already constructed
 object", it will happily clobber that object for me, leaking the
 destructor, and possibly putting the program in an invalid memory
 state.
 
 Now, it was *my* fault for calling emplace with an already built
 object, but it was the ( trusted) emplace that clobbered-it.

Well, given that the safety of the operation relies on what's being passed in, the operation itself can't reasonably be marked as safe, because you can't guarantee that the operation isn't going to corrupt memory. - Jonathan M Davis
Jun 01 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 01:12:53 Piotr Szturmaj wrote:
 W dniu 01.06.2013 23:55, Jonathan M Davis pisze:
 The guarantees of  safe hold only so long as there are no holes in it, but
 any and all holes we find get fixed. Making ref be truly  safe has been a
 large part of the recent ref discussions, as you can currently get away
 with doing something like
 
 ref int id(ref int i) { return i; }
 
 ref int foo()
 {
 
      int j;
      return id(j);
 
 }

I know that introducing another type qualifier may complicate things but this would be a compile time solution. I mean _scope_ type qualifier, so your example could be rewritten as: ref int id(ref int i) { return i; } ref int foo() { int j; return id(j); // error: could not pass scope(int) as ref int parameter } Taking an address of local variable would always yield a scope qualified type, scope(int) in this example. Obviously, scope qualified type could be passed to functions taking scope parameters: void bar(scope ref int i) { i = 10; } void foo() { int j = 0; bar(j); // ok assert(i == 10); } I think this could fill the safety holes.

Except that that makes it so that you can't return a ref argument, which is completely unacceptable. You need to be able to pass local variables to functions which accept ref in safe code, and you need functions which ref arguments to be able to return those arguments by ref. The only thing that we want to prevent is a local variable from escaping its original scope. It's perfectly valid that the id function accept a local variable by ref and returns it by ref. What's invalid is that the function that the local variable was declared in then returns it by ref. Manu suggested something similar to what you're suggesting with the addition of having making it so that you can then return variables as scope ref, in which case, the caller would see that the function was accepting by scope ref and returning by scope ref and that none of the variables that it accepted were scope ref in the caller. But this requires having having yet another annotation - scope ref - which Andrei was completely against (and Walter too IIRC), and it actually would end up making something like this illegal as well, went it shouldn't: scope ref int foo(scope ref int i, scope ref int j) { return j; } scope ref bar(scope ref int q) { int i; return foo(i, q); } The compiler can't know whether it's i or q that's being returned from foo, so it would have to given a compilation error, which is more restrictive than the runtime solution that has been proposed. So, you can do less, and you have to have mark up your functions with even more attributes, and it's yet another attribute for those learning the language to have to learn. Contrast this with simply inserting a very cheap runtime check in the rare cases where the compiler detects that a local variable might escape. No additional attributes are needed. So, the code is simpler, and there's less for people to learn. There's almost no performance hit (and if you want it to be zero, then use -noboundscheck). And we lose _zero_ functionality. None of that is the case with the scope ref proposal. Walter and Andrei do not like the idea of introducing even more attributes to solve this problem and were very excited to have this solution proposed (unfortunately, I'm not sure who proposed it though, since I missed that part of the conversation). And I'm inclined to agree with them. It's very simple and cheap. The only real downside is that it's caught at runtime rather than compile time, but it's quickly and easily caught at runtime, and the simplicity of it makes it seem like a great solution. - Jonathan M Davis
Jun 01 2013
prev sibling next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Saturday, 1 June 2013 at 22:15:00 UTC, Jonathan M Davis wrote:
 On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:
 OK. But by that standard, can't (mostly) anything be trusted?
 What about something that writes garbage, to a memory location 
 it
 was *asked* to write to? Or if wrong usage of the function can
 lead to an inconsistence memory state, but without "out of 
 bounds
 accesses"?

When a programmer marks a function as trusted, they are saying that they guarantee that the function will not do anything to corrupt memory. So, yes, a programmer could mark absolutely anything as trusted - including stuff that is blatantly unsafe and will do all kinds of nasty stuff - but that's the programmer's fault.

Well, by "mostly", I did mean stuff that's not blatantly wrong. I don't usually write stuff with the express objective of clobbering memory. But given the previous answers, I think I see why anything that should work can't be marked safe.
 But still, if I were to give emplace an "already constructed
 object", it will happily clobber that object for me, leaking 
 the
 destructor, and possibly putting the program in an invalid 
 memory
 state.
 
 Now, it was *my* fault for calling emplace with an already 
 built
 object, but it was the ( trusted) emplace that clobbered-it.

Well, given that the safety of the operation relies on what's being passed in, the operation itself can't reasonably be marked as safe, because you can't guarantee that the operation isn't going to corrupt memory.

But isn't that exactly the same as my "void foo(int* p) safe{*p = 0}" example ? That relies on what is being passed in to guarantee safety :/ confused
Jun 02 2013
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, June 02, 2013 09:59:08 monarch_dodra wrote:
 On Saturday, 1 June 2013 at 22:15:00 UTC, Jonathan M Davis wrote:
 On Saturday, June 01, 2013 23:41:32 monarch_dodra wrote:
 Now, it was *my* fault for calling emplace with an already
 built
 object, but it was the ( trusted) emplace that clobbered-it.

Well, given that the safety of the operation relies on what's being passed in, the operation itself can't reasonably be marked as safe, because you can't guarantee that the operation isn't going to corrupt memory.

But isn't that exactly the same as my "void foo(int* p) safe{*p = 0}" example ? That relies on what is being passed in to guarantee safety :/ confused

I don't know. The function isn't doing anything unsafe in and of itself. foo _can_ be marked as safe, because none of the operations that it's doing are unsafe. The problem is when the caller does something unsafe. So, maybe that's the answer. It's the caller which can't be marked as safe, because it's doing something stupid. foo isn't doing anything unsafe, so it _can_ be safe. I think that part of the problem is the fact that safety is generally viewed with the idea that unsafe stuff is called by safe stuff and not that unsafe stuff is called by safe stuff. And these cases under discussion are cases where the safe stuff ends up corrupting stuff if called from unsafe stuff. So, maybe emplace should be marked as trusted (assuming that it can't corrupt anything if called from an safe function and that its safety does not depend on its arguments). It _is_ a bit of a tough topic though. - Jonathan M Davis
Jun 02 2013
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 02 Jun 2013 03:59:08 -0400, monarch_dodra <monarchdodra gmail.com>  
wrote:

 On Saturday, 1 June 2013 at 22:15:00 UTC, Jonathan M Davis wrote:

 Well, given that the safety of the operation relies on what's being  
 passed in,
 the operation itself can't reasonably be marked as  safe, because you  
 can't
 guarantee that the operation isn't going to corrupt memory.

But isn't that exactly the same as my "void foo(int* p) safe{*p = 0}" example ? That relies on what is being passed in to guarantee safety :/ confused

provable safe depends on the precondition that its parameters are valid and safe. The easiest way to do this is to mark main as safe. Then you can't go unsafe. As people have pointed out, there are bugs/holes. They need to be fixed. trusted should be used VERY cautiously. It basically says "I know this is safe, but the compiler can't prove it". These situations should be very very rare. Think of safe functions as bricks. By themselves, they are solid and will hold up a building well. But if you put them on top of garbage, they will be as useless as cardboard. -Steve
Jun 03 2013
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, June 03, 2013 12:03:51 Steven Schveighoffer wrote:
 Think of  safe functions as bricks.  By themselves, they are solid and
 will hold up a building well.  But if you put them on top of garbage, they
 will be as useless as cardboard.

Nice analogy. - Jonathan M Davis
Jun 03 2013