www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - trusted and return ref

reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
Since neither Andrei or Walter are able to say something sensible 
on these issues when asked, I apparently need to learn something 
about the "consistency" of C memory safety.

I'm happy to listen to anyone who can explain this to me:

1. My understanding is that  trusted is supposed to give memory 
safety escapes by providing a context which reestablish a memory 
safety context on return.

Yet in this thread 
http://forum.dlang.org/thread/mcik3j$153g$1 digitalmars.com it is 
stated that this paradigm is an example of «careful use of 
 trusted»:

     count = (()  trusted => cast(uint*) malloc(uint.sizeof))();
     …arbitrary code…
     (()  trusted => free(count))();

They way I see it, this is equivalent to typing a 
reinterpret_casting malloc and free as memorysafe operations in 
isolation, basically providing a malloc!int() and free() as 
memory safe functions. But why is malloc and free not considered 
safe by default then? These  trusted functions clearly cannot 
prevent leaks within their own context. You would need a 
 trusted-only storage class on the receiving pointer to do that 
and a  trusted move type.

If this is careful use of  trusted, then I don't see the point of 
having  trusted at all. What is the purpose? What is it meant to 
cover? In order for  trusted to make sense in this code segment ( 
http://dpaste.dzfl.pl/f3d854feede9 ) I would say that the whole 
class will have to be marked  trusted. Is that possible?


2. In a recent change D now has implemented "return ref", but 
when I ask if this has implications for coroutines that take ref 
as parameters, I get no answer. So what is the deal here? Are 
coroutines banned from taking ref arguments? Will it be possible 
to retain unsafe references by yielding? Since the feature is 
implemented I presume there is a clear answer to this.

What is the scope of memory safety in D? Is it a lint-like 
feature, or is it something to be relied upon?
Feb 24 2015
next sibling parent reply "w0rp" <devw0rp gmail.com> writes:
On Tuesday, 24 February 2015 at 22:37:58 UTC, Ola Fosheim Grøstad 
wrote:

 If this is careful use of  trusted, then I don't see the point 
 of having  trusted at all. What is the purpose? What is it 
 meant to cover? In order for  trusted to make sense in this 
 code segment ( http://dpaste.dzfl.pl/f3d854feede9 ) I would say 
 that the whole class will have to be marked  trusted. Is that 
 possible?


In general,  trusted means "I have proven myself that this code 
is actually safe, eeven though it uses unsafe features." The 
compiler has to be pessimistic and assume that everything which 
can be used unsafely will be used unsafely.  trusted, as it is 
used here, is used to say, "I assure you I have used this in a 
safe manner."

I would like to see  trusted blocks, although I can see Andrei's 
argument of not making it easy to do intentionally.
Feb 24 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 24 February 2015 at 22:49:17 UTC, w0rp wrote:

 In general,  trusted means "I have proven myself that this code 
 is actually safe, eeven though it uses unsafe features." The 
 compiler has to be pessimistic and assume that everything which 
 can be used unsafely will be used unsafely.  trusted, as it is 
 used here, is used to say, "I assure you I have used this in a 
 safe manner."


 From http://dlang.org/function.html#trusted-functions :

«Trusted functions are guaranteed by the programmer to not 
exhibit any undefined behavior if called by a safe function.»

I take this to mean that anything that is wrapped up in  trusted 
should not violate memory safety when in injected into any 
arbitrary context marked as  safe.


 I would like to see  trusted blocks, although I can see 
 Andrei's argument of not making it easy to do intentionally.


If  trusted is meant to be used the way he is doing it in this 
example then there is no reason to not provide  trusted-blocks. 
An inlined lambda is a block...

But it provides poor encapsulation and makes for weak typing.
Feb 24 2015
parent reply "w0rp" <devw0rp gmail.com> writes:
On Wednesday, 25 February 2015 at 06:48:17 UTC, Ola Fosheim 
Grøstad wrote:

 On Tuesday, 24 February 2015 at 22:49:17 UTC, w0rp wrote:

 In general,  trusted means "I have proven myself that this 
 code is actually safe, eeven though it uses unsafe features." 
 The compiler has to be pessimistic and assume that everything 
 which can be used unsafely will be used unsafely.  trusted, as 
 it is used here, is used to say, "I assure you I have used 
 this in a safe manner."


 From http://dlang.org/function.html#trusted-functions :

 «Trusted functions are guaranteed by the programmer to not 
 exhibit any undefined behavior if called by a safe function.»

 I take this to mean that anything that is wrapped up in 
  trusted should not violate memory safety when in injected into 
 any arbitrary context marked as  safe.


The key phrase is "guaranteed by the programmer." Which means 
that the programmer, not the compiler, is providing a guarantee 
that calling a  trusted function will not violate memory safety. 
If the programmer cannot make that guarantee, the function should 
be marked as  system instead. It's a mechanism which allows 
humans to achieve something the compiler isn't capable of 
achieving, at least at this point in time.

Much in the same way that a compiler cannot prove in general that 
programs will terminate, it can be very difficult for a compiler 
to prove that your program will not violate memory safety when 
the language is capable of calling into C code, etc. If you don't 
have an annotation like  trusted, the amount of code which could 
be run from  safe functions would be very small indeed.
Mar 03 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 3 March 2015 at 20:56:50 UTC, w0rp wrote:

 The key phrase is "guaranteed by the programmer." Which means 
 that the programmer, not the compiler, is providing a guarantee 
 that calling a  trusted function will not violate memory 
 safety. If the programmer cannot make that guarantee, the 
 function should be marked as  system instead. It's a mechanism 
 which allows humans to achieve something the compiler isn't 
 capable of achieving, at least at this point in time.


Yes, but in that case it should not be OK to wrap up "free(obj)" 
as  trusted, since it cannot be called safely from  safe code (as 
in any constellation of  safe code). So either the wording of the 
spec has to change or this practice goes against the spec.
Mar 03 2015
prev sibling next sibling parent reply "anonymous" <anonymous example.com> writes:
On Tuesday, 24 February 2015 at 22:37:58 UTC, Ola Fosheim Grøstad 
wrote:

 1. My understanding is that  trusted is supposed to give memory 
 safety escapes by providing a context which reestablish a 
 memory safety context on return.


Yep, that's how I got it, too. A  trusted function is supposed to 
be memory-safe.


 Yet in this thread 
 http://forum.dlang.org/thread/mcik3j$153g$1 digitalmars.com it 
 is stated that this paradigm is an example of «careful use of 
  trusted»:

     count = (()  trusted => cast(uint*) malloc(uint.sizeof))();
     …arbitrary code…
     (()  trusted => free(count))();

 They way I see it, this is equivalent to typing a 
 reinterpret_casting malloc and free as memorysafe operations in 
 isolation, basically providing a malloc!int() and free() as 
 memory safe functions.


Yep. "Careful use": You have be careful when you (ab)use  trusted 
like this.

The idea is that the compiler enforces safety for the rest of the 
code. You have to be cautious about the effects of the  trusted 
malloc/free, but the compiler checks the other stuff. If the 
whole function was  trusted, the compiler wouldn't catch other 
safety violations that are not related to malloc/free.

The downside is that  safe on that function then doesn't mean 
"compiler verified memory-safe" anymore. Instead it means 
"compiler assisted  trusted".

There's also the other way around: Mark the function as  trusted 
and throw () safe{...}() covers over the non-problematic parts. 
This doesn't work when a template parameter affects the safety, 
though.


 But why is malloc and free not considered safe by default then?


Well, because they aren't.


 These  trusted functions clearly cannot prevent leaks within 
 their own context. You would need a  trusted-only storage class 
 on the receiving pointer to do that and a  trusted move type.

 If this is careful use of  trusted, then I don't see the point 
 of having  trusted at all. What is the purpose? What is it 
 meant to cover? In order for  trusted to make sense in this 
 code segment ( http://dpaste.dzfl.pl/f3d854feede9 ) I would say 
 that the whole class will have to be marked  trusted. Is that 
 possible?


The goal is to have human verified, compiler recognized 
memory-safety, when E allows for it.

You can't:
* mark nothing  safe/ trusted, because malloc/free are not safe;
* mark the methods  trusted, because E may be unsafe.

 trusted malloc/free is a hack, but it allows the compiler to 
infer  safe iff E is safe.
Feb 24 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 25 February 2015 at 00:12:41 UTC, anonymous wrote:

 If the whole function was  trusted, the compiler wouldn't catch 
 other safety violations that are not related to malloc/free.


You don't need to. In C++ you use a separate " trusted" 
data-structure for capturing ownership, aka unique_ptr (which 
provides a type related to linear typing).


 The downside is that  safe on that function then doesn't mean 
 "compiler verified memory-safe" anymore. Instead it means 
 "compiler assisted  trusted".


Not only on that function, on the whole data structure? You no 
longer have a scope for what code is considered dangerous.



 There's also the other way around: Mark the function as 
  trusted and throw () safe{...}() covers over the 
 non-problematic parts. This doesn't work when a template 
 parameter affects the safety, though.


That sounds more attractive than the provided example, but the 
right thing to do is to establish proper encapsulation. That 
means you need a protection level that is stronger than "private" 
that restricts "unsafe state" to a  trusted vetted construct. 
Like unique_ptr informally does in C++.



 But why is malloc and free not considered safe by default then?


 Well, because they aren't.


So that should change?



 The goal is to have human verified, compiler recognized 
 memory-safety, when E allows for it.

 You can't:
 * mark nothing  safe/ trusted, because malloc/free are not safe;
 * mark the methods  trusted, because E may be unsafe.

  trusted malloc/free is a hack, but it allows the compiler to 
 infer  safe iff E is safe.


You mean outside RCArray, iff RCArray as a whole is manually 
verified?  But that would surely mean that the  trusted region is 
RCArray and neither the constructor or malloc/free?

And that assumes strong typing, which D currently does not 
provide. Without strong typing it will be very difficult for the 
compiler to infer anything across compilation units.
Feb 24 2015
parent reply "anonymous" <anonymous example.com> writes:
On Wednesday, 25 February 2015 at 07:07:00 UTC, Ola Fosheim 
Grøstad wrote:

 On Wednesday, 25 February 2015 at 00:12:41 UTC, anonymous wrote:


[...]

 That sounds more attractive than the provided example, but the 
 right thing to do is to establish proper encapsulation. That 
 means you need a protection level that is stronger than 
 "private" that restricts "unsafe state" to a  trusted vetted 
 construct. Like unique_ptr informally does in C++.


I'm not knowledgeable enough to agree or disagree here.


 But why is malloc and free not considered safe by default 
 then?


 Well, because they aren't.


 So that should change?


We can't make malloc and free actually memory-safe, can we? We 
must not mark public unsafe functions  safe/ trusted.


 You mean outside RCArray, iff RCArray as a whole is manually 
 verified?  But that would surely mean that the  trusted region 
 is RCArray and neither the constructor or malloc/free?


RCArray as a whole is the actually trusted region, yes, since it 
must be manually verified that RCArray.array isn't leaked. But 
you can't mark it  trusted, because E may be unsafe.

The  trusted malloc/free dance is a mean hack to solve a problem. 
There may be other solutions that don't require breaking 
 trusted. Those may be better.


 And that assumes strong typing, which D currently does not 
 provide. Without strong typing it will be very difficult for 
 the compiler to infer anything across compilation units.


I don't follow.
Feb 25 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 25 February 2015 at 18:58:13 UTC, anonymous wrote:

 We can't make malloc and free actually memory-safe, can we? We 
 must not mark public unsafe functions  safe/ trusted.


My point was that there is no conceptual difference between 
having a named function trusted_malloc!int() and trusted_free() 
and wrapping them up individually unnamed.


 RCArray as a whole is the actually trusted region, yes, since 
 it must be manually verified that RCArray.array isn't leaked. 
 But you can't mark it  trusted, because E may be unsafe.


But the semantic analysis should verify that code isn't injected 
unless it is also  trusted?


 And that assumes strong typing, which D currently does not 
 provide. Without strong typing it will be very difficult for 
 the compiler to infer anything across compilation units.


 I don't follow.


C is not strongly typed, and neither is D. That means there are 
holes in the type system.
Feb 25 2015
parent reply "anonymous" <anonymous example.com> writes:
On Wednesday, 25 February 2015 at 22:16:14 UTC, Ola Fosheim 
Grøstad wrote:

 My point was that there is no conceptual difference between 
 having a named function trusted_malloc!int() and trusted_free() 
 and wrapping them up individually unnamed.


An ad-hoc declared  trusted malloc is just as unsafe as a public 
one, of course. But there's a difference in exposure. People 
working on RCArray are supposed to know about the rule-breaking 
that's going on there. A public trusted_malloc would invite the 
un-initiated to shoot their feet.


 RCArray as a whole is the actually trusted region, yes, since 
 it must be manually verified that RCArray.array isn't leaked. 
 But you can't mark it  trusted, because E may be unsafe.


 But the semantic analysis should verify that code isn't 
 injected unless it is also  trusted?


You mean the compiler should enforce E to be  safe/ trusted? That 
wouldn't happen with an  trusted RCArray, because  trusted code 
may call  system code.

It would be done with an  safe RCArray. But I guess that's deemed 
too limiting. RCArray is supposed to work with unsafe E types, 
too.


 And that assumes strong typing, which D currently does not 
 provide. Without strong typing it will be very difficult for 
 the compiler to infer anything across compilation units.


 I don't follow.


 C is not strongly typed, and neither is D. That means there are 
 holes in the type system.


I'm a bit lost. What I meant was that the compiler infers  safe 
for methods of templated structs when they don't call any  system 
code.

Here: RCArray's this/~this are inferred  safe iff E's 
__postblit/__dtor are  safe/ trusted.

This is how it works right now, regardless of any holes in the 
type system.
Feb 25 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 25 February 2015 at 22:59:01 UTC, anonymous wrote:

 rule-breaking that's going on there. A public trusted_malloc 
 would invite the un-initiated to shoot their feet.


That's entirely social...


 You mean the compiler should enforce E to be  safe/ trusted? 
 That wouldn't happen with an  trusted RCArray, because  trusted 
 code may call  system code.


If you can inject new code that is not marked  trusted explicitly 
into already written code that is marked  trusted, then the 
"trust type system" is broken.


 I'm a bit lost. What I meant was that the compiler infers  safe 
 for methods of templated structs when they don't call any 
  system code.


It infers " safe", but it does not provide validation.
Feb 26 2015
parent reply "anonymous" <anonymous example.com> writes:
On Thursday, 26 February 2015 at 10:15:07 UTC, Ola Fosheim 
Grøstad wrote:

 On Wednesday, 25 February 2015 at 22:59:01 UTC, anonymous wrote:

 rule-breaking that's going on there. A public trusted_malloc 
 would invite the un-initiated to shoot their feet.


 That's entirely social...


Sure.

A  trusted function that's not actually memory-safe is against 
the definition of  trusted. Breaking that rule on a public 
function is not acceptable. Breaking it locally is apparently ok 
as long as the surrounding entity (RCArray) is actually 
memory-safe in the end.

I'm not trying to argue that this is good. Maybe the current 
system allows for a better way to solve such issues. If it 
doesn't, maybe the  safe/ trusted system needs an upgrade.

[...]

 If you can inject new code that is not marked  trusted 
 explicitly into already written code that is marked  trusted, 
 then the "trust type system" is broken.


The whole point of  trusted is to be able to call  system code. 
It doesn't matter if that code is "injected" or not.  safe 
prevents calling  system code.

[...]

 It infers " safe", but it does not provide validation.


Yup. RCArray is de-facto trusted, i.e. verified memory-safe by 
the programmer. It's not compiler verified safe, even though it 
ends up being labeled  safe.
Feb 26 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Thursday, 26 February 2015 at 12:50:04 UTC, anonymous wrote:

 The whole point of  trusted is to be able to call  system code. 
 It doesn't matter if that code is "injected" or not.  safe 
 prevents calling  system code.


But it should matter, because when you mark a unit  trusted you 
basically are signing off a "certificate" that says it acts like 
 safe in  safe code. How can you verify anything if you allow 
injections? You can only verify what is known when the 
verification took place. After that it should be frozen so that 
injections of non-verified code only goes into  safe "slots".


 [...]

 It infers " safe", but it does not provide validation.


 Yup. RCArray is de-facto trusted, i.e. verified memory-safe by 
 the programmer. It's not compiler verified safe, even though it 
 ends up being labeled  safe.


Well, but  safe code is not verified either... It is inferred 
 safe based on a fixed set of criterions, but not verified. To 
verify you need more, and you have to start with strong typing.
Feb 26 2015
next sibling parent "anonymous" <anonymous example.com> writes:
On Thursday, 26 February 2015 at 20:56:52 UTC, Ola Fosheim 
Grøstad wrote:

 But it should matter, because when you mark a unit  trusted you 
 basically are signing off a "certificate" that says it acts 
 like  safe in  safe code. How can you verify anything if you 
 allow injections?


If you allow  system injections, you can't verify safety, 
obviously. Such code must not be  trusted. That's why RCArray is 
not  trusted.


 You can only verify what is known when the verification took 
 place. After that it should be frozen so that injections of 
 non-verified code only goes into  safe "slots".


I don't know if it "should" work like that. I don't have a good 
enough understanding of the matter to argue for or against any 
one design.

[...]

 Well, but  safe code is not verified either... It is inferred 
  safe based on a fixed set of criterions, but not verified. To 
 verify you need more, and you have to start with strong typing.


The criterion being: doesn't call  system code. If that's not 
enough to ensure memory-safety (given proper usage of  trusted) 
then I guess the current system has failed, or the implementation 
has bugs.

I don't think that the ability to break the type system (as done 
in RCArray) means that  safe has failed, though. We can 
forcefully break the type system in many ways. That doesn't mean 
it's not sound.

I'm probably not the guy to discuss "weak" and "strong" typing 
with. As I don't have a good understanding of type system theory.
Feb 26 2015
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 26 February 2015 at 20:56:52 UTC, Ola Fosheim 
Grøstad wrote:

 Well, but  safe code is not verified either... It is inferred 
  safe based on a fixed set of criterions, but not verified. To 
 verify you need more, and you have to start with strong typing.


 safe is supposed to provide safety, if you can give an example 
when it doesn't, you can report a bug. There are indeed bugs in 
implementation of safety, like escaping of local variables, but 
they are supposed to be fixed eventually.
Feb 27 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 27 February 2015 at 08:34:24 UTC, Kagamin wrote:

 safe is supposed to provide safety, if you can give an example
 when it doesn't, you can report a bug. There are indeed bugs in 
 implementation of safety, like escaping of local variables, but 
 they are supposed to be fixed eventually.


The bug is in using an approach that does not use proofs.
Feb 27 2015
parent reply "Kagamin" <spam here.lot> writes:
If you can't give an example of unsafety easily, that's already 
quite important. Compare to C, where one can provide such an 
example easily. If you want to write a mathematical prover, that 
won't hurt, though such tools don't need language support, lints 
and provers were written even for C.
Feb 27 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 27 February 2015 at 09:33:43 UTC, Kagamin wrote:

 If you can't give an example of unsafety easily, that's already 
 quite important. Compare to C, where one can provide such an 
 example easily.


Yes, that is true. Also, if you are conservative in C++ you also 
get pretty good safety with unique_ptr etc. So weak memory safety 
is ok, but then I also think it is ok to provide  trusted 
convenience since you are already saying that the programmer is 
competent:

unsafe_malloc!T()
unsafe_free!T()
unsafe_memmove!T()
unsafe_mmap…


 If you want to write a mathematical prover, that won't hurt, 
 though such tools don't need language support, lints and 
 provers were written even for C.


Yep. But what I meant is that a type system (including memory 
safety) ought to be proven sound. I.e:

1. You construct a simple language/type-system P and prove that P 
is sound and safe.
2. You construct a transform T(x) that can transform language D 
into x.

=> D is proven safe.
Feb 27 2015
next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 27 February 2015 at 10:49:25 UTC, Ola Fosheim Grøstad 
wrote:

 2. You construct a transform T(x) that can transform language D 
 into x.

 => D is proven safe.


Eh:

2. You construct a transform T(x) that can transform programs in 
language D into P...
Feb 27 2015
prev sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 27 February 2015 at 10:49:25 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 27 February 2015 at 09:33:43 UTC, Kagamin wrote:

 If you can't give an example of unsafety easily, that's 
 already quite important. Compare to C, where one can provide 
 such an example easily.


 Yes, that is true. Also, if you are conservative in C++ you 
 also get pretty good safety with unique_ptr etc. So weak memory 
 safety is ok, but then I also think it is ok to provide 
  trusted convenience since you are already saying that the 
 programmer is competent:

 unsafe_malloc!T()
 unsafe_free!T()
 unsafe_memmove!T()
 unsafe_mmap…


The programmer, who wrote the trusted code is competent... or 
rather was competent, when he was writing the code. The problem 
is in cognitive load, not in competence. 
http://dlang.org/safed.html - did you read this?


 If you want to write a mathematical prover, that won't hurt, 
 though such tools don't need language support, lints and 
 provers were written even for C.


 Yep. But what I meant is that a type system (including memory 
 safety) ought to be proven sound. I.e:

 1. You construct a simple language/type-system P and prove that 
 P is sound and safe.
 2. You construct a transform T(x) that can transform language D 
 into x.

 => D is proven safe.


The compiler doesn't prove the type system, only enforces it.
Mar 02 2015
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
It's not only code itself to manage here, but also its evolution: 
http://forum.dlang.org/post/mbaksa$1ers$1 digitalmars.com
That's why it was proposed to change trusted so that it won't 
allow implicit usage of system code.
Feb 25 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 25 February 2015 at 09:49:41 UTC, Kagamin wrote:

 It's not only code itself to manage here, but also its 
 evolution: 
 http://forum.dlang.org/post/mbaksa$1ers$1 digitalmars.com
 That's why it was proposed to change trusted so that it won't 
 allow implicit usage of system code.


But that will make things worse, not better, since the major 
issue is a lack of tools and process:

http://forum.dlang.org/thread/mb0uvr$2fdb$1 digitalmars.com?page=13#post-pqcattbwjsaqoekisrrc:40forum.dlang.org
Feb 25 2015
parent "Kagamin" <spam here.lot> writes:
It improves things without tools. Tools are always welcome, e.g. 
dfix already does something.
Feb 25 2015
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 2/24/15 5:37 PM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= 
<ola.fosheim.grostad+dlang gmail.com>" wrote:

 Since neither Andrei or Walter are able to say something sensible on
 these issues when asked, I apparently need to learn something about the
 "consistency" of C memory safety.

 I'm happy to listen to anyone who can explain this to me:

 1. My understanding is that  trusted is supposed to give memory safety
 escapes by providing a context which reestablish a memory safety context
 on return.

 Yet in this thread
 http://forum.dlang.org/thread/mcik3j$153g$1 digitalmars.com it is stated
 that this paradigm is an example of «careful use of  trusted»:

      count = (()  trusted => cast(uint*) malloc(uint.sizeof))();
      …arbitrary code…
      (()  trusted => free(count))();


This isn't exactly what is happening.

First, malloc should be safe, in the same way new is safe. The fact that 
 trusted is needed is somewhat incorrect in my opinion.

Where  trusted SHOULD be needed is for `free`.

Now, I disagree that using a  trusted delegate for the free is the right 
thing to do. As soon as you free count, it is no longer safe, because 
it's a dangling pointer.

I would say THIS is somewhat correct:

(()  trusted {free(count); count=null;})();

This takes something that is validly safe, and keeps the safety by 
ensuring the pointer is no longer dangling.

However, we have an issue here. At any point inside the code, you could do:

oldcount = count;

And now, there is still potentially a dangling pointer somewhere. This 
means every place count is used must be checked. In this case, all uses 
of count have to be re-checked when the file is edited.

Because count leaks the memory details of the trusted call to free (In 
other words, the fact that count, or any copy of it, can become a 
dangling pointer), I believe every call in that type that deals with 
count should be marked trusted.

This is a great example of why  trusted is mostly a convention thing, 
and not an enforceable thing. And it's very difficult to get right. 
There is no way to say "compiler, count is tainted. Please error 
whenever anyone uses count without marking that code as  trusted."


 They way I see it, this is equivalent to typing a reinterpret_casting
 malloc and free as memorysafe operations in isolation, basically
 providing a malloc!int() and free() as memory safe functions. But why is
 malloc and free not considered safe by default then? These  trusted
 functions clearly cannot prevent leaks within their own context. You
 would need a  trusted-only storage class on the receiving pointer to do
 that and a  trusted move type.

 If this is careful use of  trusted, then I don't see the point of having
  trusted at all. What is the purpose? What is it meant to cover? In
 order for  trusted to make sense in this code segment (
 http://dpaste.dzfl.pl/f3d854feede9 ) I would say that the whole class
 will have to be marked  trusted. Is that possible?


I agree. I think it's possible, like this:

struct RCArray(E) {
 trusted:

But it doesn't help with mechanical checking --  trusted is still 
basically more dangerous  system code.

-Steve
Feb 26 2015
next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Thursday, 26 February 2015 at 16:25:59 UTC, Steven 
Schveighoffer wrote:

 First, malloc should be safe, in the same way new is safe.


If it is typed and do the sizeof...


 I would say THIS is somewhat correct:

 (()  trusted {free(count); count=null;})();

 This takes something that is validly safe, and keeps the safety 
 by ensuring the pointer is no longer dangling.

 However, we have an issue here. At any point inside the code, 
 you could do:

 oldcount = count;

 And now, there is still potentially a dangling pointer 
 somewhere. This means every place count is used must be 
 checked. In this case, all uses of count have to be re-checked 
 when the file is edited.


Yes, so count should only be accessible directly from  trusted. 
So you need to mark it " trusted-only".


 This is a great example of why  trusted is mostly a convention 
 thing, and not an enforceable thing.


Well, it would be enforceable if you had a proper type system 
built around it.


 I agree. I think it's possible, like this:

 struct RCArray(E) {
  trusted:

 But it doesn't help with mechanical checking --  trusted is 
 still basically more dangerous  system code.


What you need is proper encapsulation. So that "dangerous state" 
and actions on that "dangerous state" is tied together.

And a strong type system.
Feb 26 2015
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 2/26/15 3:49 PM, "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= 
<ola.fosheim.grostad+dlang gmail.com>" wrote:

 On Thursday, 26 February 2015 at 16:25:59 UTC, Steven Schveighoffer wrote:

 First, malloc should be safe, in the same way new is safe.


 If it is typed and do the sizeof...


Right, one can easily make a  safe wrapper around malloc as long as free 
is never called :)


 I would say THIS is somewhat correct:

 (()  trusted {free(count); count=null;})();

 This takes something that is validly safe, and keeps the safety by
 ensuring the pointer is no longer dangling.

 However, we have an issue here. At any point inside the code, you
 could do:

 oldcount = count;

 And now, there is still potentially a dangling pointer somewhere. This
 means every place count is used must be checked. In this case, all
 uses of count have to be re-checked when the file is edited.


 Yes, so count should only be accessible directly from  trusted. So you
 need to mark it " trusted-only".


Right, the lure of  trusted delegates is that it does not take into 
account the leaking of memory details. You essentially make any  safe 
code that uses the same data as  trusted code now  trusted, but without 
a marking. It was the reason I was suggesting at one point that  trusted 
should internally mark variables as  trusted accessed so they could only 
be used in  trusted code.

Even with the suggestion I had above, the danger is still there for 
maintenance to this code to foil any guarantees made on the previous 
version.


 This is a great example of why  trusted is mostly a convention thing,
 and not an enforceable thing.


 Well, it would be enforceable if you had a proper type system built
 around it.


I'm not sure it's worth it. For something like RCSlice, it might just be 
good enough to mark the whole thing as  trusted, so the danger signs are 
put into the correct place. This is going to be a low-level primitive, 
that hopefully is fully encapsulated.

But of course, this is still by convention. I agree that any mechanical 
checking of  trusted is going to be impossible without a type notation 
for the data it touches.

I think one thing is for certain that I have learned through the 
discussions about  trusted -- we should avoid  trusted as much as 
possible in all code.

-Steve
Feb 26 2015
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 26 February 2015 at 16:25:59 UTC, Steven 
Schveighoffer wrote:

 However, we have an issue here. At any point inside the code, 
 you could do:

 oldcount = count;

 And now, there is still potentially a dangling pointer 
 somewhere. This means every place count is used must be 
 checked. In this case, all uses of count have to be re-checked 
 when the file is edited.

 Because count leaks the memory details of the trusted call to 
 free (In other words, the fact that count, or any copy of it, 
 can become a dangling pointer), I believe every call in that 
 type that deals with count should be marked trusted.


The counter is freed in the destructor, nothing can happen after 
that.
Feb 27 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 2/27/15 3:29 AM, Kagamin wrote:

 On Thursday, 26 February 2015 at 16:25:59 UTC, Steven Schveighoffer wrote:

 However, we have an issue here. At any point inside the code, you
 could do:

 oldcount = count;

 And now, there is still potentially a dangling pointer somewhere. This
 means every place count is used must be checked. In this case, all
 uses of count have to be re-checked when the file is edited.

 Because count leaks the memory details of the trusted call to free (In
 other words, the fact that count, or any copy of it, can become a
 dangling pointer), I believe every call in that type that deals with
 count should be marked trusted.


 The counter is freed in the destructor, nothing can happen after that.


So the code is now etched in stone and cannot be changed? Is there an 
attribute for that? :P

-Steve
Feb 27 2015
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 27 February 2015 at 14:52:56 UTC, Steven Schveighoffer 
wrote:

 The counter is freed in the destructor, nothing can happen 
 after that.


 So the code is now etched in stone and cannot be changed? Is 
 there an attribute for that? :P


Changes introduces in the destructor shouldn't affect other code, 
because nothing can happen after the destructor.
Mar 02 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/2/15 3:38 AM, Kagamin wrote:

 On Friday, 27 February 2015 at 14:52:56 UTC, Steven Schveighoffer wrote:

 The counter is freed in the destructor, nothing can happen after that.


 So the code is now etched in stone and cannot be changed? Is there an
 attribute for that? :P


 Changes introduces in the destructor shouldn't affect other code,
 because nothing can happen after the destructor.


Then you didn't grok my point:

(() trusted => count = new int)();

...

someOtherMember = count; // no  trusted needed here!

...

auto x = rcarray.someOtherMember; // oops, details leaked

...

~this() { ... /* destroy count via trusted */ }

...

*x += 1; // oops, still have a dangling pointer, and this can be  safe

The point is that, EVERY change to the  safe code inside RCArray has to 
be reviewed with this consideration (i.e. see if it makes a copy of 
count). And that means  safe code needs to be reviewed for safety -- 
something Walter does not want.

-Steve
Mar 02 2015
next sibling parent "Kagamin" <spam here.lot> writes:
Hmm... that means in OOP class is a unit of safety instead of a 
method, but it also applies to free methods, which access static 
members.
Mar 03 2015
prev sibling parent reply "Kagamin" <spam here.lot> writes:
If one wants to prevent a leak, then counter can be wrapped
---
struct Unsafe(T)
{
    private T _payload;
    T payload()  system { return _payload; }
    alias payload this;
}
---
And somehow disallow Unsafe template in safe function signatures, 
then having
Unsafe!(int*) _counter;
would be ok?
Mar 03 2015
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/3/15 9:28 AM, Kagamin wrote:

 If one wants to prevent a leak, then counter can be wrapped
 ---
 struct Unsafe(T)
 {
     private T _payload;
     T payload()  system { return _payload; }
     alias payload this;
 }
 ---
 And somehow disallow Unsafe template in safe function signatures, then
 having
 Unsafe!(int*) _counter;
 would be ok?


This is a pretty good idea actually. It's not foolproof (you still have 
to opt-in to this, the compiler doesn't force it), but it gives you a 
tool to ensure people aren't using it in an unsafe way without having to 
double-check every usage. I like it.

-Steve
Mar 03 2015
next sibling parent "Kagamin" <spam here.lot> writes:
Being a safety measure, it becomes trusted code's responsibility 
to provide this safety. BTW it also needs  system postblit; meh, 
I hope it's enough to make untouchable.
Mar 03 2015
prev sibling parent "Kagamin" <spam here.lot> writes:
Unsafe!(int*)* _c;

class A
{
   Unsafe!(int*) _counter;
   void escape()  safe { _c = &_counter; }
}

Not sure if it's legal. It should be really untouchable.
Mar 03 2015